Writing Your First CUDA Kernel: A Simple Vector Addition 🎯

Executive Summary ✨

This comprehensive guide walks you through writing your first CUDA kernel for performing vector addition. We’ll demystify the process, explaining each step with clear explanations and code examples. You’ll learn how to allocate memory on the GPU, transfer data, define and launch a CUDA kernel, and retrieve the results. This foundational example unlocks the power of parallel processing, significantly accelerating computational tasks. This is essential knowledge for anyone seeking to leverage the capabilities of NVIDIA GPUs for high-performance computing, and this will get you started with CUDA kernel vector addition.

Ready to dive into the world of parallel computing? CUDA, NVIDIA’s parallel computing architecture, offers a powerful way to accelerate tasks by harnessing the massive processing power of GPUs. This tutorial guides you through the process of writing your first CUDA kernel – a simple vector addition. We’ll break down each step, from setting up your environment to launching the kernel and retrieving the results. Get ready to unlock the potential of parallel processing!

Understanding CUDA Architecture

CUDA (Compute Unified Device Architecture) is a parallel computing platform and programming model developed by NVIDIA. It allows software developers to use a CUDA-enabled GPU for general purpose processing – GPGPU. Before writing a CUDA kernel, it’s helpful to grasp some core concepts:

• Host and Device: The host refers to your CPU and system memory (RAM), while the device is the GPU and its memory.
• Kernels: Kernels are the functions executed on the GPU. They are the heart of CUDA programs.
• Threads, Blocks, and Grids: CUDA organizes threads into blocks and blocks into grids. This hierarchy allows for scalable parallel execution. A grid contains multiple blocks, and each block contains multiple threads.
• Memory Hierarchy: CUDA provides different types of memory (global, shared, constant, registers), each with its own characteristics in terms of speed and scope.

Setting Up Your CUDA Development Environment

Before you can start writing CUDA kernels, you need to set up your development environment. This typically involves installing the CUDA Toolkit, which includes the necessary compilers, libraries, and tools.
The specific steps may vary depending on your operating system, but generally it involves:

• Installing the NVIDIA Driver: Ensure you have the latest NVIDIA drivers installed for your GPU.
• Downloading the CUDA Toolkit: Download the CUDA Toolkit from the NVIDIA website (developer.nvidia.com). Make sure to choose the version compatible with your operating system and GPU.
• Setting Environment Variables: Configure your system’s environment variables (e.g., PATH, CUDA_PATH) to point to the CUDA Toolkit installation directory.
• Verifying Installation: Use the `nvcc –version` command to verify that the CUDA compiler is installed correctly.

Writing the CUDA Kernel for Vector Addition 💡

Now comes the exciting part: writing the CUDA kernel! A CUDA kernel is a function that is executed in parallel by multiple threads on the GPU. Here’s how we implement CUDA kernel vector addition:

First, define the kernel function in a `.cu` file. The `__global__` keyword indicates that this function will be executed on the device (GPU) and called from the host (CPU):

    __global__ void vectorAdd(float *a, float *b, float *c, int n) {
        int i = blockIdx.x * blockDim.x + threadIdx.x;
        if (i < n) {
            c[i] = a[i] + b[i];
        }
    }
    

• `__global__` Keyword: Marks the function as a CUDA kernel, executable on the GPU.
• `blockIdx.x`, `blockDim.x`, `threadIdx.x`: These built-in variables provide the global thread index within the grid. `blockIdx.x` is the block index, `blockDim.x` is the block dimension (number of threads per block), and `threadIdx.x` is the thread index within the block.
• `if (i < n)`: This check ensures that the thread does not access memory beyond the bounds of the vectors.
• `c[i] = a[i] + b[i]`:** The core vector addition operation, performed in parallel by each thread.

    #include 
    #include 

    int main() {
        int n = 1024; // Size of the vectors
        float *h_a, *h_b, *h_c, *d_a, *d_b, *d_c; // Host and device pointers

        // Allocate memory on the host
        h_a = (float*)malloc(n * sizeof(float));
        h_b = (float*)malloc(n * sizeof(float));
        h_c = (float*)malloc(n * sizeof(float));

        // Initialize host vectors (example)
        for (int i = 0; i < n; i++) {
            h_a[i] = (float)i;
            h_b[i] = (float)(n - i);
        }

        // Allocate memory on the device
        cudaMalloc((void**)&d_a, n * sizeof(float));
        cudaMalloc((void**)&d_b, n * sizeof(float));
        cudaMalloc((void**)&d_c, n * sizeof(float));

        // Copy data from host to device
        cudaMemcpy(d_a, h_a, n * sizeof(float), cudaMemcpyHostToDevice);
        cudaMemcpy(d_b, h_b, n * sizeof(float), cudaMemcpyHostToDevice);

        // ... (Kernel launch and data retrieval - see next section) ...

        // Free memory (Important!)
        free(h_a);
        free(h_b);
        free(h_c);
        cudaFree(d_a);
        cudaFree(d_b);
        cudaFree(d_c);

        return 0;
    }
    

    // Define block and grid dimensions
    int blockSize = 256;
    int numBlocks = (n + blockSize - 1) / blockSize; // Calculate the number of blocks needed

    // Launch the kernel
    vectorAdd<<>>(d_a, d_b, d_c, n);

    // Copy the result from device to host
    cudaMemcpy(h_c, d_c, n * sizeof(float), cudaMemcpyDeviceToHost);

    // Verify the results (example)
    for (int i = 0; i  1e-5) {
            std::cout << "Error at index " << i << ": " << h_c[i] << " != " << (h_a[i] + h_b[i]) << std::endl;
            break;
        }
    }

    std::cout << "Vector addition complete and verified!" << std::endl;