A Painless CUDA Programming in Linux

1. Introduction

CUDA is a parallel computing platform that allows developers to use GPUs for general-purpose computing. It offers high performance compared to traditional CPUs for certain types of workloads, such as data processing and machine learning. In this article, we will explore how to perform painless CUDA programming in Linux. We will start with an introduction to CUDA and its architecture and then move on to installing and configuring CUDA on Linux. Finally, we will create a simple program that uses CUDA to accelerate matrix multiplication.

2. CUDA Architecture

Before diving into CUDA programming, it's important to understand the architecture of NVIDIA GPUs. A GPU consists of many cores that can perform calculations in parallel. Each core has its own instruction unit and is capable of executing independent instructions. GPUs also have specialized memory, such as shared and global memory, which can be utilized for faster data access. The architecture of a GPU allows for massive parallelism and fast data processing, which makes it ideal for data-intensive applications.

2.1 CUDA Programming Model

CUDA programming is based on a programming model that includes the following concepts:

Host: The CPU and its memory

Device: The GPU and its memory

Kernel: A function that runs on the GPU and processes data in parallel

Thread: The smallest unit of execution in a GPU kernel

Block: A group of threads that can communicate and synchronize with each other

Grid: A collection of blocks that execute a kernel

2.2 CUDA Libraries

NVIDIA provides several libraries that can be used with CUDA to speed up common tasks, such as linear algebra, fast Fourier transforms, and image processing. These libraries are written in CUDA C and optimized for maximum performance on NVIDIA GPUs. Some of the most popular CUDA libraries include cuBLAS, cuFFT, and cuDNN.

3. Installing CUDA on Linux

Before we can start CUDA programming, we need to install the CUDA toolkit on our Linux system. The following steps explain how to install CUDA 11.3 on Ubuntu 20.04:

sudo apt update
sudo apt upgrade
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/11.3.0/local_installers/cuda-repo-ubuntu2004-11-3-local_11.3.0-465.19.01-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu2004-11-3-local_11.3.0-465.19.01-1_amd64.deb
sudo apt-key add /var/cuda-repo-ubuntu2004-11-3-local/7fa2af80.pub
sudo apt-get update
sudo apt-get -y install cuda

After installing the CUDA toolkit, we need to set the environment variables to enable CUDA support in our programs. We can do this by adding the following lines to our .bashrc file:

export PATH=/usr/local/cuda-11.3/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-11.3/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

4. Writing a Simple CUDA Program

Now that we have CUDA installed on our system, let's write a simple program that uses CUDA to accelerate matrix multiplication. The following code shows how to allocate memory on the GPU, transfer data from the CPU to the GPU, perform the matrix multiplication on the GPU, and transfer the results back to the CPU:

#include <stdio.h>
#include <cuda.h>
#define N 1024
__global__ void matrix_mul(float *a, float *b, float *c, int n)
{
    int i = blockIdx.x * blockDim.x + threadIdx.x;
    int j = blockIdx.y * blockDim.y + threadIdx.y;
    if (i < n && j < n) {
        float sum = 0.0;
        for (int k = 0 ; k < n ; ++k)
            sum += a[i * n + k] * b[k * n + j];
        c[i * n + j] = sum;
    }
}
int main()
{
    float *h_a, *h_b, *h_c; // host arrays
    float *d_a, *d_b, *d_c; // device arrays
    int i, j;
    // allocate memory on the host
    h_a = (float*) malloc(N * N * sizeof(float));
    h_b = (float*) malloc(N * N * sizeof(float));
    h_c = (float*) malloc(N * N * sizeof(float));
    // init the matrices
    for (i = 0 ; i < N ; i++)
        for (j = 0 ; j < N ; j++)
            h_a[i * N + j] = h_b[i * N + j] = 1.0;
    // allocate memory on the device
    cudaMalloc((void**) &d_a, N * N * sizeof(float));
    cudaMalloc((void**) &d_b, N * N * sizeof(float));
    cudaMalloc((void**) &d_c, N * N * sizeof(float));
    // copy data to the device
    cudaMemcpy(d_a, h_a, N * N * sizeof(float), cudaMemcpyHostToDevice);
    cudaMemcpy(d_b, h_b, N * N * sizeof(float), cudaMemcpyHostToDevice);
    // launch the kernel
    int block_size = 16;
    dim3 dim_grid((N + block_size - 1) / block_size, (N + block_size - 1) / block_size);
    dim3 dim_block(block_size, block_size);
    matrix_mul<<<dim_grid, dim_block>>>(d_a, d_b, d_c, N);
    // copy data back to the host
    cudaMemcpy(h_c, d_c, N * N * sizeof(float), cudaMemcpyDeviceToHost);
    // verify the result
    for (i = 0 ; i < N ; i++)
        for (j = 0 ; j < N ; j++)
            if (h_c[i * N + j] != N)
                printf("Error: mismatch at (%d,%d) (%f != %d)\n", i, j, h_c[i * N + j], N);
    // free memory on the host and device
    free(h_a);
    free(h_b);
    free(h_c);
    cudaFree(d_a);
    cudaFree(d_b);
    cudaFree(d_c);
    return 0;
}

The above program initializes two matrices with all elements set to 1, allocates memory on the host and device, transfers the matrices to the device, performs matrix multiplication on the GPU using a kernel function, transfers the result back to the host, verifies the result, and frees the memory. The program uses the CUDA C syntax and APIs to leverage the GPU's parallel processing power.

Conclusion

CUDA programming can be challenging, but it offers significant performance benefits for certain types of workloads. In this article, we learned about the CUDA architecture and programming model, how to install CUDA on Linux, and how to write a simple program that uses CUDA to accelerate matrix multiplication. With the knowledge gained from this article, you should be able to start exploring the world of CUDA programming and optimizing your applications for maximum performance.