2 minute(s) to read
GPU Programming
Introduction to GPU programming with Nvidia CUDA
(The illustrations used are from Pierre-François Bonnefoi’s course, source: https://p-fb.net/master1/gpgpu/cours/cuda.pdf)
About GPU Programming
GPU programming (or GPGPU—General-Purpose computing on Graphics Processing Units) aims to harness the computational power of graphics cards for tasks that are massively parallelizable.
While a CPU is primarily dedicated to handling rapid, sequential tasks, graphics cards offer a programming architecture that is highly conducive to parallelism. Nvidia pioneered this field; since 2007, the company has been developing a hardware interface and a C-based programming language known as CUDA (Compute Unified Device Architecture).
CUDA? How does it work? 🤔
With CUDA, the CPU architecture (referred to as the Host) is separated from the GPU architecture (referred to as the Device). The host has its own RAM containing its programs and variables, whereas the GPU is responsible only for executing the specific portion of the code assigned to it, known as the kernel. The two architectures exchange data via a PCI bus.
The CPU therefore retains control over both architectures:
- It allocates memory on the graphics card using the cudaMalloc instruction.
- The CPU copies the data to be transferred to the graphics card’s RAM using the cudaMemcpy instruction.
- When a kernel is called, the CPU passes the memory address that the GPU is to use during kernel execution.
- The CPU then retrieves the result produced by the kernel before deallocating the associated memory using the cudaFree instruction.
So, what does a GPU look like?
That’s not an easy question to answer, my friend!
In GPU programming:
- Your graphics card is divided into several multiprocessors, each containing multiple processors.
- All these multiprocessors share the same global memory (GDDR).
- Each multiprocessor has its own shared memory; processors within the same multiprocessor can exchange data here much faster than by going through global memory—though you have to watch out for concurrent access!
- Each processor has its own registers, though the capacity is quite small—on the order of 16 kilobytes.
In CUDA, memory is structured similarly, but hierarchically:
- At the lowest level, there is the thread, which has its own local memory.
- Next come blocks, composed of multiple threads executing concurrently and sharing access to a shared memory space.
- At the highest level, there are grids, composed of multiple thread blocks that utilize global memory.
There is also the concept of a warp within a block (a warp being a group of 32 synchronized threads); if you’d like to learn more, I encourage you to check out Pierre-François Bonnefoi’s course 😉: https://p-fb.net/master1/gpgpu/cours/cuda.pdf
Uses of GPU Programming
Naturally, the most well-known application of GPU programming is in the fields of video games and special effects.
However, it is also used in the scientific community: by providing greater computing power than a CPU, it enables simulations, analysis, and data processing to be performed in much more reasonable timeframes!
During my CRYPTIS Master’s program (and thus in the context of RSA cryptography), I developed CUDA programs capable of cracking the prime factorization of massive numbers. 😉
