How the CPU and GPU work together

In one line

The CPU is in charge: it starts the machine, runs the operating system, loads the data, copies it to the GPU and tells the GPU what to do; the GPU does the big parallel maths and hands the results back.

Real-life analogy

Diagram

Step by step

You know the CPU and the GPU are good at different things. But who decides what happens when? Let’s follow a GPU job from power-on to result.

Step 1: the CPU boots the machine

Remember lesson 1.5? When you press the power button, the CPU runs the firmware, the bootloader, and then the operating system. The GPU plays no part in deciding any of this.

Step 2: the CPU loads the data

A program that wants to use the GPU, for example an AI server, starts on the CPU. The CPU reads the data it needs from storage into RAM.

Step 3: the CPU copies data to the GPU

The CPU then copies the data from RAM into the GPU’s VRAM (lesson 3.3).

Step 4: the CPU launches the work

Now the CPU tells the GPU what to do. In NVIDIA’s CUDA system, this means launching a kernel: a function that the GPU will run with a huge number of workers at once, each on its own piece of data.

(Careful: in lesson 1.5, “kernel” meant the core of the operating system. In GPU programming, it means a function the GPU runs in parallel. Same word, different meaning.)

Step 5: the GPU computes

The GPU’s workers all run the kernel, each on their own small part of the data, all at the same time.

Step 6: the CPU collects the results

When the GPU is done, the CPU copies the results back (or tells the GPU to keep them for the next step), and does whatever comes next: for example, sending the answer back to your phone over the Internet.

Why can’t we use a GPU alone?

If the GPU is so powerful, why do we still need a CPU?

• Someone has to start the machine. Booting and running the operating system are step-by-step jobs, full of decisions. That is the CPU’s strength.
• Programs start on the CPU. In the way GPUs are programmed today, an application begins on the CPU and hands work to the GPU.
• Most work is not a big parallel job. Reading files, talking to the network, handling a request, deciding what to do next: these are serial, decision-heavy jobs, where a strong CPU core is faster than a GPU worker.
• The GPU waits to be told. It does not decide what to do by itself; it runs the work it is given.

So the GPU is a powerful specialist, and the CPU is the manager that keeps everything running.

Key takeaways

• The CPU boots the computer, runs the operating system and starts every program.
• The CPU loads data, copies it to VRAM, and launches kernels on the GPU.
• The GPU runs the parallel work; the CPU then collects the results.
• A GPU cannot work alone: it needs the CPU to start the machine and give it work.

Recap: the head chef opens the restaurant, reads the orders, carries ingredients to the prep room and gives the helpers one clear job. The helpers do it all at once, and the chef sends the plates out.

Now you understand how the CPU and GPU share the work, and why the CPU is still the boss. Next, the final lesson in this module: why AI fits GPUs so well.