naveenopslearn
Search
Course outline
  1. 01How a computer works
    1. 1.1What is a computer?(completed)
    2. 1.2The CPU: the brain(completed)
    3. 1.3Memory: RAM vs storage(completed)
    4. 1.4How the CPU and memory work together(completed)
    5. 1.5What happens when you turn on a computer(completed)
  2. 02How the Internet works
    1. 2.1What is the Internet?(completed)
    2. 2.2Addresses and names: IP and DNS(completed)
    3. 2.3Packets: how information travels(completed)
    4. 2.4Client and server: opening a website(completed)
    5. 2.5Inside a busy server(completed)
    6. 2.6Why this matters for AI(completed)
  3. 03Introducing the GPU
    1. 3.1What is a GPU?(completed)
    2. 3.2CPU vs GPU(completed)
    3. 3.3The GPU's own memory: VRAM(completed)
    4. 3.4How the CPU and GPU work together(completed)
    5. 3.5Why GPUs suit AI(completed)
  4. 04Tensors and models
    1. 4.1What is a tensor?soon
    2. 4.2What is an AI model?soon
    3. 4.3Training vs inferencesoon
  5. 05Splitting work across GPUs
    1. 5.1Why split work across GPUs?soon
    2. 5.2Data parallelismsoon
    3. 5.3Tensor parallelismsoon
    4. 5.4Pipeline parallelismsoon
    5. 5.5Expert parallelismsoon
    6. 5.6Which to use whensoon
  6. 06Serving LLMs with vLLM
    1. 6.1What is LLM serving?soon
    2. 6.2Meet vLLMsoon
    3. 6.3Serving a model with vLLMsoon

Lesson 3.2

CPU vs GPU

A few strong workers versus many simple workers, and when each one wins.

beginner 3 min readChecked

In one line

A CPU has a few strong workers that are fast at step-by-step work; a GPU has many simple workers that are fast when the same work is done on lots of data at once.

Real-life analogy

Imagine two kitchens.

Kitchen A has 4 master chefs. Each one can cook any dish, handle a tricky special request, and change plans in a second. Perfect for a menu where every order is different.

Kitchen B has hundreds of helpers. Each one only knows a few simple moves, and they work in groups doing the same move together. Useless for a tricky one-off dish, but unbeatable when the order is “peel 10,000 potatoes”.

Kitchen A is the CPU. Kitchen B is the GPU.

Where the comparison stops working: in real kitchens, helpers chat and help each other freely. GPU workers mostly work alone on their own little piece, and sharing results between them takes care. Also, both “kitchens” usually sit inside the same computer and work as a team, which is the topic of lesson 3.4.

Diagram

CPU: A FEW STRONG WORKERSstrong corestrong corestrong corestrong coreFast at one task at a time.Great at decisions andstep-by-step work.GPU: MANY SIMPLE WORKERSEach worker is simpler, butthousands work at once. Greatat the same sum on lots of data.
CPU: a few strong cores for step-by-step work. GPU: many simple workers for the same work on lots of data.

Step by step

Which is better, a CPU or a GPU? That is the wrong question. The right question is: better at what?

Same chip space, different choices

Every chip has a limited amount of space for its tiny electronic parts. CPU and GPU makers spend that space differently:

  • A CPU spends a lot of its space on making a few cores very clever and very quick: big local memory (caches) and smart control parts that keep one line of work flowing fast, even when it is full of decisions.
  • A GPU spends most of its space on doing maths: a huge number of simpler workers, so it can run thousands of small tasks at the same time.

The comparison table

CPU GPU
Workers A few strong cores Many simple workers
Best at Step-by-step work, quick decisions The same work on lots of data at once
One task on its own Very fast Slower
Huge pile of similar tasks Slower Very fast
Typical jobs Running the operating system, opening apps, handling requests Graphics, science simulations, AI maths
Restaurant The head chef A big team of helpers

Speed of one vs speed of many

There are two different kinds of “fast”:

  • How quickly one task finishes. The CPU wins. If you need one answer right now, a strong core gets it quickly.
  • How much work gets done in total. For big piles of similar work, the GPU wins, because thousands of workers are going at once.

Engineers call the first one latency (how long one thing takes) and the second one throughput (how much gets done per second).

Which jobs suit which chip?

Ask two questions about a job:

  1. Can it be cut into many small pieces that do not need to wait for each other?
  2. Do all the pieces do roughly the same thing?

If the answer to both is yes, the job suits a GPU. If the work is a long chain of steps where each one depends on the last, or it is full of different decisions, it suits a CPU.

Key takeaways

  • CPU: a few strong cores, best at step-by-step work and quick decisions.
  • GPU: many simple workers, best at the same work on lots of data at once.
  • Latency is how long one task takes (CPU strength); throughput is how much gets done in total (GPU strength).
  • They are partners, not rivals.

Recap: four master chefs for a menu full of special requests; hundreds of helpers for 10,000 potatoes.

Now you understand the real difference between a CPU and a GPU, and how to guess which one suits a job. Next: the GPU’s own memory, called VRAM.

Quick check

Pick an answer, then press Check answer. Nothing is recorded.

  1. Which job suits a GPU better than a CPU?
  2. A CPU core is ___ than a GPU worker, but a GPU has ___ workers.
  3. Does a GPU replace the CPU?

Sources

This lesson was checked against these pages.

  1. What's the Difference Between a CPU and a GPU? · NVIDIA
  2. 1.1. Introduction (CUDA Programming Guide) · NVIDIA
  3. GPU Performance Background User's Guide · NVIDIA