naveenopslearn
Search
Course outline
  1. 01How a computer works
    1. 1.1What is a computer?(completed)
    2. 1.2The CPU: the brain(completed)
    3. 1.3Memory: RAM vs storage(completed)
    4. 1.4How the CPU and memory work together(completed)
    5. 1.5What happens when you turn on a computer(completed)
  2. 02How the Internet works
    1. 2.1What is the Internet?(completed)
    2. 2.2Addresses and names: IP and DNS(completed)
    3. 2.3Packets: how information travels(completed)
    4. 2.4Client and server: opening a website(completed)
    5. 2.5Inside a busy server(completed)
    6. 2.6Why this matters for AI(completed)
  3. 03Introducing the GPU
    1. 3.1What is a GPU?(completed)
    2. 3.2CPU vs GPU(completed)
    3. 3.3The GPU's own memory: VRAM(completed)
    4. 3.4How the CPU and GPU work together(completed)
    5. 3.5Why GPUs suit AI(completed)
  4. 04Tensors and models
    1. 4.1What is a tensor?soon
    2. 4.2What is an AI model?soon
    3. 4.3Training vs inferencesoon
  5. 05Splitting work across GPUs
    1. 5.1Why split work across GPUs?soon
    2. 5.2Data parallelismsoon
    3. 5.3Tensor parallelismsoon
    4. 5.4Pipeline parallelismsoon
    5. 5.5Expert parallelismsoon
    6. 5.6Which to use whensoon
  6. 06Serving LLMs with vLLM
    1. 6.1What is LLM serving?soon
    2. 6.2Meet vLLMsoon
    3. 6.3Serving a model with vLLMsoon

Lesson 3.1

What is a GPU?

A chip built to do many simple calculations at the same time.

beginner 3 min readChecked

In one line

A GPU (graphics processing unit) is a chip with a huge number of simple workers that do the same kind of small calculation on lots of data at the same time.

Real-life analogy

Our restaurant gets an order for a wedding of 2,000 guests. Every plate needs the same thing: a slice of cake with a strawberry on top.

The head chef (CPU) could do it alone, plate by plate. It would take for ever.

Instead, the restaurant brings in a huge team of helpers. Each helper does one simple job, “put a strawberry on this plate”, and they all do it at once. The whole order is ready in moments.

That team of helpers is the GPU.

Where the comparison stops working: kitchen helpers can each think for themselves. GPU workers are much simpler: they work in groups that follow the same instructions together, each on its own piece of data. Give them a job that is not like that, full of “if this, then that” decisions, and they are not much help.

Diagram

ONE BIG JOB: COLOUR EVERY PIXELmillions of pixels,each needing thesame small sumTHE GPU: MANY SIMPLE WORKERSEach small square is one worker doing the same simple step ona different piece of data, all at the same time. (Real GPUshave far more workers than shown here.)
A GPU splits one big job into many small, identical pieces and works on them all at once.

Step by step

In Module 2 we saw that AI requests end up at a server with a GPU. So what is this chip, and why is it different from the CPU?

It started with pictures

Look at your screen. It is made of millions of tiny dots called pixels. To show a moving game or video, the computer must work out the colour of every pixel, many times a second.

Here is the important part: the sum for one pixel is simple, and it is almost the same for every pixel. And pixel number 1 does not need to wait for pixel number 2.

That is the perfect job for many workers at once. So chip makers built a special chip for it: the graphics processing unit, or GPU.

Many simple workers

Where a CPU has a few strong cores, a GPU has a huge number of smaller, simpler ones, able to run thousands of small tasks at the same time.

Each GPU worker is slower and simpler than a CPU core. But because there are so many of them, a GPU can get through a pile of similar, independent calculations much faster than a CPU can.

Doing things “in parallel”

When many pieces of work happen at the same time, we call it parallel work. When they happen one after another, we call it serial work.

  • CPU: built to be very fast at serial work and quick decisions.
  • GPU: built for parallel work, where one big job splits into thousands or millions of small identical pieces.

Not just for graphics any more

People soon noticed that many other problems look like “the same small sum on lots of numbers”. Science simulations do. And, most importantly for us, AI does.

That is why GPUs are now used far beyond games: in supercomputers, data centres, robots, and the servers that answer your AI questions.

Key takeaways

  • A GPU is a chip with a huge number of simple workers.
  • It is great at parallel work: the same small calculation on lots of data, all at once.
  • It was first built for graphics, where every pixel needs a similar sum.
  • The same strength makes GPUs very useful for AI.

Recap: a wedding of 2,000 plates, and a big team of helpers each adding one strawberry at the same time. That is a GPU.

Now you understand what a GPU is and why it was invented. Next, we will put the CPU and GPU side by side and compare them properly.

Quick check

Pick an answer, then press Check answer. Nothing is recorded.

  1. What is a GPU especially good at?
  2. What was the GPU first designed for?
  3. In our restaurant, the GPU is...

Sources

This lesson was checked against these pages.

  1. What's the Difference Between a CPU and a GPU? · NVIDIA
  2. 1.1. Introduction (CUDA Programming Guide) · NVIDIA