Why GPUs suit AI

In one line

AI models spend most of their time doing huge piles of simple multiply-and-add sums, arranged as matrix multiplication, and that is exactly the kind of work a GPU’s many workers do best.

Real-life analogy

Imagine the restaurant must work out the price of every dish on a giant menu for 1,000 different restaurants, each with its own ingredient prices.

For every dish at every restaurant, the sum is the same kind of thing: “amount of ingredient × its price, added up”. Each of those sums is easy, and none of them needs to wait for another.

Give one sum to each of your thousands of helpers and the whole menu is priced almost instantly. That is why AI loves GPUs.

Where the comparison stops working: pricing a menu is something you understand. An AI model’s numbers do not have an obvious meaning like “price” or “amount”; they are values the model learned during training. But the kind of sum is really the same: multiply, then add.

Diagram

Matrix multiplication: every answer box is an independent multiply-and-add job, perfect for many GPU workers at once.

Step by step

We now know what a GPU is good at. But what does an AI model actually do with all those workers? Let’s take a short peek at the maths. Don’t worry: it is only multiplying and adding.

Numbers in tables

An AI model works with numbers arranged in tables. A table of numbers is called a matrix (plural: matrices).

Your question is turned into numbers.
The model itself is mostly a huge collection of learned numbers (we will look at these properly in Module 4).

The key move: matrix multiplication

The most common thing an AI model does is multiply a table of input numbers by a table of the model’s numbers. This is called matrix multiplication.

Let’s do a tiny one, like the diagram:

Numbers in	Model’s numbers
2, 3 / 4, 5	1, 0 / 6, 2

Each answer box takes one row from the left and one column from the right, multiplies them pair by pair, and adds the results:

top-left: 2×1 + 3×6 = 20
top-right: 2×0 + 3×2 = 6
bottom-left: 4×1 + 5×6 = 34
bottom-right: 4×0 + 5×2 = 10

That is it. Multiply, then add. This “multiply-add” is the most common operation inside neural networks.

Why this is perfect for a GPU

Look again at the four answer boxes. Notice two things:

Every box is the same kind of job: multiply pairs and add them up.
No box needs to wait for another. You can work out the bottom-right before the top-left.

Those are exactly the two questions from lesson 3.2. Both answers are yes, so this is a GPU job.

Now scale it up. A real model multiplies tables with thousands of rows and columns, again and again. That means an enormous number of small, independent multiply-adds. Thousands of GPU workers can share them out and finish together. Many modern GPUs even have special parts (NVIDIA calls them Tensor Cores) built just to speed up this matrix maths.

And memory matters too

All those numbers have to be fed to the workers from VRAM. That is why, as we saw in lesson 3.3, VRAM size (does the model fit?) and VRAM speed (can we feed the workers fast enough?) matter so much for AI.

Where we go next

In Module 4 we will see what these tables of numbers really are: tensors and models. In Module 5 we will see what happens when a model is too big for one GPU and the work must be split across many GPUs.

Key takeaways

AI models work with tables of numbers called matrices.
Their most common operation is matrix multiplication, which is just lots of multiply-and-add sums.
Those sums are all alike and independent, which is exactly what GPUs are built for.
Feeding the numbers from VRAM quickly matters as much as doing the maths quickly.

Recap: price every dish on a giant menu by giving each simple “multiply, then add” sum to a different helper, all at once. That is AI on a GPU.

Now you understand why AI and GPUs are such a good match. That completes Module 3, and the foundations of this course: you now know how a computer works, how a request reaches a server, and why that server has a GPU.

Quick check

Pick an answer, then press Check answer. Nothing is recorded.

What is the most common kind of calculation inside an AI model?
Multiply two numbers and add the result to a running total (multiply-add)Sorting words alphabeticallyOpening files
Why does matrix multiplication suit a GPU?
Because each answer box can be worked out on its own, without waiting for the othersBecause it needs lots of decisionsBecause it only uses one number
Using the diagram, what is the top-left answer (2×1 + 3×6)?
112036

Answers:

Multiply two numbers and add the result to a running total (multiply-add): AI models are built from huge numbers of multiply-and-add steps, usually organised as matrix multiplications.

Because each answer box can be worked out on its own, without waiting for the others: Independent, identical small jobs are exactly what thousands of GPU workers are built for.

20: 2×1 = 2 and 3×6 = 18, so 2 + 18 = 20. That is one multiply-add job; a GPU does a huge number of these at once.