naveenopslearn
Search
Course outline
  1. 01How a computer works
    1. 1.1What is a computer?(completed)
    2. 1.2The CPU: the brain(completed)
    3. 1.3Memory: RAM vs storage(completed)
    4. 1.4How the CPU and memory work together(completed)
    5. 1.5What happens when you turn on a computer(completed)
  2. 02How the Internet works
    1. 2.1What is the Internet?(completed)
    2. 2.2Addresses and names: IP and DNS(completed)
    3. 2.3Packets: how information travels(completed)
    4. 2.4Client and server: opening a website(completed)
    5. 2.5Inside a busy server(completed)
    6. 2.6Why this matters for AI(completed)
  3. 03Introducing the GPU
    1. 3.1What is a GPU?(completed)
    2. 3.2CPU vs GPU(completed)
    3. 3.3The GPU's own memory: VRAM(completed)
    4. 3.4How the CPU and GPU work together(completed)
    5. 3.5Why GPUs suit AI(completed)
  4. 04Tensors and models
    1. 4.1What is a tensor?soon
    2. 4.2What is an AI model?soon
    3. 4.3Training vs inferencesoon
  5. 05Splitting work across GPUs
    1. 5.1Why split work across GPUs?soon
    2. 5.2Data parallelismsoon
    3. 5.3Tensor parallelismsoon
    4. 5.4Pipeline parallelismsoon
    5. 5.5Expert parallelismsoon
    6. 5.6Which to use whensoon
  6. 06Serving LLMs with vLLM
    1. 6.1What is LLM serving?soon
    2. 6.2Meet vLLMsoon
    3. 6.3Serving a model with vLLMsoon

Lesson 2.6

Why this matters for AI

When you ask an AI a question, your request travels to a server that has a GPU.

beginner 3 min readChecked

In one line

When you ask an AI a question, it travels across the Internet as a normal request to a server, but that server has a GPU that runs the AI model.

Real-life analogy

So far, our restaurant has a few chefs. Now imagine a dish that needs thousands of tiny, identical jobs, like cutting ten thousand vegetables into cubes.

A few chefs could do it, but slowly. So the restaurant hires a huge team of kitchen helpers who each do one simple cut, all at the same time. That team is the GPU.

The order still arrives the same way, and the head chef (the CPU) still runs the kitchen. But the big repetitive job goes to the helpers.

Where the comparison stops working: kitchen helpers can think for themselves a little. A GPU’s many small workers cannot. They are very good at doing the same simple maths on lots of numbers at once, and not much else. We will explore this properly in Module 3.

Diagram

Your phone:"Explain GPUs"InternetAI SERVER IN A DATA CENTRECPU:receives andpreparesGPU:runs theAI modelthe answer travels back, often word by word
An AI request is a normal web request, but the server has a GPU that runs the AI model.

Step by step

Why did we spend a whole module on the Internet when this course is about AI? Because every time you use an AI app, everything in this module happens.

Step 1: you ask

You type “Explain GPUs simply” into an AI app on your phone. Your phone is the client.

Step 2: the request travels

Your phone looks up the AI service’s name with DNS, connects to its server, and sends your question as a request, cut into packets. Exactly like opening a website.

Step 3: the server receives it

The request arrives at a server in a data centre. Its CPU does the familiar jobs: receives the request, checks it, and gets your question ready for the model.

Step 4: the GPU runs the model

Here is the new part. The server also has one or more GPUs. The AI model (a huge collection of learned numbers, which we will meet in Module 4) is loaded onto the GPU, and the GPU does the heavy maths needed to produce an answer.

Using a trained model to answer a new question is called inference.

Step 5: the answer comes back

Text AI models usually produce their answer a small piece at a time. Many apps send those pieces back as soon as they are ready, which is why you often see the answer appear word by word.

Why busy servers matter even more for AI

Remember the busy restaurant from lesson 2.5? It is even more important here:

  • A GPU is expensive and there are only so many of them.
  • Each request needs memory on the GPU while it is being answered.
  • When too many people ask at once, requests wait in a queue or are turned away.

Making AI fast and affordable is largely about using GPUs well: fitting more requests onto each GPU, and spreading work across many GPUs. That is what the rest of this course builds towards.

Key takeaways

  • An AI request is a normal web request: DNS, packets, client and server.
  • The server’s CPU receives and prepares the request; its GPU runs the AI model.
  • Using a trained model to answer is called inference.
  • Busy-server problems (queues, memory, turning people away) matter even more with GPUs.

Recap: the order arrives as usual, the head chef runs the kitchen, and a huge team of helpers does the big repetitive job.

Now you understand how an AI request reaches a GPU, and why Module 2 matters for AI. That completes Module 2! Next, in Module 3, we finally open up the GPU.

Quick check

Pick an answer, then press Check answer. Nothing is recorded.

  1. When you type a question into an AI chat app, where is the answer usually worked out?
  2. Using a trained AI model to answer a new question is called...
  3. Which lessons from Module 2 still apply to an AI request?

Sources

This lesson was checked against these pages.

  1. What Is AI Inference? · NVIDIA
  2. How the web works · MDN Web Docs