COURSE 24Intermediate10 IN-DEPTH LESSONS

Hardware, Compute and Energy

The physical reality of AI: GPUs, TPUs, custom ASICs, memory bandwidth constraints, quantization, edge vs cloud inference, and datacenter energy footprint.

Start Lesson 1: Compute and accelerators →← All 29 Courses

Course Syllabus & Units

Unit 1

Overview

1 lessons
1.1L1

Compute and accelerators

**Compute** is the work hardware does to train and run models. **Accelerators** (GPUs, TPUs, Trainium, Instinct MI300-class devices, etc.) speed the linear algebra AI relies on. Google’s TPU docs contrast CPUs (flexible, memory-bottlenec...

Read lesson →
Unit 2

Accelerators

2 lessons
2.1L1

GPUs, TPUs, and training compute

**GPUs** and **TPUs** (and similar accelerators) speed the matrix math behind training and inference. **Training compute** is the total work spent to train a model—not the same as “how big it looks on a slide.” Google designed Cloud TPUs...

Read lesson →
2.2L1

GPU, TPU, Trainium, MI300 landscape

A literacy map of major **AI accelerator families** used in cloud/HPC: NVIDIA **H100** (Hopper), Google **Cloud TPU**, AWS **Trainium** (with Inferentia for serving), and AMD **Instinct MI300** (CDNA 3). Not a buying recommendation—an or...

Read lesson →
Unit 3

Scaling

1 lessons
3.1L1

Scaling laws and compute

**Scaling laws** are empirical patterns relating model size, data, and compute to performance on training objectives. They are guides—not destiny and not proof of understanding. Hardware reality (accelerator availability, energy, network...

Read lesson →
Unit 4

Energy

2 lessons
4.1L1

Energy and AI systems

AI systems consume **energy** in data centers and devices. Honest discussion uses **sourced ranges and methods**, not viral single-number myths. The IEA’s *Energy and AI* report (published 10 April 2025) examines both electricity demand ...

Read lesson →
4.2L1

Data center energy and AI

How **data-center electricity** relates to AI growth: servers, accelerators, cooling, networking, and workload mix. IEA *Energy and AI* (10 Apr 2025) provides global modelling and policy framing for AI electricity demand and energy-syste...

Read lesson →
Unit 5

Serving

1 lessons
5.1L1

Quantization and serving basics

**Quantization** stores/computes model weights (and sometimes activations) in fewer bits to reduce memory and often cost/latency. **Serving** is running models reliably for live traffic. Modern accelerators advertise low-precision matrix...

Read lesson →
Unit 6

Network

1 lessons
6.1L1

Networking for distributed training

Why interconnect bandwidth/latency matter when training across many accelerators—and how weak networks erase GPU spend. Vendor stacks emphasize dedicated links: NVIDIA NVLink, AMD Infinity Fabric between MI300X OAMs, Google TPU inter-chi...

Read lesson →
Unit 7

Edge

1 lessons
7.1L1

Edge vs cloud inference

Choosing whether models run **on-device** or in the **cloud**—latency, privacy, cost, capability, and energy tradeoffs. MLPerf Inference tracks both **edge** and **datacenter** categories/scenarios (see Inference docs model tables).

Read lesson →
Unit 8

Mlperf

1 lessons
8.1L1

MLPerf Training and Inference

**MLPerf** (MLCommons) is an industry-standard benchmark family. **MLPerf Training** measures how fast systems train models to a **target quality metric**. **MLPerf Inference** measures how fast systems run models across deployment scena...

Read lesson →