COURSE 04L2100% FREE
Verified 2026-08-10

Gradient descent geometry

A geometric view of **gradient descent**: loss as a landscape, gradient as steepest uphill, steps downhill with a learning-rate stride.

What it is

A geometric view of gradient descent: loss as a landscape, gradient as steepest uphill, steps downhill with a learning-rate stride.

Why it matters

Connects Course 04 calculus to Course 05 optimizers—why LR matters, why valleys slow you, why saddles confuse intuition.

How it works (plain)

Imagine foggy hills. Each step feels local slope. Too large a step jumps past valleys; too small crawls. Momentum remembers direction; adaptive methods change stride per axis.

Everyday example

Hiking down in fog with a short measured step rule.

Try it

Sketch a U-shaped curve and mark an overshooting step vs a careful step.

Myths

⚠️ Myth: Convex bowl pictures explain all deep nets.
✓ Reality: Deep loss surfaces are complex—geometry still builds intuition.

Sources