Weight initialization

How you set starting weights before training. Bad init can stall learning; good init keeps signals and gradients in a healthy range.

Explains mysterious “my net won’t train” failures alongside LR and architecture choices.

Too large → explosions; too small → silence. Modern schemes scale random weights by layer size (Glorot/He families). Frameworks often default sensibly—know what they assume (activation type).

Tuning guitar strings before a song—not random yanks.

When a lab fails to learn, check init + activation pairing before rewriting the model.

⚠️ Myth: All zeros is a fine start.

✓ Reality: Symmetry breaking fails; units stay twins.

• Course 05 vanishing gradients; activations
• Glorot & Bengio / He et al. init papers (cite when teaching)