Metrics: precision, recall, and ROC

Beyond accuracy: precision (of predicted positives, how many were right), recall (of real positives, how many you found), and curves like ROC that show tradeoffs as you change thresholds.

Imbalanced problems make accuracy misleading. Choosing metrics is choosing whose errors matter.

Spam filter: high precision → fewer false spam tags; high recall → fewer missed spam. You rarely max both without cost. Thresholds move the tradeoff.

Airport security vs email filters optimize different error costs.

For one problem, say which is worse: false positive or false negative—and pick a metric that reflects that.

⚠️ Myth: AUC solves metric choice forever.

✓ Reality: Useful summary; still check operating points and calibration.

⚠️ Myth: Accuracy is fine if it’s “high.”

✓ Reality: 99% accuracy on 1% rare events can be trivial.

• Course 03 supervised learning; Course 04 statistics
• Google ML Crash Course classification: https://developers.google.com/machine-learning/crash-course ↗
• scikit-learn metrics docs: https://scikit-learn.org/ ↗