Feedback loops in data
When a model’s outputs change the world (or the logs), and those changes become tomorrow’s training data—**feedback loops**.
What it is
When a model’s outputs change the world (or the logs), and those changes become tomorrow’s training data—feedback loops.
Why it matters
Ranking, moderation, fraud, and policing-adjacent tools can entrench errors if loops aren’t monitored.
How it works (plain)
Prediction → action → new data → retraining. If only some outcomes are observed (selection bias), the loop distorts harder.
Everyday example
A recommender shows fewer niche creators → they get less data → the model “learns” niches don’t exist.
Try it
Sketch a loop for one product decision. Mark what you *don’t* observe.
Myths
- ⚠️ Myth: Retraining weekly always improves quality.
- ✓ Reality: It can amplify a bad loop faster.
- ⚠️ Myth: Offline accuracy catches loops.
- ✓ Reality: You need online monitors and counterfactual thinking (Course 16).
Sources
- Course 16 causality; Course 17 MLOps; Course 19 bias
- NIST AI RMF: https://www.nist.gov/itl/ai-risk-management-framework ↗