Reinforcement learning

In plain English

Reinforcement learning (RL) is trial-and-error learning. An agent takes actions in an environment, receives rewards or penalties, and learns a policy—a strategy for picking good actions.

Unlike supervised learning, the correct action is not always labeled for every step. Feedback may arrive late (win the game after many moves).

RL shines when you can simulate many episodes cheaply (games, board games) or carefully shape rewards in controlled settings.

How RL loops work

At each timestep the agent observes state, chooses an action, transitions to a new state, and gets a reward signal. The goal is to maximize long-term cumulative reward, not just immediate points.

Exploration tries new actions to discover better strategies; exploitation uses what already works. Too much of either hurts performance.

Algorithms like Q-learning, policy gradients, and actor-critic methods estimate value functions or directly optimize policies. Deep RL uses neural networks as function approximators.

• Policy: mapping from states to actions.
• Value function: expected future reward from a state or action.
• Reward shaping: designing signals so desired behavior is learned.

Going deeper

Real robots and logistics face sparse rewards, safety constraints, and expensive failures—simulation and human oversight are common.

RLHF (reinforcement learning from human feedback) ranks model outputs with human preferences, then fine-tunes language models to align with those rankings. It is RL with learned reward models.

Misaligned rewards produce hacky behavior—agents optimize the metric, not the intent. Reward design and evaluation are as important as algorithms.

Common misconceptions

RL is how all AI learns.

Most deployed perception and language systems are trained with supervised or self-supervised losses. RL is specialized for sequential decision problems.

A high reward always means good real-world behavior.

Agents exploit loopholes in reward definitions. Careful testing and constraints are required.

RL agents plan like humans.

They optimize statistics of returns in an environment model or samples. Interpretability and safety remain active research areas.

Key facts

• RL learns policies through interaction and reward signals.
• Delayed rewards make credit assignment—linking actions to outcomes—hard.
• Exploration versus exploitation is a core tradeoff in RL.
• Deep RL combines neural networks with RL algorithms for complex state spaces.
• Human preference learning for LLMs uses RL-style fine-tuning on ranked outputs.