Alignment and safety

Plain English

Alignment asks a simple question: does the system do what we actually want—not just what we literally said? A chatbot that helps with homework is aligned for tutoring; the same model giving exam answers without context is misaligned with academic integrity.

Safety work covers accidental harm (wrong medical advice), intentional misuse (malware help, harassment at scale), and systemic risks (deepfakes, automated propaganda). No single switch makes AI safe; teams stack data choices, training objectives, runtime filters, and human oversight.

Research on superhuman systems (/learn/narrow-vs-general-ai) shares vocabulary with today's product guardrails—but shipping safety for current tools is mostly engineering, policy, and testing, not abstract philosophy alone.

How it works

Training-time techniques include curated datasets, refusal examples, reinforcement learning from human feedback (RLHF), and constitutional or rule-based self-critique loops—often building on /learn/pretraining-and-finetuning and /learn/reinforcement-learning.

Inference-time controls: moderation classifiers, blocklists, tool permission scopes, rate limits, and logging. Red teams adversarially prompt models before launch; incident response updates filters when jailbreaks spread.

Evaluation ties to /learn/benchmarks-and-evaluation—safety benchmarks probe bias, toxicity, and capability misuse. Alignment is not only 'be nice'; it includes honesty about limits (/learn/hallucinations) and secure tool use (/learn/tool-use-and-function-calling).

• Specification: write down intended and forbidden behaviors.
• Mitigation: train, filter, and restrict capabilities.
• Monitoring: detect drift, abuse spikes, and novel failure modes.
• Governance: who approves releases and emergency patches.

Going deeper

Open-weight models shift the balance: safeguards in one app do not constrain all copies. Watermarking, provenance standards, and legal liability (/learn/ai-and-society) join technical mitigations.

Interpretability research asks why models behave as they do—still incomplete for large networks. Hybrid human-in-the-loop workflows remain the default for high-stakes decisions in medicine, law, and finance.

Common misconceptions

Safety prompts guarantee safe behavior.

Prompt-level rules are brittle; training, access control, and monitoring must reinforce them.

Alignment only matters for future AGI.

Today's deployed models already affect hiring, content moderation, and customer support—near-term alignment is operational.

Key facts

• Alignment matches behavior to operator and user intent.
• Safety stacks span data, training, inference filters, and governance.
• RLHF and related methods steer models using human preferences.
• Red teaming finds jailbreaks and harmful outputs before users do.
• Open deployment increases need for societal and legal guardrails.