COURSE 18Intermediate11 IN-DEPTH LESSONS

Evaluation, Benchmarks and Red Teaming

Rigorous evaluation and adversarial testing: capability benchmarks, safety eval suites, automated red-teaming (HarmBench), and proxy vs gold metrics.

Start Lesson 1: Evaluation and benchmarks overview →← All 29 Courses

Course Syllabus & Units

Unit 1

Overview

1 lessons
1.1L1

Evaluation and benchmarks overview

**Evaluation** checks whether a system meets a stated goal. **Benchmarks** are shared tests the field uses to compare systems—useful, incomplete, and sometimes overfit. Anthropic’s *Challenges in evaluating AI systems* (2023) argues that...

Read lesson →
Unit 2

Red Team

1 lessons
2.1L1

Red teaming basics

**Red teaming** intentionally probes an AI system for failures—unsafe answers, policy bypasses, privacy leaks, or brittle tools—so you can fix them before attackers or accidents do. OpenAI describes red teaming as part of iterative deplo...

Read lesson →
Unit 3

Llm

1 lessons
3.1L1

LLM-as-judge caveats

Using a model to **score** other model outputs—useful, biased, and gameable if you aren’t careful. Anthropic groups this with the broader challenge of evaluating generative systems: open-ended behavior is hard to reduce to a single number.

Read lesson →
Unit 4

Online

1 lessons
4.1L1

Online evaluation and A/B

Measuring AI changes on live traffic with experiments and guardrails—complementing offline golden sets. Anthropic notes open-ended dialogue A/B tests with humans as a core (if expensive) way to rank helpfulness and harmlessness in realis...

Read lesson →
Unit 5

Safety

1 lessons
5.1L1

Safety eval suites

Bundles of tests for disallowed behaviors, privacy leaks, and tool misuse—run like a regression suite. Distinct from capability leaderboards: here the goal is **robust refusal / safe behavior**, not higher quiz scores.

Read lesson →
Unit 6

Validity

1 lessons
6.1L1

Contamination awareness

When eval items (or very close paraphrases) appear in training data—so scores look better than true generalization. Anthropic’s evaluation challenges essay compares this to students seeing the test questions beforehand.

Read lesson →
Unit 7

Metrics

1 lessons
7.1L1

Task success vs proxy metrics

Separating **true task success** (did the user finish the job safely?) from **proxy metrics** (BLEU, thumbs, latency, token count, MMLU) that only partially correlate. OpenAI’s GDPval work (2025) pushes toward economically valuable knowl...

Read lesson →
Unit 8

Nist

1 lessons
8.1L1

NIST GenAI Profile and TEVV

NIST’s **AI Risk Management Framework (AI RMF 1.0)** (Jan 2023) is a voluntary, cross-sector guide to manage AI risks and promote trustworthy use. The **Generative AI Profile** (NIST AI 600-1, published July 26, 2024) is a companion prof...

Read lesson →
Unit 9

Helm

1 lessons
9.1L1

HELM Capabilities literacy

**HELM** (Holistic Evaluation of Language Models) is Stanford CRFM’s framework for multi-scenario, multi-metric evaluation with prompt-level transparency. **HELM Capabilities** (e.g. v1.14.0) is a curated leaderboard/benchmark focused on...

Read lesson →
Unit 10

Redteam

1 lessons
10.1L1

Red teaming literacy

**Red teaming literacy** is the civic/technical vocabulary to read System Cards, lab blogs, and gov eval notes without confusing “we red-teamed” with “safe forever.” OpenAI’s Red Teaming Network formalizes a community of external experts...

Read lesson →
Unit 11

Automated

1 lessons
11.1L1

Model-written evals and HarmBench

**Model-written evaluations** (Anthropic, Dec 2022) use language models to help generate large numbers of test items (e.g. yes/no questions, multi-stage schemas) so labs can discover behaviors faster than pure crowdwork. **HarmBench** (M...

Read lesson →