COURSE 13Intermediate10 IN-DEPTH LESSONS

Speech and Audio

Acoustic modeling and sound: speech recognition (ASR), audio event detection, text-to-speech (TTS), and open toolkits (Kaldi, ESPnet, SpeechBrain).

Start Lesson 1: Speech recognition overview →← All 29 Courses

Course Syllabus & Units

Unit 1

Asr

1 lessons
1.1L1

Speech recognition overview

**Automatic speech recognition (ASR)** turns spoken audio into text. It sits next to related jobs such as wake-word detection, language ID, speaker diarization, and speech translation. Stanford’s CS224S frames spoken language technology ...

Read lesson →
Unit 2

Family

1 lessons
2.1L1

Audio and speech models

A **family map** of models that handle speech and broader audio: recognition (ASR), synthesis (TTS), enhancement/separation, spoken language understanding (SLU), speech translation, music/audio generation, and non-speech event detection....

Read lesson →
Unit 3

Tts

1 lessons
3.1L1

Text-to-speech basics

**Text-to-speech (TTS)** turns written text into spoken audio. Modern neural TTS can sound natural. The same progress raises impersonation risk—this chapter teaches **how TTS systems are built and evaluated**, not how to clone a person w...

Read lesson →
Unit 4

Diarization

1 lessons
4.1L1

Speaker ID and diarization overview

**Speaker recognition** asks *who* is talking (identify or verify against enrolled voices). **Diarization** answers *who spoke when* in a recording—labeling time segments by speaker turns, often without knowing names in advance (“Speaker...

Read lesson →
Unit 5

Risk

1 lessons
5.1L1

Voice clone risk literacy

A literacy chapter on **voice cloning risks**: impersonation scams, consent, and verification habits. No cloning how-to.

Read lesson →
Unit 6

Events

1 lessons
6.1L1

Audio event detection

**Audio event detection (AED)** / **sound event detection (SED)** finds non-speech (or mixed) sounds in audio: alarms, glass break, dog bark, machine faults, sirens—not “what words were spoken,” but “what happened.” <!-- IMAGE: waveform ...

Read lesson →
Unit 7

Ssl

1 lessons
7.1L1

Self-supervised speech representations (wav2vec 2.0)

**Self-supervised learning (SSL) for speech** trains a model on raw audio **without** human transcripts first, then fine-tunes on a smaller labeled set for ASR (or other tasks). **wav2vec 2.0** (Baevski et al., 2020) is a landmark framew...

Read lesson →
Unit 8

Whisper

1 lessons
8.1L1

Multilingual ASR and Whisper

**Whisper** (Radford et al., 2022) is a family of encoder–decoder speech models trained with **large-scale weak supervision**: predict transcripts (and related tasks) from internet-scale audio–transcript pairs. The headline idea: scale m...

Read lesson →
Unit 9

Toolkits

1 lessons
9.1L1

Speech toolkits: Kaldi, ESPnet, SpeechBrain

Three widely used open toolkits for speech research and engineering: | Toolkit | Character (plain) | | ------- | ----------------- | | **Kaldi** | Long-standing ASR toolkit for researchers/professionals; strong classic hybrid + decoding-...

Read lesson →
Unit 10

Translation

1 lessons
10.1L1

Speech translation: cascades to end-to-end

**Speech translation (ST)** converts speech in one language into text (or speech) in another. For three decades the field moved from **cascades** (ASR then MT) toward **tighter coupling** and **end-to-end** models that map speech → targe...

Read lesson →