Lecture Captions & Transcripts (VTT, SRT)

The finished effect

What it looks like when it is done.

Burned-in captions from a produced lecture: short phrases in a band along the bottom, each word lit as it is spoken, held through every layout and every cutaway.

17 s · 1920 × 1080 · with sound

Overview

What it is, and why it exists.

Transcript and Captions puts the spoken words on screen for the whole lecture — for students watching without sound, for those who read faster than they listen, and for courses that must be captioned throughout. It is burned into the picture, so it plays the same on every device and in every player.

Its defining feature is word-level timing. Every word carries its own start and end time from the recording, so the caption does not just show the right sentence at roughly the right moment: it lights the exact word being said, as it is said. That same timeline drives the animated lecture's cues and the cutaway boundaries, which is why the captions, the deck and the cuts always agree.

It sits at the end of the ingest chain. Speech recognition transcribes the cleaned studio audio; cleanup re-times the words onto the cleaned clock; the lecturer's own script, where there is one, repairs the spelling; the words are grouped into caption phrases — nine words at most for the band, six for the column. Every layout and every cutaway then draws from those phrases.

The caption takes one of three forms depending on what is on screen: a band under the full-frame deck in the chapter's own colours, a column beside the presenter in the caption layout, and a white band on a graded shade over pictures and the centred presenter. The words, the phrases and the highlight are identical in all three.

up to 9 wordsBand phrase

up to 6 wordsColumn phrase

a gap over 0.42 sPause break

after 3.6 s of phraseDuration break

0.12 s before the first wordLead-in

fades from 1.9 s, gone at 2.8 sHold into silence

40% opacityUpcoming words

48 px down to 30 px, centred at y 950Band type

How it is made

Step by step, in the order the engine runs it.

Transcribe with word times

Speech recognition runs over the studio audio in chunks of about 24 s, each cut at the pause nearest the target so no word is split. Sub-word pieces become whole words, each with a start and end; a word's end is clamped to the next word's start and no word is shorter than 0.06 s.

Re-time onto the cleaned lecture

Cleanup removes dead air, false starts and lead-in; the words that survive are shifted onto the cleaned clock, and the words that were cut go with the audio they belonged to. Every later stage uses this clock.

Repair spelling from the script

Where the lecturer's script is available it is aligned against the transcript word by word. Small, well-anchored disagreements take the script's spelling on the transcript's timings; larger ones are listed for a person rather than applied. The repairs are stored as data for that lecture alone.

Apply name repairs

A short list of one-for-one substitutions catches names the recogniser has never heard. Each is a substitution or a deletion, so every remaining word keeps its time.

Group into phrases

The repaired words are grouped twice: into band phrases of up to nine words and column phrases of up to six, each breaking on punctuation, on a pause of more than 0.42 s, or once the phrase has run past 3.6 s, and never leaving a lone word mid-sentence.

Draw the band in the deck

The animated lecture draws the band into every frame of its render, in the chapter's palette, below the content and above the chrome.

Draw the column

For the Presenter + Captions layout a separate full-length plate is rendered: the blue stage, the chapter title, the column phrases fitted large beside the presenter, and the chrome along the foot.

Burn the band over cutaways and the centred presenter

For every cutaway and every Presenter Centred stretch, the band is rendered on transparency for exactly that window's frames and laid over the picture, with its graded shade. The narration underneath is never touched.

Verify

Short windows of audio are transcribed again independently and compared with the words the video displays; each assembled piece records the words and phrases it was drawn from, so a changed transcript forces the pieces to be rebuilt.

Elements

Everything this effect puts on screen, one piece at a time.

Each element below is shown as the engine actually produced it, in the real lecture, with its parts, and the rules that govern it.

Word timeline Spelling repair from the lecturer's script Hand repairs for names Band phrase Column phrase Word states Current-word highlight Phrase timing Fitting and line breaks Band in the full-frame animated lecture Band inside the Presenter + Animated slide card Band over a generated still Band over a generated video clip Band over a drawn cutaway Scrim Caption column Band under the centred presenter Emphasis on the highlight

The band at the start of a phrase, every word waiting for its own moment.

Word timeline

Every spoken word with its own start and end time

Everything in this effect is drawn from one list: every word the lecturer said, in order, each with a start and end time on the cleaned lecture's clock. Speech recognition produces it from the studio audio in chunks of about 24 s, each cut at a natural pause so no word is split between chunks; sub-word pieces are merged into whole words, and each word's end is clamped to the next word's start. The cleanup pass then re-times every surviving word onto the cleaned timeline. From that point no stage re-times a word: repairs change spelling, never timing.

Anatomy

Chunks: about 24 s, cut at the pause nearest the target, between 12 s and 32 s after the previous cut
Pause detection for chunking: -28 dB for at least 0.28 s
A word ends 0.24 s after its last sound piece unless the next word starts sooner; no word shorter than 0.06 s
Words removed by cleanup are dropped; the rest are shifted onto the cleaned clock