Subtitles: we run the model ourselves
Transcription is the one AI workload here that is constant and predictable. That makes it the one worth owning.
In short
Generated Worlds transcribes episodes with faster-whisper, run on its own systems rather than through a hosted API: the large model, voice-activity detection on, language detected per episode, and the weights kept in memory across a batch. Translating the finished cues is the only part of the subtitle pipeline that calls out to anyone.
Every episode needs a subtitle track. Not as an accessibility afterthought — as the product: search reads the cues, clip selection reads the cues, and a viewer watching without sound reads them directly. Which means transcription is not an occasional job. It runs on everything, forever.
01Three decisions that did the work
- Voice-activity detection on. Without it the model will happily transcribe a music bed, and what it produces is a loop of the same invented line — the single worst-looking failure in the whole pipeline, because it appears on screen.
- Language detected per episode rather than declared once. Declaring a language is faster and is wrong the first time a series arrives in another one.
- The weights stay loaded between episodes. Loading a multi-gigabyte model once per file is pure waste across a batch, and it is a one-line fix that nobody makes until they measure it.
02Why not a hosted transcription API
Because the workload is steady and it is ours. A per-minute API is the right answer when the volume is spiky or small. Owning it also makes a re-run nearly free, which changes how willing you are to re-run everything after improving a setting — and improving settings is most of the work.
Translation is the exception. Turning finished cues into another language is a language-model job, batched dozens of segments at a time, and it is where we pay for a stronger model: across a whole episode the thing that breaks is not vocabulary but consistency — a character's name drifting, a register slipping between formal and casual. That is exactly what the cheaper option gets wrong first.
What it does not do
Automatic transcription gets proper nouns wrong, reliably: a character's name is the least trustworthy word in any cue file, which is why dialogue search is good at sentences and bad at names. And running a model yourself means the work is scheduled rather than instant — the wrong trade for anyone whose volume arrives in bursts.
Questions
- How are subtitles generated on Generated Worlds?
- Automatically, with faster-whisper: voice-activity detection on, language detected per episode, and cue files written per language. Translation into other languages is done by a language model over the finished cues.
- Is it cheaper to run Whisper yourself than to use an API?
- It depends on whether your volume is steady. Running the model yourself pays off on constant workloads and is the wrong choice for spiky or occasional ones, where a hosted API rents you the peak instead of making you own it.
Keep reading
- Never ask a language model what time it is
Our episode analysis asks a language model which line of dialogue a clip should end on, and looks the time up itself. Asking the model for the timestamp produced numbers that fit nothing.
- Why the crop is centred
Following the subject with a face detector holds the subject slightly better and looks worse. On a catalogue with burnt-in captions it also destroys the captions.
All posts from the Generated Worlds Tech Blog.