The plan sounded easy when I said it out loud. The tool joins the clips, then adds captions automatically. One sentence. But when I sat down to build it, that word automatically unfolded into three separate problems, each hard on its own: where the words come from, when each word appears, and how the text looks on screen. This post is about all three, honestly, including the parts I did not conquer.
Problem one, the words. A caption needs text, and no tool can invent the right text. Either you write the captions yourself, or something must listen to the audio and transcribe it, which is speech recognition, a whole field of its own. For my recap videos I was writing the caption lines myself, so my tool’s honest job was not to create words but to place the words I gave it. Knowing exactly which part is your job and which part you are pretending is automatic was the first clarity this feature forced on me.
Problem two, the timing. Each caption must appear at the right moment and leave at the right moment. FFmpeg’s drawtext filter can do this with the enable option, which turns the text on only between two timestamps:
vf = (
"drawtext=text='%s'"
":fontfile=/path/to/font.ttf"
":fontsize=42:fontcolor=white"
":borderw=2:bordercolor=black"
":x=(w-text_w)/2:y=h-120"
":enable='between(t,%s,%s)'"
) % (line_text, start_s, end_s)
Read the pieces: x=(w-text_w)/2 centres the line horizontally whatever its length, y=h-120 keeps it near the bottom, borderw=2:bordercolor=black gives white text a black outline so it stays readable on any footage, and enable='between(t,start,end)' is the timing, the text exists only inside that window. For several caption lines you chain several drawtext filters, each with its own window. It works, but every timing is a number a human decided, so automatic here really means the tool applies your timings tirelessly, not that it knows them.
Problem three, the styling. Escaping broke me more than once. A caption containing an apostrophe or a colon collides with FFmpeg’s own filter syntax and the whole command fails with a cryptic parse error. The blunt, reliable fix for real subtitle work is to stop passing text on the command line entirely and use a subtitle file instead, where an SRT file carries the text and timings and one filter renders it:
cmd = ["ffmpeg", "-i", video_in, "-vf", "subtitles=captions.srt", video_out]
An SRT is just a plain text file of numbered entries, a time range per entry, and the caption lines, and FFmpeg burns it in with sane default styling. Moving the words and timings into a file removed the escaping wars and made the captions editable without touching the command.
A few things people ask me about this
Why does my drawtext command fail with a parse error? Almost always unescaped characters in the text, apostrophes and colons collide with filter syntax. Escape them, or better, move the text into an SRT file and use -vf subtitles=file.srt.
How do I show text only during part of the video? Add enable='between(t,START,END)' to drawtext, with times in seconds. Chain multiple drawtext filters for multiple lines.
Can FFmpeg generate captions from speech? No. FFmpeg places and renders text, it does not transcribe audio. Transcription needs a speech recognition system, then you feed its output to FFmpeg as subtitles.
Next
With the toolkit feature-complete, its speed became the pain. Processing was slow enough to break my flow, and that pushed me to look outside Python at a language built for speed. What I actually learned from trying Rust is the next post.
