Media and audio

By now your video moves and reads right; this level gives it a voice. HyperFrames owns media playback; a companion media pipeline resolves everything else — voice, music, sound effects, images, icons, logos, captions, and background removal. You reach all of it by describing what the composition needs, and the agent resolves each need to a frozen local file. The craft here is precision: vague media asks (“add some music,” “no sound”) are the ones that come back wrong, because the pipeline does exactly what the words say.

​

Voiceover (TTS)

Text-to-speech runs locally through Kokoro — no API key needed — with a HeyGen TTS upsell behind it. Describe the content and the agent picks a fitting voice, or name the voice, tone, and speed directly:

Generate narration for this script with a professional female voice.

Add TTS voiceover, British male voice, at 1.1× speed.

The Vocabulary table maps content types to Kokoro voices (for example af_heart / af_nova for a product demo, am_adam / bf_emma for a tutorial, af_sky / am_michael for marketing). Name one directly if you already know it; otherwise describe the read and let the agent choose.

• ❌ add a voice
• ✅ warm, unhurried female narration of the quoted script — tone and pace are what actually change the delivery

​

Background music

Music resolves from a large catalog by mood, and it should almost always sit under the narration, not compete with it. Give the mood and a loudness target — the pipeline can duck and normalize to a level, so an explicit target lands a mix instead of a guess:

Add subtle electronic BGM, kept under −18 dB so it stays beneath the voiceover.

Upbeat tech-launch music bed at a low level, ducking under narration.

• ❌ add background music — you’ll get a full-volume track fighting the VO
• ✅ subtle background music, ducked ~12 dB under the voice — a mix instruction the pipeline can execute

​

Sound effects

SFX resolve from a small bundled library plus the catalog. Cue them to specific moments — a transition, a stamp-in, an impact — rather than sprinkling them:

Add a whoosh on each of the three scene transitions.

Put a soft click on the button press at 0:04.

​

Pace reveals to the narration

Once a video has a voice, the voice is the clock — the next instruction that matters is telling the agent that on-screen elements land on their spoken cues — the stat appears as the narrator says it, not at some independent time the builder eyeballed. Without this, narration and visuals drift into two parallel tracks that happen to share a file:

VO-paced reveals: each scene’s elements land on their spoken cues; secondary elements keep resolving while the narrator is mid-thought; the scene is complete just as the narration moves on.

The capstone film applies exactly this rule to every region — its Direction block reads:

VO-paced reveals: each region’s elements land on their spoken cues as the camera arrives; secondary elements keep resolving while the camera is present; the region is complete just as the camera accelerates away.

Two practical notes: the agent gets word timings for free (the narration is transcribed with per-word timestamps, the same machinery behind captions), so “on its spoken cue” is a real, executable instruction — and the inverse rule matters just as much: the narration never waits for the visuals. Pace the camera and reveals to the voice, not the voice to the animation.

​

Captions and transcription

Captions come from word-level timestamps. When you generate a voiceover, the timing comes with it; for existing footage, transcription produces the timing (Parakeet by default, with a whisper.cpp fallback). Scaffolding a project from a source video can generate captions from its audio directly.

Transcribe the narration and add karaoke-style captions synced to it.

Generate captions from assets/interview.mp4 and style them hype, scale-pop.

Caption look is its own vocabulary (tone, size, per-word emphasis) — see Captions catalog for the styles. This page is about producing the timed text; that page is about styling it.

​

Background removal (transparent cutouts)

The remove-background command mattes a subject out of a video or image locally and hands you a transparent WebM you can drop into any scene as a <video>:

Remove the background from assets/presenter.mp4 and float the subject over the scene.

One caveat is load-bearing: the built-in model is purpose-built for people — head-and-shoulders or full-body, reasonably stable framing, a background that contrasts with the subject. It returns a mostly-empty mask on non-human subjects (products, animals, objects). If you need to cut out a product, say so — the agent should route to a different tool rather than run the person model and get nothing.

• ❌ remove the background from this product shot with the built-in command — the human-matting model can’t see it
• ✅ matte the presenter out of assets/talk.mp4 (person) — or, for a product, flag that it’s a non-human subject so a different matter is used

The Remove background guide covers the person-only caveat, the two-layer plate for text-behind-subject, and alternatives for objects and hair-fine mattes.

​

Video-in-video and picture-in-picture

Layering footage — a talking head over a scene, a subject in front of a headline, PiP inset — is a compositing prompt. Two grounded rules keep it frame-accurate, and the agent applies them for you, but naming the layout you want helps:

Put the transparent presenter cutout in the bottom-right, over the chart scene.

Layer the headline behind the presenter so their silhouette occludes the text.

​

Bring any footage

You don’t need to pre-convert supplied footage before naming it in a prompt. If a clip’s codec doesn’t play back cleanly in a browser — HEVC (H.265) is the common case, straight off an iPhone or a screen recorder — the framework probes the asset and builds a bounded H.264 proxy automatically, cached under .transcode-cache/. preview, play, Studio, and published player pages use the proxy for playback; a render always decodes the original file, so nothing about final quality or color is touched. hyperframes lint also flags the asset at info level (hevc_preview_codec) so you know a proxy is in play, and it’s optional — --no-proxy per command, or media.autoProxy: false in hyperframes.json project-wide. The same mechanism covers alpha-channel sources too (ProRes 4444, alpha WebM proxy to VP9+Opus WebM instead of being refused), so a transparent cutout in a hostile codec isn’t a blocker either. None of this changes how you phrase the ask: name the footage by path like any other supplied asset, and describe the composition you want built from it.

Build a short picture-in-picture piece from source-hevc.mp4 — inset it bottom-right over a full-bleed background scene, with a soft rounded border.

Rendered from the prompt above, unedited — the source clip is a plain H.265/HEVC file; render decoded it directly via FFmpeg, while preview would have used the automatic H.264 proxy. See the Rendering guide for the mechanics — proxy generation, caching, and which codecs it covers.

​

The supplied-assets rule

For any asset you already have, an explicit path is the instruction that removes the most ambiguity. The agent will search when you describe an asset, but a path removes every ambiguity about which file — and for your own brand assets, it’s the only way to guarantee the right one:

• ❌ use my logo
• ✅ use assets/logo.svg

This matters even when resolution would otherwise work: brand and entity assets should point at your file, not a resolved lookalike. (Third-party logos are a separate case — the pipeline pulls official marks from a logo cascade and never hand-redraws them, so “add the LinkedIn logo” is fine; “add my company’s logo” needs a path.)

​

Say what “no sound” actually means

The most common audio mistake is a negative that means less than you think. “No narration” removes the voiceover — it does not silence music or sound effects. If you want genuine silence, say so:

• ❌ no narration when you mean a completely silent video — music and SFX can still be added
• ✅ no audio at all — the unambiguous way to ask for silence

This mirrors the negatives discipline in Anatomy: close the gap explicitly, because the engine acts on the literal words.

​

Related

This is the clause in the full capstone prompt that buys the piece — prompt language you can lift for your own video:

[…] a real talking-head clip (generate a short clip of a person speaking one neutral line via the media pipeline’s avatar video generation […] transcode it to HEVC hvc1 so the automatic proxy subsystem carries preview) sits as a clip on the wire. The order of operations IS the story: as the camera arrives and BEFORE the person speaks, the framework mattes the footage — the background peels away via background removal […] THEN they speak, and the main keywords of their own line — derived from the clip’s transcription — land word-synced […] The clip’s own audio ducks the BGM briefly; the VO resumes as the camera pulls away.

That clause, rendered — the region cut from the finished film. Next: Design systems and brand — pointing the agent at a source of brand truth instead of describing a vibe.