Trevor PaughSystems engineer
Trevor PaughProject Chronicles

Chronicles / Clipper Overlord

Render v3: framing that follows the shot

Design render v3

After reviewing the first batch, the renderer became shot-aware: it detects cuts, picks one framing mode per shot, keeps on-screen text in frame, and trims silence against the actual audio instead of the transcript.

The first renderer did one thing: find a face, crop a vertical window around it, and follow it with a smoothed pan. On a single talking head that looks fine. On a two-camera podcast it looks like someone sneezing on the tripod.

What changed

Cut detection. The renderer now finds the shot boundaries in a clip first. Each shot gets its own framing decision, so the window never slides across a hard cut.

One framing mode per shot. For each shot it picks a mode: a tight single-face crop, a wider two-person frame, or a static full-width letterbox when there is no reliable face. Within a shot the mode never changes, so the frame stops twitching between speakers.

On-screen text detection. A text detector (PP-OCRv3) marks regions with burned-in graphics such as lower-thirds and highlight boxes. The crop widens to keep those in frame instead of slicing a caption in half.

Silence trim against the audio. The transcript’s word timestamps are close but not exact. Clip edges are now checked against the actual waveform, so the last word plays in full.

Why

Three defects in the first batch, each one visible in a specific clip: last words cut off, cropped on-screen text, and the frame flipping between faces. This was the first of several passes on framing. Later rounds added follow framing without face hops, holding on whoever is speaking, and widening the frame just before on-screen text appears.