Skip to content

Why AI Video Gets Worse Every Time You Extend It

Chaining clips by exporting the last frame and feeding it back is a lossy loop, and the loss compounds at every join. Keeping the model latents and cutting on its own frame grid avoids cumulative pixel round-trip loss.

AI-assisted draft. Reviewed and edited by the Phosphene team before publication.

On this page
Two strips of 35mm film joined with clear splicing tape on a dark surface, one strip richer than the other

Six seconds of AI video is easy now. Forty is where people start complaining, and almost everyone blames the length.

Length is not the problem. The joins are. Watch a chained video closely and the damage is local: right where clip two begins, the jacket picks up a slightly different red, the jawline softens, the grain resets. Run that five times and a character has quietly become somebody else. The model did not get worse at long videos. The workflow destroyed information at every seam.

That seam damage has a specific cause, and it is the technique nearly every "extend video" button uses: export the last frame as an image, feed it back as the first frame of the next generation. It works, it is easy to explain, and it is lossy by construction.

The round trip is the bug

Video diffusion models do their work in latent space, a compressed representation the model thinks in. The decoder turns latents into pixels for you to watch. When you save the last frame as a PNG and reload it as a starting image, you have gone latents, pixels, file, pixels, latents. Two lossy conversions per join, and the model treats the second-hand result as ground truth.

So each clip starts from an approximation of the previous one, complete with that approximation's own compression artifacts, slightly shifted colour, and slightly re-guessed geometry. The next generation amplifies those. This is why the fourth clip in a chain looks like a photocopy of a photocopy while clip one looks fine.

You can patch the symptoms with colour matching, histogram alignment, or optical flow between clips. Those fix the step at the cut and leave the underlying state just as damaged.

The cleaner fix is to never leave latent space. Pass the model its own state instead of a picture of its own output.

A sidecar makes the master file the continue-able one

The open-source ComfyUI pack comfyui-obvpm-timeline does this. Every generation is saved as a pair: the MP4 you watch, and a .mctx.safetensors sidecar next to it holding the raw video and audio latents plus a header with frame counts, resolution, fps, the pinned windows used, and a SHA-256 of the MP4 it belongs to.

The size is real and worth planning for. Roughly 2 to 3 MB per second at 1280x704, so a ten-second take costs 20 to 30 MB of sidecar on top of the video. Cheap next to reshooting, expensive if you keep everything.

Two design decisions there are worth stealing even if you never install it.

The archive format becomes the thing you can continue, not the thing you can play. A normal MP4 pipeline throws away the only state that matters for extension and keeps the delivery copy. The pack inverts that: the sidecar is the master, the MP4 is a preview. A clip that still has its sidecar can be extended at full fidelity weeks later, on a different day, in a different session.

The hash binding stops you from lying to yourself. The sidecar records the hash of the video it was made with, and every load verifies the pair. Re-encode, trim, or swap the video and latent continuation is refused for that clip; the pixel fallback still runs but logs that it is pixel grade. This matters more than it sounds. Most "why is my continuation wrong" bugs are continuations of something other than the file you are looking at.

Pins are context, not glue

The mechanism underneath is keyframe anchoring, a native capability of MiniMax H3 rather than an external composite. The pack calls the operation a pin: a window of a source clip's latents is carried into the next generation as fixed context.

Placement decides the direction. A pin before the new clip means the generation continues out of it, which is an extend. A pin after it means the generation runs toward it, which is a prepend. One on each end means the generation has to leave one clip and arrive at another, which is a bridge. Window sizes come from H3's context ladder at 5, 22, 39, or 56 frames, with 22 the everyday choice. Longer windows preserve more motion and leave less room for anything new.

The part that separates this from frame freezing: pinned frames are scaffolding. The model renders them as part of the take, then a trim node cuts them off, so the delivered clip contains only newly generated frames and the sidecar records how much was pinned on each side. The model is told that a stretch of timeline is already decided, and it samples around that decision. Pinning removes the compositing cut at the joint, but it does not remove the model's own re-render, so a brightness or colour step can still appear where the pin ends. What pinning guarantees is that the frame never leaves latent space on the way through.

The frame grid is not arbitrary

H3 runs at 24fps and raw take lengths sit on a 17k+5 frame grid, from 5 to 3592 frames. A latent slice has to begin at cycle phase zero, and that requirement is where the 17-frame group grid comes from.

Cut off grid and you get one of two outcomes. The explicit pin path refuses the cut and names the nearest frames that would work. The convenience modes shift the edge silently and log where the seam actually landed. Snap cuts exists so a mouse drag lands on a legal boundary in the first place.

The pixel route accepts any cut frame you like, and that freedom is the tell. An encode defines phase zero wherever its window starts, so the cut you dragged is the cut you get. Convenience bought by discarding the thing that made the join exact.

Three ways a chain still breaks

Resolution is locked for the whole chain. Latents cannot be resized, so every clip in a continuation has to share one resolution, and mismatches are rejected up front. Plan the chain at one size. Changing your mind halfway means starting a new chain.

Audio slips out of sync without announcing itself. H3 produces about 8 ms more audio than video per clip, which sounds like nothing until a five-clip chain is 40 ms out of sync on a door slam. The trim and save nodes truncate audio to exactly frames divided by fps, which keeps the error from compounding. If you hear sync slipping across joins, the smoke is coming from somewhere in that arithmetic, not from the model.

A whole-timeline upscale hallucinates. The joint refine needs the latent upscaler's temporal chunking switched on. Run the entire timeline through the upscaler in one pass and it normalises over the whole sequence at once, producing a prior the refine then invents detail against, at every strength and under every sampling scheme. The pack refuses a run configured that way, which is a good example of failing loudly instead of shipping a pretty wrong frame.

Extending backward is harder than extending forward

A prepend reproduces its pinned content exactly, but whether the generated motion converges into it smoothly depends on the seed. In practice one seed in a handful converges, and the rest produce a visible hard cut into the pinned frames. More sampling steps and a longer window help.

So build forward and treat backward generation as a repair tool. That is also the honest use for a bridge. When a middle section has drifted, or a clip ends badly, bridging from the good part of one clip to the good part of another regenerates the connective tissue at latent grade instead of cutting around the damage.

The failure that looks like a colour bug

A join can step in brightness and colour even when every latent is intact, and the cause is usually an unrelated node pack.

Some H3 extension packs do not just add nodes. They replace core ComfyUI functions at import time with copies taken from an older core. One known example replaces MiniMaxH3.extra_conds and PackedLayout.__init__. Current ComfyUI passes latent masks into the model as per-token denoise masks so preserved context frames run at the clean-conditioning timestep. The older copy drops those masks, so the model treats preserved frames as fully noised and re-renders the continuation against context it cannot properly see. The replaced layout code also rejects the interior anchors that pin nodes depend on.

The detection signal matters because it is asymmetric. Pin nodes stop with an error that names the foreign patch. Plain extends do not. They run, they finish, and they simply look wrong.

When joins shift colour, read the pack list before you rewrite the prompt.

Explore cheap, finish once

The workflow that makes long clips practical is two-stage. Build the whole sequence at low resolution, where a take is minutes rather than an hour, and move clips around on a timeline until the cut works. Then run one upscale pass across the finished timeline.

Trying to get the story right at final resolution is how a forty-second video eats an afternoon and fails at eighty percent. The two costs are different in kind: iteration is time, and final resolution is VRAM. Separating them is the whole trick.

If you do not run ComfyUI

The principles survive without the tooling.

Never let the delivery codec be your master file. Keep one resolution per chain. Cut on the model's grid, not your instinct. Judge joins at full size, because a scrubbed preview hides the drift that ruins a shot. And keep the source context for any shot you might continue.

Hosted generators mostly hand you an MP4 and nothing else, which rules out latent continuation entirely. In that world the right move is to plan continuity at the shot level, with reference images carrying identity between generations rather than a chained video, which is the approach in our first-frame video guide and the consistency workflow breakdown. Phosphene generates through hosted models too, so this is our situation as much as yours; treat this piece as the mechanics, not a feature announcement.

There is no visual test that tells you which pipeline you are on. A worse second join is consistent with a pixel round trip, but a seed that converged badly or a foreign node pack produces the same symptom, and matching joins prove nothing either, because a pixel route fed a clean frame can look fine. Check the artifact instead. The sidecar header records the pinned windows and the SHA-256 of the MP4 it belongs to, and the loader verifies that pairing every time it opens the clip. When verification fails, latent continuation is refused for that clip and the pixel path runs anyway, with a log entry that says it is pixel grade. That line in the log, not the look of the seam, is the receipt you want.

The Japanese creator Ai-Hakase published a walkthrough of this workflow on note.com in early November; the hands-on half sits behind a membership, while the node pack and its documentation are open.

Sources

Share this guide
X LinkedIn