The Retake Is the Expensive Part: Painting a Plate Instead of Re-Rolling a Shot

Every AI video workflow I have used has the same weak point, and it is not the generation. It is the moment after generation when you notice one thing that is wrong. A light stand in the corner of a period shot. A logo on a shirt that should not be there. A street lamp that reads as a street lamp from 2026 in a scene set in 1970.

At that point you have two bad options. You can re-roll the shot and lose the take, which is cheap in credits and expensive in everything else, because the performance, the camera move and the light you liked are gone. Or you can inpaint frame by frame, which produces a clean frame and a dirty shot: each frame solves its own little problem and the results disagree with each other, so the patch crawls and boils while everything around it stays still.

Paint a plate, an open-source ComfyUI app released this week by AInVFX, takes a third route. You paint the correction on the first frame, and the app carries that single decision through the rest of the shot. The tool is three days old, has two stars on GitHub and is an entry in a ComfyUI contest, so treat it as a design document rather than a product. The design is worth reading anyway, because it is the first time I have seen a retake workflow in this space behave like a plate workflow instead of a generator.

What it actually does

The interaction is three tools on the first frame of a clip it calls a plate:

• Paint adds what you describe where you paint. It may alter the frame slightly around your strokes, so a painted fire can light what it touches.
• Erase removes the object under your stroke. With Whole object, SAM 3.1 finds all of it from a name like "red umbrella", and your stroke may even cross the background.
• Light relights the shot. You pick a colour from swatches, paint where the light comes from, and optionally describe it.

Underneath, it is a chain of open-weight models doing four different jobs:

The pipeline is not one model doing everything. It is a keyframe edit, a matte, a tracking pass and a video-to-video pass, sequenced as separate ComfyUI workflows with names like snap, edit, matte, clean, shot, comp, review and export. A quick 768-pixel pass lands on the first frame in seconds so you can judge the idea, then the full-size passes run.

That sequencing matters more than it sounds. It means every stage can fail visibly and separately. If the matte slipped, you see it. If the shot pass drifted, you see it against the plate. In tools that do the whole thing in one opaque call, you get one result and no diagnosis.

The sentence that should get the attention

The README's claim about export is the part I would put in front of any studio evaluating this:

Export then writes it in the plate's own format: same file type, bit depth, frame numbers, colour tags, timecode, audio and EXR layers, with every untouched pixel identical to the plate.

Every untouched pixel identical. That is a compositing contract, not a generation feature, and it changes what the output is. The result is a patch over existing footage rather than a re-render of it. The area you painted holds generated pixels. Everything else is the original, bit for bit.

This is why the app carries an OpenColorIO stack that follows OpenRV, reads the plate's colour space from the file when it opens, and defaults to the ACES studio config. It is why the viewer has a wipe (F6) that compares an EXR against its EXR at full precision, and why exposure, contrast, gamma and saturation behave like an RV session rather than a web preview. The tool is built on the assumption that you will grade this thing and hand it to someone else.

The practical difference shows up in three places:

Delivery. Frame numbers, timecode and audio survive, so the clip still cuts into an edit. A re-encode with a different starting frame is a small technical disaster that costs an afternoon to discover.

Grading. A patch that sits in the plate's colour space can be graded with the plate. A patch that arrives in a different transfer curve has to be matched by eye, shot by shot.

Review. You can flip between plate and result and see only the edit. That is the only honest way to judge a cleanup, because the eye forgives a lot when it cannot compare.

The edit as an object, not a seed

Two smaller decisions in the repo point at the same philosophy.

Edits live in a History panel. Clicking one brings its words back and lets you repaint, rub out or retype it, and dependent edits above it follow. The plate, its colour space and every edit can be saved as a scene.

And there is a job index on disk, so the same edit on the same plate never runs twice, even in a later session. That is a caching detail with a large operational consequence: retakes become repeatable. You can open a shot next week, change the light, keep the erase, and not pay for the erase again. Compare that to the seed-and-pray loop, where the only way to revisit a result is to try to reproduce the exact conditions that produced it.

None of this is glamorous. It is the unglamorous part that decides whether a tool gets used on a real job.

The limits list is the real specification

I trust a tool's limits section more than its feature list, and this one is unusually honest. Read as a group, the failure modes are all the same failure:

• A cast shadow outside the erased object stays, so you erase it with a second stroke.
• An erased object can come back if its name reaches the shot's prompt, so the page never names erases there.
• Very thin objects need their name for Whole object: "floodlight pole" finds more than "floodlight".
• An erased head comes back as a head. Paint the new head over the old one instead.
• An addition painted across a foreground object breaks up mid-shot. Paint additions in open space.
• Edits are painted on the first frame, shots run up to 121 frames, edits up to 1920 pixels on the long side.

Every one of those is occlusion or parallax. You are asking a model to infer, from one frame, what should happen to a region for four seconds while the camera moves and other objects pass in front of it. Where the geometry is unambiguous, it works. Where something is hidden at the start and revealed later, or where a foreground object crosses the painted region, the model has to guess, and the guess changes as the guess changes.

That gives you working rules, and they are the same rules a compositor would give you about a single-frame cleanup:

1. Never name the thing you removed anywhere in the shot prompt. This is the most valuable line in the whole README, because it is a model behaviour most people would never predict.
2. Treat the erase and the shadow as two edits. The object is a shape; its shadow is a second shape with different physics.
3. Keep additions out of the path of foreground objects. Open space propagates cleanly. Replacing a head or a sign that something walks in front of does not.
4. If the correction is essential to the shot and the parallax is heavy, this is not the tool. Plate work still has a floor.

What it costs to run

This is a workstation tool, and the numbers are blunt. The recommended and tested configuration is 32 GB of VRAM or more. The model set is about 60 GB: FLUX 3 Image Klein at 5.8 GB, its text encoder at 4.9 GB, the prompt upsampler at 6.7 GB, LTX 2.5 at 21.5 GB, its Gemma text encoder at 15.4 GB. On a smaller card the documented path is a rented GPU, and the repo ships a Runpod start script that installs ComfyUI, links the pack, downloads the models and starts the server in about three minutes.

One warning that belongs in any shared setup: the Runpod page has no password on port 8188, and the README says so plainly. Anyone with the pod address can use it. If you try this on a rented machine, treat the pod as public until you put something in front of it.

The licence is not one licence

The code is Apache 2.0, which will get quoted as "free" in a lot of threads. The weights are a stack of separate agreements, and the two that matter are stricter than the code.

FLUX 3 Image Klein is under the FLUX Kommunity License v1.0. Non-commercial use is the default. Commercial use of the outputs is permitted for what the licence calls a Qualifying User, defined as an entity whose gross annualised revenue, affiliates included, is under 5 million USD. Qualifying users also take on two duties: filter or review outputs before distribution, and disclose AI generation where the law requires it. Commercial fine-tuning, distillation and redistribution of the model sit outside that exception.

LTX 2.5 and its three IC-LoRAs are under the LTX-2.x Community License. Commercial and production use is free below 10 million USD in annual revenue, measured across the whole entity including subsidiaries and affiliates. At or above 10 million USD you need a paid commercial agreement, and transferring fine-tunes may need one at any revenue. SAM 3.1 has its own licence, and the prompt upsampler's system prompts inside the repo are Black Forest Labs' own text under Apache 2.0.

So "is this pipeline licensed for commercial work" does not have one answer. It depends on the model and on the intended use. A small studio generating finished shots for clients sits inside both exceptions. A studio at or above 5 million USD cannot rely on the FLUX commercial-output exception at all, and one at or above 10 million needs the LTX agreement whatever the deliverable. The terms live on model cards, not in the repository licence file, which is usually where people stop reading.

Where this leaves the retake problem

The tool is at version 1.0.4 and shipped a rename this week, and its optional SDR-to-HDR pass is switched off and hidden because its shots flicker. That is not a criticism; it is what an honest first release looks like. It also tells you the author is testing output rather than shipping a demo.

The reason I think it matters beyond a two-star repository is that it names the correct unit of work. AI video tools have mostly been designed around the prompt, and the prompt is the wrong object to edit, because a prompt has no location, no matte and no history. A plate has all three. It sits in a pipeline, it can be compared against itself, and a change to it can be described in the language a client already understands: we painted something out.

The general lesson is not "use this app". It is that the retake, not the generation, is where the money and the hours go, and that the way to make retakes cheap is to make them local, reviewable and repeatable. That is a smaller ambition than generating a film, and it is the one that decides whether generated footage actually gets used in anything with a deadline. The wider question of what clients pay for once generation is commodity is one we wrote about separately, and cleanup sits squarely inside that answer.

If you want to see the other end of this pipeline's economics, the LTX 2.5 runtime profile we published is a useful companion: the same model, running 384 silent fallbacks per clip. A cleanup pass is only affordable if the underlying sampler is not quietly wasting your GPU.