Swapping the Person in Footage You Already Have

There are two jobs in AI video that look almost identical from the outside and behave nothing alike.

The first is generating new footage of a specific person. You hold a reference, the model produces a scene, and the character appears in it. The second is taking footage that already exists and replacing whoever is in it. Same reference image, same cast, entirely different problem, because in the second case you are not creating motion, you are inheriting it.

Ai-Hakase, the Japanese ComfyUI builder who documented the RefMod face-anchor technique, published a Character Swap workflow at the start of October that separates those two jobs on purpose. The distinction is her first slide, and it is the one to keep: RefMod puts a face into footage that does not exist yet, Character Swap replaces the person in footage that does. Confusing the two is how you end up with results that feel wrong for reasons you cannot name.

Three inputs, and only one of them is new

The workflow takes a source video, a reference image and a short prompt, and the roles are strict:

• The source video carries the motion. Timing, rhythm, camera movement, body language.
• The reference image carries the appearance. Face, hair, build, style.
• The prompt carries the assignment, meaning who becomes whom. It is not there to describe the scene.

That division is why the prompt can stay short. In a normal text-to-video or image-to-video generation you use the prompt to describe the subject, because nothing else will. Here the subject already exists in both directions, so the only open question is which one maps onto which.

Anyone who has done a reference-to-video workflow will recognise the pattern, and it generalises past this specific graph: when the motion source and the appearance source are separate inputs, you stop writing prompts that describe two things at once, and the results stop fighting each other. The same logic sits behind reference-based image editing, where a pinned character frees the prompt to describe only the change.

Why the swap and the repair are separate stages

Formally, this is reference-to-video person replacement on MiniMax H3 driven by a Character Swap LoRA. The workflow splits that into two jobs: the LoRA does the replacement, and a mask repair stage pulls the background and the edges back toward the original clip.

You can try the one-shot route with the LoRA alone. The reason to keep the stages distinct anyway is diagnostic. When a result misses, the two failures look different and need different fixes. If the person is wrong, the problem is in the reference, the LoRA's influence, or the assignment prompt. If the area around the person is wrong, no amount of prompting the LoRA will help, because that is repair's territory.

That the repair stage exists at all tells you something useful. Replacements do not stop politely at the outline of a person. The swap disturbs what the subject sat in front of, and a dedicated pass to restore it is the standard remedy. Treat the seam, not the face, as the piece of the frame you will spend time on.

The workflow also exposes a vision-language model you can enable, which accepts short instructions written in Japanese. It is a convenience layer rather than a quality setting, but it is a good reminder that the sentence you type into a graph node does not have to be written in the framework's native English.

The hardware answer is unromantic

This graph is heavier than the author's usual work, and she says so directly. On RunPod she normally uses RTX 3090-class machines; for this one she recommends an A40-class card with more VRAM and system RAM, because even short clips chew through both. The workflow includes the usual lightening measures, and quantised GGUF weights may bring it within reach of a local 24GB card, but the advice is to start with very short clips.

The cost logic that follows is worth internalising. A GPU that runs slowly does not make the project cheap, it cuts the number of attempts you can afford. If iteration count is the thing that decides whether a shot works, paying for headroom is usually cheaper than paying in patience.

One honest boundary: the detailed workflow file and the accompanying RunPod notebook live behind the author's membership, so what you can adopt directly from the public write-up is the mental model, the input roles and the two-stage design. Building the graph is the part you do yourself.

When swapping beats regenerating

The case for this technique rests on where the value sits in a shot. Motion is expensive to describe and unreliable to reproduce. A dance take that lands, a walk cycle with the right weight, a camera move that reads well: if you have one of those, generating the same performance again is a gamble with the thing you already won.

Swapping keeps the performance and changes the identity. That is the same trade visual effects have always made with a plate and a replacement, and it is why the technique is being applied to short-form dance and character videos first. Those are formats where the performance is the content and the performer can be substituted.

The limit is equally clear. If the motion is wrong, no swap will save it. Go back and generate the shot.