RefMod: A 1.5MB Face Anchor for MiniMax H3 That Skips LoRA Training

The most annoying part of a MiniMax H3 character workflow is not the generation. It is the setup, every single time. Eight reference PNGs, resized, pushed through the VAE again, VRAM spiking, and a prompt that slowly fills up with <Picture 1> tags until the thing you actually wanted to describe gets crowded out by plumbing.

RefMod (short for Reference Latent Adapter) is a community technique that moves that work out of the generation loop. A Japanese ComfyUI builder who publishes as Ai-Hakase wrote up both halves of it in late September and early October, and the idea is simple enough to describe in one sentence: encode the character once, save the result, load it forever after.

What it actually does

RefMod encodes a reference image set, or video, through MiniMax H3's VAE a single time and writes the result to a .safetensors file of roughly 1.1 to 1.6MB. In later generations you load that file and apply it into the conditioning, instead of resizing and encoding a stack of stills on every run.

The creator's own summary of what that buys you:

• Speed: building a pack takes tens of seconds to a few minutes, and subsequent generations skip the per-run encoding work.
• Memory: the VRAM spike that comes from pushing a batch of reference images through the VAE mostly goes away.
• Prompt hygiene: identity lives in the file, so the prompt stays readable.
• Multi-character work: several packs can be applied without turning the prompt into a list of image slots.

The important detail is the third and fourth points, and they are really one argument. Once identity is a file rather than a prompt fragment, the prompt goes back to describing what the shot is doing.

It is not training, and that distinction has teeth

The framing in the community is "LoRA without the training," which is close enough to be useful and far enough to be dangerous.

RefMod is not a weight-level fine-tune. There is no training pass over the DiT, and the creator is explicit that pack creation is not an overnight job. That has a practical upside: the feedback loop collapses from hours to minutes, so you can rebuild a pack five times in an afternoon while you figure out which source photos belong in it.

It also explains something that would otherwise look like a bug. Because there is no training, the familiar "train on faces versus train on vibe" dial does not exist. In the creator's account, whether a pack leans toward facial detail or toward overall atmosphere comes down to two things: the storage mode (full or compressed) and which source materials you fed in. If your pack keeps coming out moody and soft, that is a dataset problem, not a mysterious algorithm setting.

And the honest limit: a 1.5MB latent holds less information than an adapter trained on thirty images for several hours. Expect it to be faster and cheaper, not stronger. My own expectation is that it degrades first on hard angles, strong expressions, and fine detail like jewelry or a specific hairline, which is exactly where a trained adapter earns its cost. Test that on your own face packs rather than trusting the general claim, in either direction.

Why text-to-video plus an anchor beats reference-to-video

The part I find most useful is not the file format. It is the division of labor.

With image-to-video, or with a single reference image or reference video fed into a reference-to-video mode, the reference locks more than identity. It drags composition and mood along with it, and your prompt gets squeezed into describing a shot that has effectively already been decided. The creator's framing is that T2V plus RefMod splits the job cleanly: the face anchor is a file, and the performance, camera, and location are prompt text.

That split is not free. T2V gives you less control over the first frame, so you get more variance in framing. The trade is deliberate: trading guaranteed framing for a free camera, and holding identity across all of it.

If you are choosing between anchoring strategies for a series, the shape of the decision looks like this:

• Describing a character in prompt text is free and unreliable. Fine for background cast, hopeless for a hero character who must match across twenty shots.
• Reference images per run are accurate and expensive per run, and they tend to lock composition. Good for one-off edits and stills.
• A trained LoRA is the strongest identity hold available, and it costs hours per attempt plus rebuilds whenever the base model or its VAE changes.
• A cached latent adapter sits between the last two: minutes to build, cheap per run, weaker fidelity than a full adapter, and easy to throw away and rebuild.

For episodic work where the cast must survive wardrobe changes and camera moves, the middle option is usually the right default.

The workspace split is worth copying

Alongside the technical trick, there is a workflow decision worth stealing: keep creation and generation in two separate graphs.

The Create graph does one job. Point it at a folder of photos, choose how many to use, choose a save name, and it writes the pack. The Generate graph does the other job: load the pack, take a plain-language instruction, and produce video. The creator puts a small vision-language model in the middle of the Generate graph so the instruction gets formatted into H3's expected prompt structure, with the pack name woven in automatically.

Mixing both jobs into one graph is how a working setup turns into a graph you can no longer debug at 1am. There is also a community node covering a different job, generation tweaks for H3 in ComfyUI, ComfyUI-Fizgig-H3-Tweaks, which adds skin detail and contrast controls, a scene re-roll and a prompt-strength dial. The creator mentions it as an available helper but prefers building and applying packs from inside ComfyUI. A handful of other auxiliary nodes (Inspect, Axis, Step Curve, Bridge) are mentioned but were explicitly not verified by the author, so treat those as unexplored rather than recommended.

One disclosure: the hands-on steps and the workflow files in both articles sit behind the creator's paid membership. The technique description is public, the graphs are not. Everything above is from the public portion, so if you want to try it, expect to build the graph from the description.

Verify the pack before you build a series on it

A face anchor is an asset, and assets need acceptance tests. Before you commit a 20-shot project to a pack:

1. Fix the seed, the prompt, and the resolution. Render three different scenes with the raw reference upload and then with the pack applied.
2. Measure the peak VRAM and the wall clock for both paths. The speed claim is about the reference-encode step, so the gap should show up on longer clips and wider frames, not on short tests.
3. Put the two sets side by side at 100% and look at skin texture, the hairline, and any small accessories. Loss in a compressed latent shows up in fine detail before it shows up in the silhouette.
4. Blind-test identity: send both sets to someone who has not seen the character and ask a single question per frame, same person or not.
5. Rebuild and re-compare after any base model, VAE, or node update. A latent is only meaningful to the decoder that produced it, so a pack built against one revision is an assumption about another.

Do that once and you have a number instead of a feeling, which is what you need when a client asks why the character's face changed in episode three.

Two loose ends to leave loose

Audio is the first one. Referencing audio through a pack is possible in the node graph, but the creator's own recommendation for voices is to use reference audio on the T2V side rather than leaning on an audio pack. Voice identity and face identity have different failure modes, and bundling them into one file means a bad voice take forces a rebuild of the face too.

The second is where the pack is in the pipeline. Face refinement and upscaling are separate passes that come after generation, so a pack does not replace the face refinement pass or the finishing chain. It removes repeated encoding work in front of the render, and that is all. If you are still choosing your H3 speed levers, the ComfyUI speed guide covers quantization, step count, and caching, which stack with this. And if you have decided that hours of training is worth it after all, the LoRA dataset strategy still applies.

The reason this technique spread quickly is not that it beats a trained adapter. It is that it turns identity into a small file you can keep, version, and rebuild in the time it takes to make tea, which fits how most creators actually iterate.