World Models Predict Consequences. Video Generators Predict Appearances.

A video model asked for a glass falling off a table will give you a convincing glass, a convincing table, and a fall that looks right at normal speed. What it will not reliably give you is the state of the floor afterwards. The glass may shatter into pieces that do not add up, or land intact, or leave no trace in the next shot.

That gap is the difference between predicting appearance and predicting consequence, and it is the reason the term world model keeps appearing in research papers that have nothing to do with making pictures.

What the reporting actually says

Heise's technology desk published an explainer on world models this week, and the framing is useful. These systems learn from video and 3D simulation to build an internal picture of how the world behaves, so that they can predict not only what a scene looks like but what follows from an action taken inside it. The practical targets named are robots and self-driving cars, which can rehearse ordinary and dangerous situations in simulation instead of on a road. The same piece is candid about maturity: predictions are often hard to follow or verify.

Thomas Brox, who leads pattern recognition and image processing research at the University of Freiburg and trains world models himself, makes the historical point. The concept is recent in computer science and old in biology and neuroscience, where it describes the internal representation an animal uses to anticipate and act. A creature builds that picture from senses and experience, and only a system that knows what its actions cause can plan toward a goal.

He also gives the cleanest comparison available. AlphaGo had something like a world model, but its internal representation stopped at the edge of a Go board. The real world is too complicated to capture in a few rules, which is why current work learns the model from data instead of writing it down.

Where a creator feels the difference

For anyone generating images and video, this sounds like a research problem happening somewhere else. It is not, and the place it shows up is specific: anywhere the camera implies space that the model has to keep track of.

Occlusion is the simplest case. A character walks behind a pillar and comes out the other side. An appearance model reconstructs a plausible person on exit. A model with a persistent scene representation knows it is the same person, in the same clothes, still holding the same object. The first can look fine in a single shot and break the moment you cut back.

Camera movement is the second case. A slow orbit around a product implies a continuous object with a back. Models that only ever saw the front three-quarters will invent something for the reverse angle, and the invention is usually visible at the edges: a seam where two surfaces meet, a label that changes spelling, a handle that appears on both sides.

Physical consequence is the third, and the one that ruins otherwise good shots. Liquid poured into a glass that then does not rise. Cloth that moves without weight. A stack of objects that falls upward in the frame because the motion prior liked that shape better. None of these are failures of resolution. They are failures of a model that was never asked to represent what happens next, only what the next patch of pixels resembles.

A test you can run in an afternoon

You do not need benchmark access to find out which behaviour you are getting. You need pairs of prompts that differ in exactly one causal variable, and a rule for scoring them.

Take a still or a short clip where the causal chain is unambiguous. Then ask for the same shot twice: once with a change that should alter the outcome, once without. A ball rolls toward a cup and stops a hand's width short, versus the same ball rolling into the cup and knocking it over. Score an outcome the collision has to produce, such as the cup tipping or the ball changing direction. If neither happens in the second clip, the model is not tracking the collision, it is tracking the composition.

Run each pair on several seeds before you believe the result. One pass tells you about one sample, and a single convincing failure is as misleading as a single convincing success.

Do the same for occlusion, using a fixed camera and a subject that passes behind something. Count the frames where the subject is hidden and check the frame after it reappears for identity drift. Do it once more for a rotation, keeping the object identical and changing only the camera angle.

Score on the frame where it breaks, never on the best frame in the clip. That single rule will change your estimate of a model more than any leaderboard, because every model produces at least one beautiful frame and the useful question is what happens on frame 74.

Interpretability is a production problem too

The reporting notes that these predictions are hard to understand. That gets filed as a safety concern for autonomous vehicles, and it is. It is also a workflow concern for anyone shipping generated video, because an unexplainable failure cannot be debugged.

When a model fails visibly and you can name the reason, you fix the prompt, the reference, or the shot. When it fails for no stated reason, your only available action is to roll again and hope, and rolling again is the expensive part of the pipeline. This is the same argument behind our notes on seeds and why variation is not random: the value of a controllable variable is that it turns a slot machine into a decision.

A model that could tell you "the cup moved because I predicted a collision" would let you argue with the prediction. Right now the honest workflow is to assume nothing and verify everything that matters, which is why shot-level checks survive every model upgrade while prompt folklore does not.

What to do about it today

Do not design shots that require physics you cannot check. A hand closing around an object across a cut is a promise you have to keep in the edit, and if the model cannot keep it, the shot belongs in a different framing.

Put cuts where the physical state can reset. A cut to a new angle hides a cup that changed level. A continuous camera move does not. Editors have used this for a century, and it works on generated footage for the same reason.

Prefer reference-locked stills for anything that has to stay right. If the object matters, generate it as an approved image first, then animate from it. This is the core of the image-first workflow and it exists precisely because frame-level consistency is easier to enforce than sequence-level reasoning.

Keep a failure log per model, not per prompt. Three notes about what a model cannot do are worth more than thirty prompt variants that happened to work once.

The honest summary

Video generation has improved fast enough that the appearance gap between real and generated footage is now small in the right conditions. The consequence gap is where the interesting work sits, and it is being attacked from the robotics side by people who need a model they can plan against.

My guess is that the two lines merge rather than one replacing the other, because a model that can predict what a camera will see after it moves is useful for both a robot and a director. Until then, treat every generated sequence as an appearance until you have run a test that shows otherwise. The test is cheap. Being wrong on a client's deadline is not.