VideoCoCo fixes AI video's broken physics by thinking in Blender code

Watch a text-to-video clip long enough and you will spot the tell. A cup tips, a cloth folds, a ball bounces, and somewhere near the third second the pixels stop obeying physics. The frames look almost photoreal. The dynamics often are not. The paper behind the latest round of benchmark numbers argues this is not a rendering problem. It is a reasoning problem, and video models are not the only ones wrestling with it: even the best VLMs fail at 3D action.

VideoCoCo, posted to arXiv on July 29 under Computer Vision and Pattern Recognition, starts from a specific diagnosis. A text-to-video model has to infer the temporal evolution of a scene implicitly from a highly compressed text prompt, and that implicit inference is where physical consistency dies. The paper also explains why earlier chain-of-thought attempts did not fix it. The intermediate plans and visual states they produce are typically non-executable or temporally sparse. A model can gesture at what should happen. Nothing runs it.

Making the chain of thought executable

VideoCoCo is an agentic dual-engine framework, and the design only makes sense if you track which part does what. A coding agent takes the prompt and synthesizes a Blender program that explicitly specifies the scene and its evolution over time. An executable simulation engine runs that program to produce a deterministic spatiotemporal draft, one concrete version of the motion resolved before any pixels are drawn. A generative video engine then transforms the draft into a photorealistic clip through draft-conditioned editing. The coding agent does not describe the scene in words and hope. It writes a program that states what exists and what changes.

The point is the decomposition. Process-level reasoning, the decisions about how the scene behaves, lives in executable code and can be run, inspected, and corrected like any program. The film-like look belongs to the generative engine. Neither half has to be good at the other half's job. The generative engine no longer has to invent physics from pixels. It has to make an honest simulation look like cinema, which is a narrower and more tractable task. This split has precedent beyond video: the TRACTA benchmark found neuro-symbolic systems beating raw neural models on temporal reasoning tasks.

To get the video engine comfortable with simulated input, the researchers built VideoCoCo-3K, a curated dataset of draft-instruction-target triplets that adapts the editor to the drafts the simulator produces. The entry currently shows 64 upvotes on Hugging Face.

Reading the scoreboard

Then the numbers. On PhyGenBench, VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558. On VBench-2.0, it moves from 52.18 to 77.88, a 25.7-point jump, and the paper reports the best average score on both benchmarks.

The scores live on different scales, PhyGenBench in fractions and VBench-2.0 in points, so the two gains should not be compared directly. Benchmark numbers also deserve one footnote before they harden into headlines. These suites test a defined slice of behavior, and the slice can flatter a model. In coding, top models post 96% on SWE-bench Verified but barely clear 23% on private enterprise code, per the SWE-bench enterprise gap study. The results are still one team's numbers from its own evaluation, waiting on independent replication. But the jump shows up on both suites, which is not nothing. It gives the core claim room to be taken seriously.

The structural bet is more interesting than the score. If a simulator resolves the physics, the practical failure mode of AI video changes. Instead of a model that cannot reason about the world, you get a draft you can debug. The same prompt yields the same simulation, and editing the Blender script becomes a way to steer the result. Deterministic and inspectable is a genuinely different operating point for text-to-video.

What the abstract leaves open

Three gaps are worth naming. The pipeline inserts a full simulation stage ahead of generation, and the abstract says nothing about what that costs in compute or latency. Blender drafts are deterministic geometry, and whether a video engine trained on them can still reach expressive or stylized motion is unproven. And the results are one group, two benchmarks, no independent check yet. That is how most good results start. The wider agentic-AI record follows a similar curve: across the 957,253-record Messier corpus, agents surge in coding but stall where enterprises need them.

VideoCoCo's wager is that coherence in AI video runs through a 3D suite, not through more adventurous prompting. On its own benchmarks, the wager paid off. Whether it holds beyond them is a question for replication, and that is the one worth watching.