How AI video escaped the spaghetti era

In March 2023, Reddit user chaindrop posted a ModelScope-generated video of Will Smith eating spaghetti. The resemblance was close enough for people to recognize him, but the video was still very, very wrong: His face, the utensils, and the food changed shape as the video played, giving the clip an uncanny valley feeling. As a result, the clip became a meme, then an informal way to measure how much AI video generation had improved.

Eating spaghetti turned out to be a demanding test for AI. The model has to keep track of the person's face while their expression changes, maintain the shape of a fork as it moves, and represent strands of food that bend, overlap, and disappear behind other objects. It also has to get the sequence of events right: The spaghetti should reach the mouth before it disappears, and the person's jaw should move in a way that makes sense for taking a bite and chewing.

An image generator can produce a plausible picture without solving any of those problems. This gets at the core of what's harder about video generation: It requires maintaining relationships between objects as they move through time.

Three years later, AI video looks substantially better. The improvement owes at least something to diffusion transformers, an architecture introduced in a December 2022 paper by William Peebles and Saining Xie. But the architecture is only part of the story. Researchers also had to make generation computationally practical, develop representations that preserve motion, and improve the data used to teach models what happens between frames.

We wanted to understand how those developments fit together, and how much of the original spaghetti problem they actually solved. We also wanted to see AI video's progress firsthand, so we ran a new version of the spaghetti dinner prompt through three recent video models:

• Google's Gemini Omni 1.1 Flash

• Black Forest Labs' FLUX 3 Video

• MiniMax H3

The new clips are much more convincing. Looking closely at how they handle the fork, the spaghetti, and the movement of the person's mouth allows us to examine what these systems have learned, and what's still a challenge in AI video generation today.

First, learn to undo the noise

One of the foundations of modern image generation is an approach that starts with random noise and gradually turns it into an image. The model learns this process during training by solving the reverse problem: Given an image that has been corrupted with noise, predict the noise that was added.

Jonathan Ho, Ajay Jain, and Pieter Abbeel described a practical version of this approach in their 2020 paper, "Denoising Diffusion Probabilistic Models," or DDPM. By training on images at different noise levels, a network learns how to guide noisy inputs toward samples from the image distribution it has learned. During generation, the model begins with noise and repeatedly updates it using those predictions.

This introduces an important distinction when we extend diffusion from images to video. A diffusion model has its own sequence of computational steps, which isn't the same as the sequence of frames in the video it's generating.

Consider a model that uses 50 denoising steps to produce a five-second clip. Those 50 steps aren't 50 frames of the finished video. Each step may update a representation of the entire clip, including information about multiple moments in the scene. The video has a playback timeline, while the model has a separate denoising process that gradually constructs the result.

The distinction matters because denoising is expensive. Every sampling step requires computation through the network, so generating a video can involve repeatedly processing a large representation of many frames.

Researchers were already finding ways to reduce that cost. The 2020 paper "Denoising Diffusion Implicit Models," or DDIM, showed that models trained using the DDPM objective could use a different sampling process requiring fewer steps. That made generation faster and gave researchers more room to experiment with increasingly complex models.

Faster sampling helped, but it didn't resolve another basic problem: A video contains much more information than an image.

The cost of carrying every pixel

A five-second video at 24 frames per second contains 120 frames. At a resolution of 1280 × 720, that's 110,592,000 pixel locations, or more than 331 million values when we count the three color channels.

And those numbers only account for the raw video. The neural network also needs intermediate representations, and diffusion repeatedly processes its inputs during generation. Working directly on full-resolution pixels becomes increasingly expensive as resolution and duration grow.

Robin Rombach and colleagues addressed the corresponding problem for images in "High-Resolution Image Synthesis with Latent Diffusion Models," first posted in December 2021 and published at CVPR 2022.

Their approach separates image compression from the generative process. An encoder maps the image into a smaller, learned representation called a latent. The diffusion model operates on that representation rather than on the original pixels. After generation, a decoder reconstructs the image from the latent.

The advantage is computational: the denoising network processes a much smaller array of values. The tradeoff is that compression inevitably changes what information is available to the generator. The encoder has to discard some detail while retaining enough structure for the decoder to reconstruct a convincing image.

For still images, that might mean preserving the shape and appearance of a hand without representing every small variation in its pixels. For video, the encoder has to preserve information about how that hand changes position over time. If it loses a small object's shape or important information about its movement, the generative model may struggle to reconstruct those details consistently.

This is especially relevant to spaghetti. Thin strands repeatedly overlap and become partially hidden, while the fork moves and rotates. Compressing the scene makes it cheaper to generate, but any information lost during encoding may make those relationships harder to maintain.

Latent diffusion made high-resolution image generation more practical, but extending the same idea to video required researchers to make compression efficient without sacrificing the temporal information that distinguishes a moving scene from a collection of still images.

Teaching a model what happens between frames

Compressing video solves part of the computational problem. It doesn't, by itself, teach a model how motion works.

Suppose an image generator produces three individually convincing pictures of someone lifting a fork. The person's face might look slightly different in each frame. The fork might change shape, or the person's hand might move between positions without passing through the space connecting them.

Each frame could look reasonable on its own, but the sequence would still be wrong. A video model needs to learn relationships between moments, including which objects should remain consistent and how their positions and appearances can change.

Early video diffusion research approached this by adding temporal modeling to image generation architectures.

The 2022 "Video Diffusion Models" paper extended diffusion models with operations that work across both spatial and temporal dimensions. Spatial operations model visual information within frames. Temporal operations allow information from different frames to influence one another, giving the network a way to learn motion and consistency across time.

The researchers also trained the model using both images and videos. Still images provided useful information about visual appearance, while video examples supplied the temporal relationships that couldn't be learned from images alone.

Meta’s "Make-A-Video," published as a preprint later that year, explored a related approach to the training data problem. High-quality image-caption pairs were more readily available than comparable collections of captioned video. Make-A-Video used text-image data to learn visual concepts and their descriptions, then learned motion from video data without requiring corresponding text for every video example.

Its generation system combined spatial and temporal components with interpolation and super-resolution. The important contribution was that appearance, language, and motion could be learned from different sources and brought together in one system.

These approaches help explain what was happening in the original ModelScope spaghetti clip. The "ModelScope technical report" described a system derived from Stable Diffusion, with spatiotemporal blocks added to the denoising U-Net.

The 2023 model wasn't simply generating unrelated still images and stitching them into a video. It already included mechanisms intended to represent motion and maintain consistency between frames. The spaghetti clip exposed how far the results still fell short, even with those mechanisms in place.

What DiT changed

While researchers were extending diffusion models into video, another line of work was changing the network architecture used for denoising.

The December 2022 paper "Scalable Diffusion Models with Transformers," published at ICCV 2023, investigated whether transformers could replace the U-Net backbone commonly used in diffusion models. William Peebles and Saining Xie called their architecture the diffusion transformer, or DiT.

Transformers were already widely used in language modeling and other machine learning applications, following research that included the 2017 paper "Attention Is All You Need." Their attention mechanism allows a token to incorporate information from other tokens, including those at distant positions in the input.

Those tokens don't have to represent words. In DiT, they represent patches of a compressed image.

The researchers wanted to know how the architecture would respond to additional computation. They varied the transformer's depth and width, as well as the number of input tokens it processed, and evaluated the resulting image generation performance.

Across their experiments, increasing forward-pass computation was associated with better image-quality scores, measured using Fréchet Inception Distance (FID). The results showed that this transformer architecture could use additional compute effectively, whether through larger networks or finer patch representations.

That finding was significant because scaling was becoming a central question for generative models. DiT gave researchers evidence that transformers could be a productive backbone for diffusion, with measurable improvements as they increased model computation.

There was an important limitation to the paper's findings, though. Peebles and Xie were working with class-conditioned ImageNet images, not video. They showed that the transformer architecture scaled effectively for image generation, but didn't establish whether it could maintain temporal consistency across frames.

What DiT provided was a promising architecture. The next challenge was applying it to representations that included time as well as space.

From image patches to spacetime patches

In February 2024, OpenAI published a technical report describing Sora. The report connected diffusion transformers to video generation: Sora compressed video across spatial and temporal dimensions, divided the compressed representation into spacetime patches, and used a diffusion transformer to process them.

An image patch represents a region of an image. A spacetime patch represents a region of a compressed video with temporal extent, so its contents can include information spanning multiple frames.

This gives the transformer a way to process relationships across a clip. Information about a hand, fork, and bowl can be represented in tokens that account for their positions at different moments, rather than treating each frame as a separate image.

Sora's patch-based representation also allowed the researchers to work with videos of different durations, resolutions, and aspect ratios. By arranging the patches into grids appropriate to the input, the system could train on varied video formats. The report described improvements in sample quality as training compute increased, extending the scaling discussion from DiT into video.

OpenAI didn't publish a complete implementation, so the report can't tell us every detail of Sora's architecture or training process. It does establish a connection between the earlier image generation research and a video generation system that operated on spacetime patches.

For the spaghetti test, the potential advantage is clear: The model can work with information about the fork before, during, and after it reaches the person's mouth. Its predictions can account for those moments together.

That doesn't necessarily mean it will preserve the fork's shape or put the spaghetti in the correct place. The architecture makes relevant temporal information available to the model, but the model still has to learn how objects and actions behave.

One prompt, three dinners

We wanted to see how newer generators handle the same kind of action. So, on September 17, 2026, we submitted an identical text-only prompt to three models:

1. Gemini Omni 1.1 Flash

2. FLUX 3 Video

3. MiniMax H3

Rather than recreating the Will Smith clip, we asked each model to generate an anonymous chef eating spaghetti in a restaurant. Our prompt asked for one continuous shot, beginning with the chef twirling spaghetti around a fork and ending after the first bite.

The instruction to create one continuous shot is important. With no cuts, the model has to represent the action from start to finish without changing camera views to hide transitions. Keeping the other hand visible beside the bowl also gives us an additional object relationship to examine.

We made one submission per model and kept the returned files without trimming or selecting a favorite from multiple attempts. That helps document what happened, but it's important to note that three individual generations can't establish how reliably these models would perform across repeated trials.

The Gemini clip spends more time establishing the action. It opens with the hands and bowl in view while the fork turns through the spaghetti. The fork then rises toward the person's face, and chewing begins near the end of the shot. The contact points between the fork, strands, and mouth are worth watching closely.

FLUX presents the action in a shorter, more tightly framed clip. The fork gathers spaghetti and reaches the person's mouth, with the camera staying close to the bowl and lower face. The important transition to examine is what happens to the strands as they move behind the lips.

MiniMax uses a wider view, keeping the diner's face, the bowl, and the resting hand visible. The diner lifts the fork, leans forward, and draws in the hanging strands. This framing gives us more of the scene to track during the bite, particularly the positions of the diner and objects around the bowl.

Exact prompt and generation record

We used the following prompt for all three generations:

The prompt was submitted once to each model, without reference images, retries, or subsequent editing. The model identifiers were:

All three systems returned files with generated audio. Google describes its 1080p output as upscaled. FLUX returned a file measuring 1920 × 1088 for its 1080p setting, which we retained without cropping. MiniMax returned 6.58 seconds for a six-second request.

We preserved the files as received. Because the models returned different durations and supported different settings, this isn't a controlled comparison of model performance. The clips show what happened in three documented generations, including differences in framing and how much of the requested action each model completed.

The video compressor deserves more credit

Diffusion transformers helped establish a scalable architecture, but their effectiveness still depends on the information available in the tokens they process.

For video, that makes the encoder and decoder particularly important. An encoder that compresses video efficiently but discards too much motion detail can create problems that a larger transformer won't necessarily solve. Similarly, the decoder has to reconstruct a sequence that looks consistent, not merely a series of individually convincing frames.

The 2024 "CogVideoX" paper described a three-dimensional variational autoencoder (VAE) designed to compress video along both spatial and temporal dimensions. Its representation reduced each spatial dimension by a factor of eight and the temporal dimension by a factor of four.

Those reductions substantially shrink the grid that the generative model has to process. They don't imply an equivalent reduction in file size, since the resulting latent representation has its own channel dimensions.

The CogVideoX researchers also investigated how compression affected reconstruction quality. They reported that their approach helped reduce reconstruction flicker, an important concern when the decoder must produce successive frames that appear to belong to the same sequence.

There's a limit to how much detail can be removed. Small objects, subtle movements, and changes in contact between objects may be particularly sensitive to compression. If the latent representation doesn't retain enough information about these events, reconstruction can introduce inconsistencies even when the broader scene looks correct.

The 2025 Wan technical report offers another example of why improving video generation involves more than changing the denoising network. Wan combined a video VAE and diffusion transformer with extensive data preparation and a staged training process that progressively introduced higher resolutions and longer sequences.

Training data matters because a model needs examples of the particular relationships it is expected to generate. Consider the difference between a caption that says “a person eating spaghetti” and one that identifies which hand holds the fork, how the person lifts it, and when the food reaches their mouth.

The second description gives the model more specific information about the action and its sequence. Better captions can therefore improve instruction following, provided the training videos actually represent the described events clearly.

The improvements documented by systems such as CogVideoX and Wan come from work on several connected components: the representation, the generative model, the training process, and the quality of the data. It's difficult to isolate the contribution of the transformer without considering what the rest of the system makes possible.

Another change to the generation process: Flow matching

The early diffusion research we discussed involved learning to reverse a noise process. Later generative systems have also adopted related methods that formulate the problem differently.

The 2022 paper "Flow Matching for Generative Modeling" described an approach that learns a vector field for moving samples along a path from a simple distribution, such as noise, toward the data distribution.

Instead of predicting noise at a particular diffusion step, a flow-matching model learns a direction of movement along the chosen probability path. During sampling, that learned field can be used to evolve an initial noise sample toward a generated result.

The framework supports different paths, including paths related to diffusion. It gives researchers another way to formulate and train generative models without requiring a different neural network architecture.

Wan, for example, uses a transformer backbone together with flow matching. These are separate engineering choices: The transformer determines how the network processes its input, while the training objective determines what the network learns to predict.

It's important to keep that distinction clear when comparing systems. Two video models may both use transformers over compressed video representations while differing in their objectives, sampling procedures, and training recipes. Those differences can affect their performance even when their architectures share important features.

Now the fork has to sound right, too

Until recently, much of the progress in video generation centered around the visual sequence. Adding generated audio introduces another set of relationships that the system has to maintain.

The sound of a fork touching a bowl should occur at the moment of contact. Footsteps have to align with movement. If a person speaks, the audio needs to match their mouth movements as well as the timing of the scene.

These requirements are related to temporal consistency, but they involve two different kinds of information. A model generating sound and images together needs a way for events represented in one modality to influence the other.

The January 2026 paper "LTX-2: Efficient Joint Audio-Visual Foundation Model" describes one approach. LTX-2 uses separate transformer streams for audio and video, connected through bidirectional cross-attention. That allows the video stream to incorporate information from the audio representation and vice versa.

The system also uses temporal position information to help align the two streams. Audio and video are encoded separately, but the model can learn relationships between them during generation.

Commercial models now support increasingly sophisticated combinations of video, audio, and editing controls:

• On August 4, Black Forest Labs released FLUX 3 Video with native audio generation and support for clips up to 20 seconds.

• MiniMax introduced H3 on July 31, describing a multimodal system capable of generating video with native stereo audio. Its technical overview also describes a new VAE and an omni transformer.

• Google’s August 27 Gemini Omni 1.1 Flash update added features including scene extension, start- and end-frame controls, and video references. These controls let users specify more about what should happen in a shot and supply context for subsequent generations or edits.

We shouldn't assume that all these commercial systems use LTX-2's design, or that they share the same approach to audiovisual training. Product releases tell us which capabilities are available. Their internal architecture can only be established to the extent that developers publish technical details.

Google’s Omni model card, for example, describes a transformer-based multimodal model without disclosing a complete training and implementation recipe.

Still, the range of supported tasks shows how much more demanding video generation has become. A model may now need to maintain an action across a longer clip, produce appropriate sound, and respond to edits that change part of a scene without unintentionally changing the rest.

That makes temporal consistency harder to evaluate, even as the resulting generations become more convincing.

One excellent dinner is still just one dinner

We chose three strong video generation systems for our spaghetti test. In Arena’s September 4, 2026 text-to-video leaderboard snapshot:

• Gemini Omni 1.1 Flash ranked first

• FLUX 3 ranked fourth with a preliminary score

• MiniMax H3 ranked eighth

Several models had overlapping rank intervals, indicating that their relative positions weren't necessarily clear-cut. Those rankings came from Arena's broader preference comparisons. They aren't results reproduced by our three generations, and the leaderboard can change as new comparisons are collected.

Our test was deliberately small. It provides a concrete example of what each model produced, along with the prompt and generation settings, but it doesn't tell us how often any model would produce a convincing result.

With a prompt as memorable as the spaghetti test, that's an important distinction. It's easy to compare two impressive clips without knowing whether one came from the first attempt and the other was selected from dozens. Resolution, duration, prompt wording, reference images, and editing can also affect the result.

A more systematic evaluation needs to separate the different aspects of video quality. The VBench benchmark was developed for that purpose, evaluating 16 dimensions that include subject consistency, motion smoothness, temporal flicker, and spatial relationships.

A model might produce exceptionally sharp images while struggling with object consistency. Another might handle movement well but fail to follow details in the prompt. A single realism score can conceal those differences.

For a repeatable spaghetti evaluation, we'd start by recording the complete generation process: exact model versions, prompts, settings, number of attempts, and unedited outputs. Since current models also generate audio, we'd evaluate whether the sound follows the events on screen.

Then we'd vary the requested action. Eating a forkful of spaghetti is one test; putting the fork down, pausing, switching hands, or returning to the meal creates different problems. Those actions require the model to preserve objects and their relationships across longer sequences.

Keeping unsuccessful outputs would be essential. Repeated generations could tell us whether a convincing result is typical of the model or an occasional success.

Realism can improve faster than physical consistency

The original spaghetti clip was visibly, uncomfortably wrong. Its objects changed shape, the eating motion was implausible, and the errors were impossible to miss.

Newer generators are much better at producing convincing appearances, but a realistic-looking scene can still contain physical inconsistencies. A hand might pass through an object, a strand of spaghetti might disappear before reaching the mouth, or a liquid might change volume without being poured.

These problems involve how objects behave and interact over time. Improving image sharpness doesn't necessarily fix them.

The "VideoPhy" benchmark evaluated physical commonsense in generated videos, including interactions among solids and fluids. "VideoPhy-2" expanded that work to 200 actions and assessed whether generated clips both followed their prompts and respected expected physical behavior.

The results showed how difficult those requirements remained. On VideoPhy-2's hard subset, the strongest model tested achieved only 22% joint performance on semantic adherence and physical commonsense. Conservation of mass and momentum were among the challenging areas.

That result comes from the models and evaluation procedure used in the study. It isn't a measurement of the September 2026 systems in our gallery. But it demonstrates why researchers have begun evaluating physical relationships separately from visual realism.

Google’s August 2026 Omni model card also identifies complex motion and consistency across edits as limitations. As models gain more controls, users can ask them to perform more complicated actions and test whether the resulting sequences remain coherent.

The architecture alone can't guarantee that they will. A diffusion transformer can exchange information between distant parts of a video, including moments before and after an object becomes occluded. It doesn't automatically enforce a conservation law or require the model to account for where every object has moved.

Those behaviors have to emerge from what the model learns, and their reliability has to be established through testing.

That suggests what the next version of the spaghetti test should be:

1. Keep the camera fixed and use an unfamiliar subject performing an uninterrupted action.

2. Track the objects and their contact points through the sequence.

3. Repeat the generation enough times to see which failures recur.

4. Introduce a second action that depends on the first. For example, have the person put down the fork, pause, and pick it up again.

Does the fork remain on the table while the person's hand is elsewhere? When they pick it up, is it the same fork, in a position consistent with where it was placed?

These are more demanding questions than whether a single bite looks realistic. The next meaningful improvement in generative video may be easier to recognize by watching what happens between two dependent actions than by comparing another pair of polished frames.