Google just proved why diffusion models invent, not just copy

Diffusion models have become a go-to architecture for everything from high-resolution image synthesis to molecular discovery. But they do something that has long vexed researchers: they produce outputs that are clearly new, not just copies of training examples, while staying coherent and realistic. A flurry of recent tools, from video generation to multimodal models, suggests novelty is becoming the norm, and a mathematical explanation was overdue.

A study from Google Research, presented at ICLR 2026, gives the first rigorous mathematical explanation for this behavior. The paper argues that what we call creativity in diffusion models is a direct, predictable result of how neural networks learn approximate score functions under regularization.

"Our work suggests that what we call the 'creativity' of diffusion models might actually be a predictable mathematical result," said Zhengdao Chen, the study's lead author and a research scientist at Google Research.

The denoising framework and the memorization trap

Understanding the finding requires a look at how diffusion models work. The training process starts by gradually adding noise to real data, like a set of cat photos, until the images become pure noise. The model then learns to reverse this corruption step by step, in a process called denoising.

If the model learned the perfect reverse process, it would reconstruct exact copies of the training images during deployment. That is memorization. In that scenario, the model becomes a retrieval tool, not a generative engine.

Yet in practice, diffusion models do more. They generate images the model has never seen, combining features from multiple training examples. The question is: what stops the model from just memorizing?

The role of score smoothing

Chen and his colleagues show the answer is in how neural networks approximate the score function, the mathematical force field that guides noisy particles back toward meaningful data during the denoising process.

The perfect score function, derived directly from the training data, would be extremely sharp. It would divide the data space into steep cliffs, each particle pulled forcefully toward the nearest training point. That leads to memorization.

But neural networks, even when they try to learn this perfect function, cannot reproduce those sharp cliffs exactly. Training with regularization, such as weight decay or the implicit regularization from gradient-based optimization, forces the network to learn smoother versions of the score function.

This score smoothing has a concrete effect: it creates a dead zone between training points where particles flow slowly, eventually settling in the space between known data. Instead of collapsing onto training points, the generative process lands on interpolated points, new data that combines characteristics of multiple training examples.

Mathematical proof and empirical validation

To demonstrate this effect, Chen's team ran controlled experiments in a one-dimensional setting with only two training data points: +1 and -1. They trained two-layer ReLU networks to learn the score function under different degrees of weight decay.

The results were clear: stronger regularization produced smoother learned score functions, and the resulting denoising process produced points that fell in the interpolation zone between -1 and +1, rather than at either pole. The team quantified this connection by combining function-space theory of neural network regularization with the mathematics of denoising.

The phenomenon extends to high-dimensional data. In multi-dimensional settings, score smoothing becomes direction-dependent. Along directions tangential to the hidden data manifold, it creates the same slowing-down effect seen in the one-dimensional case. But along directions pointing toward the manifold, the perfect score function is already smooth, so further smoothing makes little difference. This directional nuance has implications beyond theory, it touches how models are deployed in the real world, where both quality and novelty matter.

"Score smoothing does not slow down movement toward the manifold, but only reduces the tendency to collapse toward the training data along the tangential directions," Chen explained. "In this way, the model achieves a balance between quality and novelty."

Implications for generative AI design

The findings have practical implications for developers of diffusion models. By treating creativity as a controllable mathematical outcome rather than a mysterious property, researchers can design architectures that deliberately tune the interpolation effect.

The paper's framework suggests the trade-off between memorization and novelty can be managed by adjusting the degree of regularization during training. Models trained with heavier weight decay will interpolate more aggressively, generating more novel outputs but risking blurriness if the interpolation zone grows too large. Models trained with minimal regularization will produce sharper outputs that more closely resemble the training set.

For applications such as drug discovery, where generating novel molecular configurations is the goal, the paper provides a theoretical foundation for encouraging interpolation. In image generation, the findings could guide the development of models that are guaranteed to produce novel but coherent outputs. The controlled novelty the paper describes echoes similar trade-offs in balancing speed and quality in AI development.

Limitations and open questions

The current work focuses on relatively simple architectures and controlled settings. Whether the same mechanisms scale to state-of-the-art diffusion models with billions of parameters remains an open question.

"Our work only serves as an initial effort toward elucidating this mechanism, and it remains to be seen what happens when the data distribution or the neural network architectures grow more complex," Chen noted.

Nonetheless, by providing a rigorous mathematical foundation for the interpolation effect, the study opens the door to purposefully building models that are better interpolators. "We can begin to intentionally build models that are better interpolators to ensure they remain creative engines while avoiding the pitfalls of blind memorization," Chen added. That sentiment aligns with the broader push to make AI systems measurable and accountable.

The paper and associated code have been released, allowing the research community to replicate and extend the findings.