When AI agents rebuilt the internet's most creative experiment, they hit a wall humans never see

Open-endedness is the holy grail of artificial creativity: a system that keeps generating novel, meaningful artifacts without converging on a single solution. Natural evolution does it. Human culture does it. AI systems mostly do not. A new paper from researchers at Sakana AI, NYU, and MIT, accepted at GECCO 2026, goes straight at this problem by replicating Picbreeder, the canonical human-driven evolutionary art platform, with vision-language models standing in for its human users. The study pinpoints where even frontier models lose the thread, and the results echo findings from other recent work on agent collapse, as the original Picbreeder reconstruction documented.

The VLM-generated archives are flatter, more repetitive, and less evocative than the human originals. But the study, led by Sam Earle (NYU) with Kai Arulkumaran, Andrew Dai, Akarsh Kumar, Julian Togelius, and Sebastian Risi, also systematically tests what interventions might close the gap. It is one of the more rigorous attempts to date to isolate the ingredients of open-endedness in machine systems, similar in spirit to the multi-agent revision loops explored in ARCANA's ARC-AGI-2 solver.

What Picbreeder measured

Picbreeder, launched in 2008, let users collaboratively evolve images by selecting and mutating CPPN-generated pictures, branching from each other's creations. Over years, the community built a rich phylogenetic tree of published images, faces, cars, insects, abstract patterns, that stands as a benchmark of human-driven open-ended search. The dataset compiled by Kumar et al. in 2025, with 9,758 published images and full ancestry, gives the Sakana team a direct human baseline to measure against.

The core experiment replaces each human user with a VLM agent (defaulting to Gemini 2.5 Pro after testing several models including Gemini 3 Pro, Qwen3-VL variants, and flash models). The agent sees a population of 15 CPPN images, selects parents over 20 generations, and publishes a final image to the shared archive. Branches start from a sample of the existing archive. Ten agents run in parallel. The paper calls this a faithful replication of the conditions of the original experiment, not an attempt to replicate its results.

The gap between human and VLM output is visible at a glance. The human archive has more variety and sharper, more intentional images. VLMs produce many near-duplicates, tend toward mode collapse, repeating a handful of forms, and often publish gray, abstract, or high-frequency noise. The paper tries to quantify this with four metrics: Semantic Recall (how well the archive covers known object classes from the THINGS dataset), Visual Coverage and Semantic Coverage (k-covering radii in embedding spaces), and Tree Balance (the J1 index of phylogenetic balance). The human baseline leads on all three coverage metrics and ties on recall. Random selection is the worst. This pattern of collapse under open-ended conditions mirrors the reward self-sabotage documented in the RL alignment trap.

Three interventions, three lessons

The team varied three parameters: exploratory noise (epsilon-greedy selection), context length (how much prior interaction history the VLM sees), and the number of distinct agent personalities generated via LLM-prompted traits.

Exploration vs. quality

Without noise, VLM archives suffer severe mode collapse, many variations of the same fox- or fishbone-like forms. Injecting epsilon-greedy noise up to 0.25 improves diversity without sacrificing recall, but larger noise levels degrade image quality. Even at epsilon=1, where the VLM only handles branching and rating, with all selections random, the archive still beats random baseline on several metrics, suggesting the VLM can steer meaningfully even with minimal control. But Tree Balance remains stubbornly low: VLMs branch from the same images repeatedly rather than starting from random populations.

Memory is a double-edged sword

Context length zero (no history) collapses recall as agents publish duplicates. Context of 1, seeing the current turn and the prior one, recovers performance well. But scaling context to 2, 10, or the full 20-generation session consistently degrades recall and coverage metrics. With full context, agents sometimes produce near-photorealistic soda can images across multiple seeds, but the overall archive becomes noisier. The authors suggest information overload and auto-sycophantic loops, the VLM reinforcing its own stated preferences rather than exploring broadly. This mirrors known issues in long-context agents, as explored in M3's long-context coding experiments.

Many agents, strange results

Scaling the number of distinct prompted personalities from 1 to 1,000 increases Semantic Coverage and Tree Balance significantly. The highest Semantic Coverage of any setting comes from the 1,000-agent condition. But the archive also fills with high-frequency, uninterpretable psychedelic patterns that make up 10 to 20 percent of publications, what the paper calls adversarial images, massaged by multiple agents pulling in different directions. The collaborative friction that might push agents toward bold leaps instead pushes them toward noise.

The 1,000-agent grid is the most clearly open-ended in quantitative terms, and simultaneously the most alien in qualitative terms. That tension may be the paper's deepest result: metrics designed to capture human-valued diversity do not distinguish between a refined novel form and an adversarial glitch that happens to map to many caption embeddings. The authors flag this as a direction for future work, proposing a semantic variance metric, how much the caption of a given image varies across repeated VLM captions, to separate the two. The challenge of designing metrics that truly capture creative value echoes the findings of the HSCodeComp benchmark gap between agents and human experts.

The Sakana angle

Sakana AI's involvement is not incidental. The Tokyo-based lab, co-founded by David Ha and Llion Jones, has built its research agenda around biologically inspired, open-ended systems, from evolutionary model merging to error-diffusion learning, as seen in the CIFAR-10 ablation study. This Picbreeder study fits squarely into that program: it treats open-endedness not as a property you measure after the fact but as a set of design decisions you can probe by replacing humans with models.

The paper is careful not to claim its system is open-ended. It reports what makes the VLM-driven system less open-ended than the human version, and what design choices push it closer. That is a useful contribution. The results suggest that no single fix, more noise, more memory, more agents, gets you human-like open-endedness; the combination may matter, and the paper explicitly calls for testing interventions together at larger scales.

What it means for automation

The study's broader implication is for any project that automates creative or scientific discovery with AI. If VLMs, even frontier ones, produce narrower search than humans when put in the same minimal structure, then replacing human judgment in open-ended pipelines may require careful scaffolding. The Sakana team shows that scaffolding matters: the right amount of memory, the right injection of noise, the right diversity of perspectives. But it also shows that scaffolding alone does not yet replicate human boldness, that quality the paper calls the x-factor, involving bigger semantic leaps and sharper discernment. For practitioners building agentic systems, the lesson is that open-endedness is not a model property but a system property, a point reinforced by Nvidia's evidence that better embeddings pay off in agent runtime.

The Sakana team's code is available on GitHub. The data from the original Picbreeder experiment remains one of the richest behavioral traces of human-guided evolution. Connecting the two, as this paper does, is a step toward understanding what we mean by creativity in the first place.