Sakana AI's Picbreeder reboot shows what creativity metrics miss

Open-endedness is a core goal in artificial creativity: a system that keeps generating novel, meaningful artifacts without converging. Natural evolution does it. Human culture does it. AI systems mostly do not. A new paper from researchers at Sakana AI, NYU, and MIT, accepted at GECCO 2026, replicates Picbreeder, the canonical human-driven evolutionary art platform, using vision-language models in place of human users. The study identifies where even frontier models struggle, echoing findings from other work on agent collapse, as the original Picbreeder reconstruction documented.

The VLM-generated archives are flatter, more repetitive, and less evocative than the human originals. The study, led by Sam Earle (NYU) with Kai Arulkumaran, Andrew Dai, Akarsh Kumar, Julian Togelius, and Sebastian Risi, also tests what interventions might close the gap. It attempts to isolate the ingredients of open-endedness in machine systems, similar in spirit to the multi-agent revision loops explored in ARCANA's ARC-AGI-2 solver.

What Picbreeder measured

Picbreeder, launched in 2008, let users collaboratively evolve images by selecting and mutating CPPN-generated pictures, branching from each other's creations. Over years, the community built a phylogenetic tree of published images, faces, cars, insects, abstract patterns, serving as a benchmark of human-driven open-ended search. The dataset compiled by Kumar et al. in 2025, with 9,758 published images and full ancestry, gives the Sakana team a direct human baseline.

The core experiment replaces each human user with a VLM agent, defaulting to Gemini 2.5 Pro after testing Gemini 3 Pro, Qwen3-VL variants, and flash models. The agent sees 15 CPPN images, selects parents over 20 generations, and publishes a final image to the shared archive. Branches start from a sample of the existing archive. Ten agents run in parallel. The paper calls this a faithful replication of the conditions of the original experiment, not a replication of its results.

The gap between human and VLM output is visible at a glance. The human archive has more variety and sharper images. VLMs produce many near-duplicates, tend toward mode collapse, repeating a handful of forms, and often publish gray, abstract, or high-frequency noise. The paper quantifies this with four metrics: Semantic Recall (coverage of known object classes from THINGS), Visual Coverage and Semantic Coverage (k-covering radii in embedding spaces), and Tree Balance (J1 index of phylogenetic balance). The human baseline leads on all three coverage metrics and ties on recall. Random selection is the worst.

Three interventions, three lessons

The team varied three parameters: exploratory noise (epsilon-greedy selection), context length (how much prior interaction history the VLM sees), and the number of distinct agent personalities generated via LLM-prompted traits.

Exploration vs. quality

Without noise, VLM archives show severe mode collapse, many variations of the same fox- or fishbone-like forms. Injecting epsilon-greedy noise up to 0.25 improves diversity without sacrificing recall, but larger noise levels degrade image quality. At epsilon=1, where the VLM only handles branching and rating, with all selections random, the archive still beats random baseline on several metrics, suggesting the VLM can steer meaningfully even with minimal control. Tree Balance remains low: VLMs branch from the same images repeatedly rather than starting from random populations.

Memory is a double-edged sword

Context length zero (no history) collapses recall as agents publish duplicates. Context of 1, seeing the current turn and the prior one, recovers performance. Scaling context to 2, 10, or the full 20-generation session degrades recall and coverage metrics. With full context, agents sometimes produce near-photorealistic soda can images across multiple seeds, but the overall archive becomes noisier. The authors suggest information overload and auto-sycophantic loops, the VLM reinforcing its own stated preferences rather than exploring broadly. This mirrors known issues in long-context agents, as explored in