DisCo reports a 134.3% MLE-bench gain with its backbone held fixed

The number that will travel from a new arXiv preprint is 134.3%. A research agent called DisCo, given a library of distilled "skills," scores that much higher on MLE-bench than the same agent without them, according to the paper's authors. The rest of the abstract is architecture, and the numbers only make sense inside it.

The claim rests on what did not change

The preprint, titled Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills, reports that DisCo both creates skills and uses them during research. It describes two distillation paths. Task-agnostic distillation condenses widely used repositories into reusable skills; task-oriented distillation produces the skills a specific task calls for. Run across the open ecosystem, the first path yields the AREX-Skill Library: more than 5,000 verified skills distilled from 1,000 widely used machine-learning repositories, organized into 20 areas and 178 capability families.

That library is only half the claim. The reported gains come with the GPT-5.5 backbone, the research harness and the downstream execution budget all held fixed. That sentence carries the paper's argument. If nothing else changed except the injected context, the delta belongs to the skills rather than to a bigger model or a longer run. The authors put it plainly: the gains come from adding distilled operating context under that fixed setup. Other measurements point the other way, finding that loading agents with skills carries a regression tax rather than a guaranteed gain.

The missing layer the paper calls operational knowledge

The paper's central thesis is about what an agent does not have. It does not claim a better planner or a better memory module. It argues that the know-how separating knowing a method from making it work sits outside the agent altogether, in a layer the authors call operational knowledge. An agent architecture, in their description, combines a model backbone with a harness for planning, execution, memory and verification. Operational knowledge is what the harness still lacks.

That knowledge is not missing from the field, per the preprint. It appears in repositories and papers, written for human readers and too large to load during a single task. Distillation is the fix: shrink it into compact, verified skills that get reused across tasks instead of rediscovered during each run. The duplication it targets is real. SkillZip compresses a rule once it reappears across agent branches, and without that kind of pruning, context files grow by 226% and never shrink.

Four benchmarks, no shared baseline

The abstract reports four gains, and they range from large to modest.

Benchmark	Reported gain, skills vs. no skills
MLE-bench	134.3% higher
PaperBench	34.4% higher
FrontierCS	9.2% higher
PassNet	14.0% higher

The abstract does not describe what each benchmark measures, what the baseline scores were, or how each percentage was computed, so the four figures cannot be read against one another. A large delta on one test does not carry over to a small one on another, and the spread here may reflect the tasks, the scoring, or both. Nor does the abstract give absolute scores, which is the detail that would show whether a 134.3% improvement sits on a weak baseline. It is a familiar blind spot: agents can run flawless statistics and still draw the wrong conclusions when a benchmark rewards execution over judgment.

A preprint, not a replicated result

This is a single submission, listed under computer science and artificial intelligence and posted to arXiv on 2 September 2026. The material available carries no record of peer review, and no independent group has reproduced the figures. The measurements are the authors' own, taken on their own system under their own harness. That does not make them wrong. It makes them a claim to test rather than a result to cite.

The detail to watch when someone tries to reproduce this is not the 134.3%. It is whether a frozen skill library transfers to a different backbone. The experiment reported here holds the backbone fixed, which is a clean way to isolate the effect of context, but it cannot show on its own that the context travels.

Whether skills stick as an architectural layer

The preprint puts skills beside the backbone and the harness as a third place to store know-how, one that neither weights nor scaffolding currently covers. Whether that position holds is a question the abstract leaves open. It names no competing systems, and it does not say whether other research-automation efforts are working on the same layer or pushing the problem back inside the model. Some already push it inward: BAAI's AREX agents lean on a recursive loop that compresses their own research histories instead of a shared external library.

What it does offer is a testable shape. If operating knowledge can be distilled, verified and reused, then part of what makes a research agent work is a library problem rather than a training problem. One thousand repositories and 5,000 skills is a single data point for that idea, published by the people who built the library.

134.3% is the advertisement. The held-fixed variables are the part a replicator can actually check.