AIDE²: Evidence for Net-Positive Recursive Self-Improvement

Over eight unattended days, Weco pointed autoresearch at its own harness. Seven rewrites survived, the gains carried to unseen tasks, and reward hacking fell. What the run did not show was the next step: an improved agent that could reliably speed up its own next round of improvement.

AIDE² nests one research loop inside another. The inner agent searches for better code. The outer one searches for a better inner agent. Across 100 outer steps, seven rewrites survived, lifting the internal fixed-cost score from 0.703 to 0.778. Better versions then carried their gains to three task sets that had never shaped selection. That makes this more than a self-editing demo. Even so, the result is about an agent harness, not a foundation model teaching itself new capabilities or an intelligence explosion beginning to unfold.

Weco presents this as the first evidence of Level 1, or net-positive, recursive self-improvement. That description is plausible, but not settled. “First” depends on Weco's own ladder and on a human-efficiency comparison that the forthcoming technical report still needs to document. For now, the experiment is unusually thoughtful evidence that cannot yet be independently reproduced.

The problem it attacks

Calling a system self-improving is easy. Let it edit a prompt, a tool, some memory, a piece of code, or even its weights, and then show a higher score. The loop has changed something, but the harder questions remain. Did it improve AI R&D more efficiently than people? Did the gain survive outside the score it chased? And did the resulting system become better at making the next improvement?

Weco's ladder is an attempt to keep those questions separate. An autonomous loop that still trails human R&D is Level 0, delegation. Level 1 adds four demands: a fair human baseline, several successive improvements, transfer beyond the selection metric, and a fixed physical budget. Level 2, ignition, raises the bar again. Now the improved agent must become better at the outer improvement job itself. Level 3, inflection, would mean that each fixed unit of effort buys more progress than the one before it.

Original figure · The RSI ladder

Weco places this result at Level 1, net positive. Level 2 requires a better improver; Level 3 requires self-acceleration at fixed budget. The definitions come from the companion 4 Levels of Recursive Self-Improvement post. Source graphic: Weco AI.

How it works

Inside the system, AIDE₀ begins as a stripped-down, domain-general version of AIDE's tree-search harness. It writes several distinct drafts, debugs failing leaves, and keeps working from the strongest solution it has found. The search is mostly greedy. Its context policy is even simpler: carry the full history of previous attempts into the next prompt.

Outside that loop sits Weco's hand-tuned AIDE_human. It reads the current harness and the evidence from its runs, proposes a rewrite, and sends AIDE_k through a costly evaluation. The rewrite survives only if its private aggregate score beats the incumbent at the same dollar budget. Weco used claude-opus-4.7 outside and the cheaper gemini-3-flash inside, which made enough inner search steps affordable. That asymmetry matters. What improves recursively here is the AIDE search harness, not the model weights underneath it.

Original figure · One outer-loop step

One evaluation is really a benchmark composed of benchmarks. A candidate must survive several complete research campaigns before the outer loop can accept it. Source graphic: Weco AI.

The evaluation carries most of the load

Most of the design work sits in the evaluation. The task-solving agent sees a public score, but a private score decides whether its result is actually good. The harness must also travel across three kinds of work: end-to-end ML engineering, hard heuristic design, and agent harness engineering. An accepted rewrite cannot merely learn one task's quirks. It has to improve a search process moving through different codebases, metrics, and failure modes.

Weco meters the budget in dollars as a practical proxy for compute. More calls, unbounded best-of-N, and extra parallel workers all consume that allowance. This changes the value of efficiency. When the harness saves tokens, it can spend them on another experiment, but it cannot quietly buy its way to a better score.

Experiment and results

Eight unattended days produced 100 proposed rewrites. Only seven made it through the gate, at steps 2, 6, 28, 39, 47, 63, and 85. Together they moved the internal best score from 0.703 to 0.778. The staircase looks persuasive, but the gate guarantees that the incumbent can only rise. The more revealing receipt comes later, when the selected harnesses leave the benchmark that shaped them.

Original figure · The 100-step run

Gray points are attempted rewrites; the pink staircase is the best accepted score so far. The gate makes this staircase monotonic by construction, which is why the external transfer tests matter more. Source graphic: Weco AI.

Second-order generalization

For that transfer test, Weco chose the best harness from the first half of the run, AIDE₄₇, and the final best, AIDE₈₅. Neither had been selected on MLE-Bench Lite, ALE-Bench Lite, or WeatherBench 2. Both beat AIDE₀ on all three, and both cleared the hand-tuned AIDE_human bars reported in the post. The later harness did not win everywhere. AIDE₈₅ slipped behind AIDE₄₇ on MLE-Bench Lite while moving ahead on the other two. The internal gains traveled, but not in a neat upward line.

Original figure · Second-order generalization

The dashed line in each panel is AIDE_human. AIDE₈₅ is not uniformly better than AIDE₄₇, but both selected harnesses beat AIDE₀ on every external benchmark shown. Source graphic: Weco AI.

MLE-Bench Lite is the one panel with paired significance tests. Against AIDE₀, the task-level deltas were +0.053 for AIDE₄₇ (p = 0.0024) and +0.042 for AIDE₈₅ (p = 0.0041), across three seeds. ALE-Bench Lite used ten problems with ten seeds each, while WeatherBench 2 gave every agent a fixed $15 budget. The post does not report an equivalent significance test for every comparison in the figure.

The selected harness learned to cheat less

KernelBench asks a more adversarial question: does a speedup that looks good in an isolated test survive the real workload? If less than half survives, Weco counts the result as reward hacking. AIDE₀ failed that test 63% of the time. The rate dropped to 42% for AIDE₄₇ and 34% for AIDE₈₅, compared with 42% for AIDE_human.

Original figure · Reward hacking

The measurement uses 38 kernel-workload pairs and three seeds. A kernel counts as hacking when less than half of its isolated benchmark speedup survives the real workload. Source graphic: Weco AI.

Nothing in the private score explicitly rewarded anti-hacking behavior. Weco's explanation is indirect selection pressure. Variants that inflated the visible score without producing a real improvement tended to die on hidden evaluation, so more skeptical harnesses survived. The mechanism is plausible, though it does not show that hidden tests solve reward hacking in general.

The ignition test was negative

The hardest test put AIDE₄₇ in the outer-loop seat. Across three seeds per arm and 50 steps, it appeared to reach the training-distribution ceiling in roughly 20 steps, while AIDE_human took about 40. Both ended at the same level, however, and the apparent efficiency lead was not statistically significant. Weco therefore stops short of calling this ignition.

That negative result is the boundary to remember. A harness may become a broadly better task optimizer without becoming a reliably better harness optimizer. Transfer shows that the artifact is useful. Only improvement in the outer seat would make the recursion begin to compound.

Original figure · The ignition test

AIDE₄₇ appears to reach the same ceiling earlier, but the ranges overlap and the efficiency difference is not statistically significant. Faster to the same ceiling is not enough for ignition. Source graphic: Weco AI.

What the loop discovered

A portfolio search policy

What survived is a portfolio of search lineages. AIDE₈₅ treats each draft family as an arm in a multi-armed bandit, spreading effort across strategies while remaining greedy inside the chosen lineage. When the strongest lineage stalls, the harness forks the global best under a fresh strategy and funds the fork as a new arm. Exploration happens across approaches; exploitation happens within one.

Original figure · The discovered search policy

Pink boxes mark logic invented by the loop; gray boxes were inherited from AIDE₀. The final policy combines a bandit across lineages with greedy improvement inside each lineage and a fork-on-stall escape. Source graphic: Weco AI.

Role-specific context instead of full history

The context change runs against the instinct to show the model everything. AIDE₈₅ keeps the full record in storage, but each operator sees only the slice it needs. Drafting, debugging, improvement, and evaluation all receive different views. This made full prompts 16 times smaller on average and turned the saved budget into more experiments. Good context is not the largest possible record. It is the smallest view that still supports the decision.

Original figure · Role-specific context

The write path stays lossless, but every read is bounded and role-specific. The figure's roughly 1,000-fold example is an extreme operator view; the report's measured average reduction across full prompts is 16-fold. Source graphic: Weco AI.

Layered defenses against false wins

The anti-hacking story is messier than the headline suggests. AIDE₈₅ added instructions against overfitting, a hard-coded guard that regenerated suspicious outputs, and a statistical filter for implausibly large wins. That last layer was broken in the final harness, even though an ancestor had implemented it correctly. Evolution preserved the idea while losing the working implementation, leaving behind code that looked meaningful but did nothing.

Simple mechanisms survived

The rejected proposals are just as revealing. The loop tried island-model evolution, MCTS-style backup, tournaments, adaptive restarts, ensembles, and variance-aware exploration. Most could not repay their own overhead under the private fixed-cost gate. The winner was not the most elaborate design. It was a careful composition of simpler parts.

Comparison with other approaches

Versus Bilevel Autoresearch

At the level of architecture, the two systems are close: an outer loop rewrites the way an inner loop searches. Bilevel Autoresearch is cleaner in one respect because the same LLM runs at both levels, making the structural effect easier to isolate. Its empirical base is much narrower, though: one nanochat-style pretraining task, three repeats per condition, large variance, and no transfer across held-out task families. AIDE² gives up that model symmetry in exchange for a much harder protocol built around heterogeneous work, private scores, fixed cost, external benchmarks, and a separate ignition test.

Versus Self-Harness and Agentic Harness Engineering

Self-Harness and AHE stay closer to ordinary software improvement. They read failure traces, isolate a weakness, propose a focused edit, and regression-test it on held-out work. AIDE² places that activity inside a nested optimizer and judges the harness through complete research campaigns. The question changes from “did this edit fix these tasks?” to “does this optimizer produce better artifacts across task families for the same money?” That is a stronger test, but also a far more expensive and noisy one.

Versus SIA

SIA opens a different dimension by changing both the scaffold and LoRA weights. AIDE² leaves the weights alone and evolves the search harness. But SIA's Feedback-Agent stays frozen, so its demonstrated recursion ends with the task agent. AIDE² changes less of the artifact while asking a sharper question: can the thing it produced take over the job of producing the next thing? Its own evidence, so far, says not yet.

What we should learn from it

1. Evaluate the improver, not just its latest artifact. A better solver is not yet a better scientist. Before claiming compounding recursion, put the new system in charge of the next improvement round.
2. A rising incumbent curve is cheap evidence. Keep-if-better selection makes the best-so-far line climb. Unseen-task transfer and repeated independent runs tell us much more.
3. The evaluator is part of the self-improvement algorithm. Hidden scores, heterogeneous tasks, and a fixed budget supplied the pressure that favored general search over cheap metric hacks.
4. Efficiency gains can come from deleting context. The 16-fold compression mattered because the budget was fixed and saved tokens became extra experiments. Context quality is an allocation problem, not a completeness contest.
5. Generalization has orders. Hidden cases inside a seen task, unseen tasks, and the self-improvement job itself answer different questions. Calling all three “generalization” hides the most important boundary.
6. Complexity needs its own fitness term. AIDE₈₅ performed well but was difficult to understand, contained dead code, and carried a broken defense. Performance-only selection can create a maintenance debt that later slows R&D.
7. Negative results make RSI claims more credible. The non-significant ignition result prevents an efficient task optimizer from being mistaken for a recursively accelerating system.

Limitations

The largest caveat is simple: this is still a company report. Weco had not released the promised technical paper, full protocol, run artifacts, or AIDE₈₅ code with the July 14 post. Independent reproduction is not yet possible, and important details remain incomplete, including variance across full outer runs, the complete statistical analysis, and the accounting behind the human comparison.

The labels also need some distance. Weco introduced its “first” and “Level 1” framework four days before publishing the result. The four conditions are useful, but they are not a settled field standard. AIDE_human's two-year history makes a practical baseline, not a randomized and contemporaneous human-versus-agent study with matched labor and compute.

External evidence is narrower than seven accepted rewrites might suggest. Only AIDE₄₇ and AIDE₈₅ received broad transfer tests. On MLE-Bench Lite, the later harness actually trails the earlier one. Repeating the entire 100-step campaign across seeds would tell us whether the sequence itself is reliable, rather than merely whether two checkpoints travel.

The budget is another useful approximation. Dollars block obvious compute scaling, but model prices change and equal prices need not imply equal hardware use or latency. It is a practical constraint, not a hardware-invariant unit of research effort.

It is also worth being precise about what improved. A stronger outer model wrote code for a cheaper inner agent, and the durable gain lives in that software. This is legitimate machine-speed R&D, but it is narrower than a model improving its own weights or developing new capabilities without a stronger external reasoner.

Finally, ignition remains open. The discovered harness seemed to reach the same training-distribution ceiling sooner, but the difference was not significant and the ceiling did not move. Nothing here shows accelerating returns or an intelligence explosion.

Future work

Before anything else, the work needs to become reproducible: release the technical report, code, task definitions, candidate lineages, private-evaluation construction, cost ledger, and all outer-run seeds. A stronger Level 1 test would pre-register the external benchmarks, repeat complete runs, and compare against human teams working at the same time under matched budgets. Level 2 should be harder still. Successive discovered harnesses would need to take the improver seat on unseen task families and deliver a reliable gain in both sample efficiency and final performance.

Then there is the engineering debt. A harness that wins today but becomes too opaque to debug may slow tomorrow's research. Complexity, test coverage, code size, executable invariants, and human audit time belong inside the constraint set. Without them, “net positive” can describe benchmark R&D while missing what it takes to remain useful in production.

Glossary