Swe-Bench

AI Self-Improvement Starts Outside the Model

When people talk about AI self-improvement, the usual image is a model rewriting its own weights. Lilian Weng’s July 4, 2026 post on Lil’Log argues the near-term version looks different. The recursive self-improvement (RSI) we can actually observe today shows up first in the system around the model, not inside it. That system is what she calls the harness. This post walks through her argument: what a harness is, where the optimization target is moving, what a striking SWE-bench number actually means, and what breaks if you get the risk boundaries wrong.

Read More

DeepSWE: A Benchmark for Long-Horizon Coding Agents

SWE-bench has been the default coding-agent leaderboard for a while, but it has well-known weaknesses. Most tasks come from existing public issues and PR patches, so a high score might partly reflect memorization. Most tasks are also single-file bug fixes, which is not representative of the multi-file, long-horizon work that a coding agent does in practice.

Read More
Categories
Tags