Verified AI Coding

Coding-agent evaluation is expanding beyond whether a task passes. SWE-MISBEHAVIOR measures boundary violations, Parsimony measures unnecessary code growth, and Deslop turns common AI-generated TypeScript defects into enforceable lint rules. Anthropic's report of more than 30,000 daily merges shows why this matters: as generated change volume rises, the scarce capability becomes proving that an agent changed only what it should, kept the codebase maintainable and respected operational constraints.

Verification is moving inside the agent workflow rather than remaining a final gate. AgentSWE tests whether agents can build, edit and optimize other agents against held-out acceptance cases. Amp returns GitHub review comments and CI results to the thread that wrote the code, while Snowflake renders AI-proposed semantic changes as reviewable YAML diffs. Perplexity describes per-agent sandboxes and repository commits as project infrastructure. None reports controlled defect reduction, but inspectable changes, explicit acceptance and feedback loops are becoming part of autonomous work.

Agent coding evaluation is moving toward tests that are generated and executed rather than fixed at benchmark release. SWE-Duel proposes paired coding-agent tasks with fresh features, injected bugs and sealed checks, aiming to resist leakage and saturation. Separately, an engineering practitioner argues that inexpensive agent-generated tests and explicit definitions of 'done' are becoming the lever for quality at scale. The benchmark is a proposed method and the practitioner posts are one author, not proof of broad production adoption.

Code acceptance and agent claims are separating from measured correctness. A study selects 1,262 agent performance fixes and re-executes a small subset; only six of 23 rejected claims held in its pilot, a result that cannot be generalized to all agent patches. In parallel, NVIDIA describes reversible changes and GPU-backed validation, shadcn publishes an agent-first design-system linter, and Anthropic documents a measurement-led performance workflow. The shared change is that generated code requires independent executable acceptance criteria, not a confident explanation or a merged pull request. This extends the existing verification line rather than creating a new generic coding-agent category.