Seeded tool-call failures injected into unmodified coding-agent harnesses. Which harness still finishes the task, what recovery costs, and whether it reports the outcome honestly.
Rows appear here once
cmca analyze writes them from committed job directories.Metrics
pass^1 under the moderate profile (15% of eligible tool calls fail, at most 6 per trial). The headline and the default sort.
pass^1 with no faults. The baseline each config is compared against.
Robustness Retention: Faulted Success divided by Clean, over tasks the config solves without faults. Never the primary sort.
Probability that 3 attempts on the same task all succeed under moderate faults (unbiased estimator, n = 5).
Among failed faulted trials, the share where the agent reported success anyway.
Share of plausible-but-wrong outputs whose marker token ends up in the final artifacts.
Model spend divided by successful trials under moderate faults.
Configs whose 95% intervals for Faulted Success overlap share a band. Bands replace ordinal ranks.