Antagon builds instruments that produce checkable evidence: how a model behaves when scored against a computable optimum, whether a particular output should be acted on, and whether an authorising official can rely on either.
From a Gauntlet measurement campaign
Eleven commercially available models from eight developers. Every one cleared a conventional 90% market-efficiency threshold.
Three of them finished worse than not participating at all.
A report of market efficiency alone would have passed all eleven. That gap is the measurement problem we exist to close.
Most AI evaluation reports whether a system passed. That answers a different question from the one a programme office actually has.
The question is whether the reported number supports the conclusion being drawn from it. A metric that aggregates across an environment can look healthy while individual agents inside it destroy value, and no amount of precision in that number will reveal it.
Our work is measurement validity: building environments where the correct outcome is computed exactly rather than estimated, scoring behaviour against it, and stating plainly which failures a given metric can and cannot detect.
We do not train foundation models and we do not sell a dashboard. We build instruments, and we publish the methodology so the measurements can be checked by the people relying on them.
A capability statement that hides readiness wastes everyone's time, so each one says where it actually is.
Behavioural evaluation for language-model agents, scored against an optimum computed exactly from assigned private values rather than against a rubric. Task instances are generated procedurally per session by Antavault, our dataset layer, so there is no fixed item bank for a model to have memorised.
A research model that writes a critique of an AI-generated rationale, for a person to review. A held-out evaluation has been run; the model card is the record of its method and results. No accuracy or capability claim is made for it. Open source and self-hosted. Status and details.
Assurance for the agent configuration layer: a versioned schema for an agent's authorisation slice, and a repeatable evidence artifact an ISSO or authorising official can file.
The same evidence, produced continuously for authorisation rather than once for a package — policy-audit automation for IL5/IL6 and FedRAMP High environments.
Gauntlet is the line with results on the record. The environment is deterministic and seeded; reported figures regenerate from Antagon's campaign scripts with one command. The scripts are not public; the methodology is available to evaluators on request.
Eleven commercially available models from eight developers were each run as one trader in a seeded market, three sessions per model. Market efficiency — the share of the best achievable total gain that the market realised — ranged from 95.5% to 98.0%, a spread of 2.5 percentage points. What each model's own agent earned, measured against a random budget-constrained bidder placed in the same seat, ranged from 0.95× to −0.32×. Nine earned materially less than that random bidder, the top two were within run-to-run variation of it, and three finished below zero — worse than not trading. A metric reporting only market efficiency would have found eleven near-identical systems where there were not eleven near-identical systems.
Methodology, figures and the federal registration detail are set out in full on the capabilities statement, and a paper on the decomposition behind this finding is under peer review.
Tell us which AI system you are fielding and what decision it informs. We will tell you what Gauntlet can measure about it, and what it cannot. W. Alexander Ousley, founder and principal investigator, answers technical questions directly.