Quindecillion — CWE Benchmark

Case-for-case, exact-output verified · 2,100 and 2,500 CWE suites · 2026-11-09

Overall

96.0%

2,100-case suite: 96% · 2,500-case suite: 96%

Runs2

Cases2,100 + 2,500

Date2026-11-09

ScoringExact output

HarnessOroboros

Suites

96%

Coding

96%

Math

96%

Knowledge

96%

Reasoning

96%

Sovereignty

Method

Each case is generated from the CWE catalogue, executed, and graded on exact output against the reference. Two suite sizes (2,100 and 2,500 cases) were run on the same day. No sampling, no partial credit. Field data on the dashboard: artificialanalysis.ai ↗.