Know what a number means.
A useful comparison makes its assumptions and its limits visible.
CITE THIS METHOD
Method AIS-V2
Identifier: ais-v2 · Edition: 2026.09-v2 · Status: phase-1-live · As of: Sep 6, 2026.
Rank like with like. Keep quality, capability, user outcomes, and adoption separate.
Five peer groups. Five separate outputs. Price is recorded as evidence and is never a ranking input. Peer boards stay provisional or building-cohort until their declared evidence baskets clear.
- Lab performance — may rank inside a cleared peer group (lab)
- External benchmark consensus — may rank inside a cleared peer group (external)
- Verified capability assessment — may rank inside a cleared peer group (capability)
- Qualified user outcomes — may rank inside a cleared peer group (users)
- Adoption and momentum — may rank inside a cleared peer group (adoption)
Peer groups in this edition
- Visual app builders
- provisional · Go from prompt or visual input to a hosted working app · 7 members
- Design to code
- provisional · Turn a design artifact into maintainable repository code · 5 members
- AI IDEs and copilots
- provisional · Work interactively inside an editor or existing repository · 10 members
- Autonomous coding agents
- provisional · Delegate implementation, testing and pull-request work · 7 members
- General agent platforms
- building cohort · Run persistent work across desktop, terminal, browser, tools and schedules · 1 members
This page restates the published method. It does not invent scores for unscored models or convert a provisional board into a rank.
Editorial assessment
Saved judgments about task fit. Scores depend on the selected lens and available assessment dimensions. They are not newly measured benchmark results.
Source and vendor reports
Facts captured from linked documentation, product pages, and published reports. A check date records the observation—not a guarantee that the underlying claim is independently proven.
Measured observations
Results must retain their model version, provider, settings, dates, and test conditions. Different harnesses and workloads are not silently treated as equivalent.
Workload estimates
Calculated from documented token rates and your assumptions. Estimates are not invoices and do not include undocumented discounts or unpriced services.
Missing information stays missing.
Unscored does not mean poor quality. Unknown prices do not mean free. Hard requirements exclude candidates when the required fact is not documented.
What the price calculation includes
Input tokens × the applicable input rate, with a documented cache-read discount where available, plus output tokens × the output rate. Per-workload cost is multiplied by the number of monthly workloads. Per-request prompt size selects applicable context tiers.
Usage totals may span many requests. Cache creation, cache storage, tools, media, taxes, and service-specific overhead are excluded. When a cache-read rate is not documented, input uses the full listed rate. Saved prices are included only by explicit choice.
Repeatable, edition-pinned decisions
Shared comparisons include the catalogue edition and workload assumptions. Opening a missing or unverifiable edition fails visibly; a smaller or different catalogue is not substituted.
Current catalogue edition
- Edition
- next-ee7543aaf05d745156342d97dbe210c5a34d828556b30d5dd7be440db25eabd9
- Source release
- studio-80091062dd0e86f942323457298c969b0dc845ccc27a2d64a9647f2a5ae4be06
- Source commit
- 5ac2759e3dd4826e7bd2c4eac434295e99d04dad
Privacy and storage
Your shortlist and notes are local by default. Backups include private research. Cloud snapshots are explicitly requested and encrypted in the browser; their full links are credentials. Test prompts and submitted requests are sent to the existing service only on submission.
Report a correctionCorrections filed
1 recordsAnyone can report a catalogue error. Each accepted report gets a stable CORR reference, a scope, and a visible disposition (open, fixed, not-a-defect). Fixed entries link the superseding record or release. Method changes create a new series; old snapshots stay accessible and labelled. This log is operational and loads fail-open: if it is missing, the catalogue still renders and the dialog says so.
Current corrections log; distinct from the edition-pinned catalogue. Triage owner: Nick (site owner).
CORR-2026-001 · Duplicate evidence records in audit coveragefixed
- disposition
- fixed
- filed At
- 2026-09-12
- fixed In
- Current catalogue edition: the auditor dedupes evidence records by record id before counting coverage.
- id
- CORR-2026-001
- issue
- The catalogue auditor counted duplicate evidence records once per appearance, overstating canonical-record coverage.
- scope
- method
- subject
- Duplicate evidence records in audit coverage