Methodology

How we rank LLMs

No single benchmark decides which model is state-of-the-art. Every source has known weaknesses: LMArena rewards style, static benchmarks get contaminated, private evals aren't reproducible. This page explains exactly how the sota-model.com leaderboard aggregates the signals and picks the SOTA.

Data sources

The candidate pool is pulled live from the OpenRouter models API. We keep frontier models only — models from top labs (OpenAI, Anthropic, Google, xAI, DeepSeek, Meta, Mistral, Alibaba/Qwen, Moonshot, Z.ai, MiniMax, StepFun, Cohere, NVIDIA and others), released in the last ~120 days OR priced in the flagship tier. We exclude mini / lite / nano / flash / air variants, free tiers, and image-only or audio-only models.

Benchmark scores for the ranked set are aggregated by an LLM-based analyst pass. It weighs every widely-cited public evaluation it can identify for each model: LMArena Chatbot Arena Elo, the Artificial Analysis Intelligence Index, LiveBench, SEAL, Aider Polyglot, SWE-bench Verified, SWE-bench Pro, Terminal-Bench, ARC-AGI-2, GPQA Diamond, MMLU-Pro, MATH, HumanEval+ and OpenRouter usage trends. No single benchmark decides SOTA — the model that leads across the most signals does.

Category weights

The category leaderboards (Coding, Reasoning, Agentic) use fixed weights over public benchmarks. Missing benchmarks are renormalized across whatever's present, so partial data still yields a comparable score.

coding
  • swe_bench35%
  • aider_polyglot25%
  • terminal_bench20%
  • livebench15%
  • aa_index5%
reasoning
  • gpqa30%
  • arc_agi25%
  • livebench20%
  • mmlu15%
  • aa_index10%
agentic
  • terminal_bench40%
  • swe_bench30%
  • aider_polyglot15%
  • livebench10%
  • aa_index5%

Guardrails

  • Only true flagship reasoning models can occupy the top 5. Mid-tier siblings (mini, lite, nano, flash, air, haiku) must rank below their lab's flagship AND below every competing lab's flagship.
  • Higher output price and larger parameter count are strong flagship signals; low price + high throughput is a strong signal a model is NOT the flagship.
  • All URLs from RSS ingestion and model source pages are filtered to http/https schemes only.
  • The refresh pipeline is authenticated by a private cron secret — no anonymous writes.

Refresh cadence

The pipeline runs every ~2 hours via a scheduled cron job. Each run: (1) fetches the OpenRouter models catalog, (2) filters to frontier candidates, (3) asks the analyst model to rank them and score each on 10+ benchmarks, (4) upserts the top 25 into the database, (5) updates the SOTA slug on the site. The "Last updated" timestamp on every page reflects the last successful run.

Known limits

Scores below the top few labs are estimates, not authoritative measurements. When a lab hasn't published a benchmark for a given model, the analyst estimates it — clearly worse than a first-party score but better than a blank cell for sorting purposes. For authoritative numbers, follow the source link on the model card.