AI benchmarks explained: how model rankings work and which to trust
AI benchmarks show how a model handles other people’s tasks: questions from random people, a set of tests, or English speech recordings. Public rankings come in three types: blind comparisons by people (Arena), tests on tasks (SWE-bench and others), and combined indexes (Artificial Analysis). Below we explain what each one measures, where it errs, and why a test on your own data decides for a company.
In short
- Compare chat answers blind: Arena
- Check how a model fixes code: SWE-bench Verified
- One number from 10 tests: Artificial Analysis
- Speech recognition: Open ASR Leaderboard
- The choice for a company: a test on your 100 dialogs
The list
The order follows the criteria of this page; it is not a rating.
- #1
ChatGPT
OpenAI, USA
- What it is good for
- OpenAI models are on the Arena boards and in the Artificial Analysis index: a handy reference point for comparison.
- Why it is on the list
- On 27 February 2026, OpenAI said that ChatGPT has 900 million weekly active users. TechCrunch, 27 February 2026
- Price
- free plan; paid from $8 per month (Go), Plus $20 per month (16 October 2026) plans
- Russian language
- yes per the site
- Access from Russia
- not officially supported official page
- Official site
- chatgpt.com
- Glossary term
- ChatGPT
- #2
Claude
Anthropic, USA
- What it is good for
- Anthropic models are rated both in blind comparisons and in coding tests, so they show the difference between rankings.
- Why it is on the list
- On 28 May 2026, Anthropic raised $65 billion at a $965 billion valuation; the company’s annualized revenue passed $47 billion. Anthropic, 28 May 2026
- Price
- free plan; paid from $17 per month billed annually ($20 billed monthly) (16 October 2026) plans
- Russian language
- partly per the site
- Access from Russia
- not officially supported official page
- Official site
- claude.ai
- Glossary term
- Claude
- #3
Gemini
Google, USA
- What it is good for
- Google models are on almost all Arena boards: text, vision, documents, images, video.
- Why it is on the list
- On 11 August 2026, Sundar Pichai said that the Gemini app has more than 1 billion monthly active users. TechCrunch, 11 August 2026
- Price
- free plan; paid from $4.99 per month (Google AI Plus), Google AI Pro $19.99 per month (16 October 2026) plans
- Russian language
- yes per the site
- Access from Russia
- not officially supported official page
- Official site
- gemini.google.com
- Glossary term
- Gemini
- #4
Open ASR Leaderboard
Hugging Face, open-source project
- What it is good for
- An open speech recognition ranking by word error rate and speed.
- Why it is on the list
- In the paper version of 30 March 2026, the Open ASR Leaderboard compares 86 open and commercial speech recognition systems on 12 datasets for accuracy (WER) and speed (RTFx). arXiv 2510.06961v4, 30 March 2026
- Price
- free; the leaderboard is open, code on GitHub under Apache-2.0 (16 October 2026) plans
- Russian language
- no per the site
- Official site
- huggingface.co/spaces/hf-audio/open_asr_leaderboard
Blind comparisons by people: Arena
Arena (formerly LMArena) gives a person two answers to one question and does not show which models wrote them. The person votes for one of the answers, and a table is built from thousands of such pairs. There are many boards: text, web code, vision, documents, search, agents, images, video from text and from an image, and video editing.
The table is not a simple count of wins. In December 2023, when there were more than 130,000 votes, Arena moved from an online Elo rating to the Bradley-Terry model, which gives more stable scores and honest confidence intervals. So two models with close scores can share a place.
The weak point: the voter is not your employee but a random visitor with their own questions. In “The Leaderboard Illusion” (April 2025) the authors showed that large companies test closed model versions before release and publish only the good ones: Meta checked 27 closed variants before the release of Llama 4. By the authors’ estimate, fine-tuningmodel fine-tuning. Model fine-tuning is the adjustment of an already trained model on your own examples, so that it performs a specific task more accurately or writes in the required style. on Arena’s own data raises the result on its questions by up to 112%.
Tests on tasks: SWE-bench and others
A test on tasks checks not a person’s taste but the result. In SWE-bench a model gets a real task from a GitHub repository and must write a code fix, which is then checked automatically in an isolated environment. SWE-bench Verified, released on 13 August 2024, has 500 tasks that engineers confirmed by hand as solvable.
Such tests saturate fast. By the Stanford AI Index 2026, model results on SWE-bench Verified grew from 60% to almost 100% in a year. When almost all models solve almost all tasks, the test stops telling them apart, and its makers build a new, harder one.
The second risk: the test tasks may have reached the data the model was trained on. Then a high score shows memory, not skill. What a benchmarkAI benchmark. To benchmark AI models means to compare them on one standard set of tasks with known answers. That set is called an AI benchmark. is and how to read it is covered in the glossary.
Combined indexes: Artificial Analysis
A combined index puts many tests into one number. The Artificial Analysis Intelligence Index version 4.3.2, as of 16 October 2026, includes 10 tests in four groups: agents 30%, programming 20%, scientific reasoning 20%, general knowledge 30%. The tests include Terminal-Bench, SciCode and Humanity’s Last Exam.
The plus of an index: one scale for all models, with the task price and the response speed next to it. The minus: the compiler chooses the weights. An index that leans on agents will lift models for agents, even if you need contract translation.
We use this index for the page on reasoning modelsreasoning model. A reasoning model is a language model that writes out its own steps of the solution before it answers, and spends more time and computing on a hard task., where hard tasks matter. For the general list of language models we use Arena Text: the LLM ranking as of the check date.
Narrow rankings: speech, vision, documents
Tasks outside chat have their own rankings. According to its authors’ paper of 30 March 2026, the Open ASRspeech recognition. Speech recognition, or automatic speech recognition (ASR), is software that turns spoken words into text: from a phone call, a voice message or a meeting recording. Leaderboard compares 86 speech recognition systems from 26 organizations by word error rate (WER) and speed (RTFx).
Each narrow ranking has its limits. The multilingual part of Open ASR has German, French, Italian, Spanish and Portuguese, but no Russian. So a model’s place in this ranking says nothing about the quality of transcribing Russian calls.
For vision and documents, Arena has separate Vision and Document boards. All rankings in the section by task are collected on the main page: top AI by task.
Why a ranking is not the answer for business
A ranking answers which model is stronger on average on other people’s tasks. A company needs the answer to another question: which model will handle our process, with our cost of error, our language and our data.
The difference can be large. A model high on Arena Text may confuse the terms of your industry, while a model lower in the table may be 10 times cheaper and run on your server. A detailed example of choosing for an agent: which model to choose for an AI agent.
Also, places change every month. If you build a process around the top row of the table, in six months you will have to redo everything. It is more stable to fix the task and the criteria, and to change the model when a test on your data shows a gain.
How to test a model on your own data
Step 1: collect 100 real examples of the task. These can be customer emails, calls, contracts or support questions. Remove personal data from them, or test the model on your own server.
Step 2: for each example, write down the correct answer or the signs of a correct answer. Step 3: run the examples through 2–3 models from the ranking and count the share of correct answers, the price and the time.
Repeat the test every time you change the model or its version. The set of 100 examples is collected once, and checking a new model on it takes a day, not a month of a pilot.
Step 4: look at the errors, not only at the percent. One error in a contract amount costs more than 20 awkward wordings. Such a test on your 100 dialogs says more than any table, and the AI Solutions Map shows which processes are worth giving to AI at all.
How we built this top
- What the ranking measures: people’s votes, tests on tasks or a combined index
- Whether the method is open and there is an update date
- Whether the ranking includes Russian and tasks similar to yours
The order follows the criteria of this page; it is not a rating.
Checked:
Questions and answers
Can you trust the Arena ranking?
Why do top-ranked models work poorly on my tasks?
Sources
- Arena, March 2026 update, March 2026arena.ai
- Burges Salmon, summary of the report, 15 April 2026burges-salmon.com
- arXiv 2510.06961 v4, paper by the ranking's authors, 30 March 2026arxiv.org
- LMSYS, Chatbot Arena blog, 7 December 2023lmsys.org
- arXiv 2504.20879, 29 April 2025arxiv.org
- SWE-bench, 13 August 2024swebench.com
- Artificial Analysis, methodology, 16 October 2026artificialanalysis.ai
* Meta is recognized as an extremist organization and its activity is banned in Russia.