Glossary
What is model evaluation?
In short
Model evaluation (evals) is a systematic check of AI answers on a set of your company’s tasks, to learn how accurate and safe the AI is and whether it is ready for work.
Language models · evals
MMOOZZGG.com editorsChecked: 1 min read
For a child
Before a school concert, the choir rehearses many times. The teacher listens to each song, notes where the kids went wrong and asks them to repeat the hard part. Only when everything works do they go on stage. We do the same with a neural network: before it starts helping people, it is checked on tasks many times.
In plain words
A neural network can answer smoothly and still be wrong. To learn how far you can trust it, you collect typical questions and right answers for your own task and run them through the model. A program, a person or another neural network checks the answers by clear rules: is it right, is it complete, is anything made up? The result is a score in percent. You repeat the check after each change to the instructions or switch of model, so a fix in one place does not break another. It is like a car inspection with a checklist.
For business
Without a test on your own data, you cannot honestly say whether the AI is ready to work with clients. The test set becomes an agreement: what share of right answers you need, which mistakes are unacceptable, how much an answer costs and how fast it comes. You also use it to compare vendors and to decide whether to move to a new model. The main costs are collecting and labeling a few hundred examples, setting up automatic runs and assigning an expert to disputed cases. The risks: a set of easy questions creates false confidence, and an AI judge makes mistakes and favors long answers. This is the metric that goes into a pilot, so the result is measured rather than debated.
For a specialist
Model evaluation (evals) measures the quality of a model or a system built on it (a prompt, RAG, an agent) on a set of test cases with metrics and a checking method. Unlike a public benchmark, an applied set reflects the real mix of the product’s tasks, edge cases included. Checking methods include exact match and rules in code, similarity metrics (ROUGE, cosine similarity of embeddings), expert labeling and a judge model (LLM-as-a-judge) with a rubric. Zheng et al. (2023) showed that a strong judge agrees with human preferences in more than 80% of cases but is prone to biases: position, verbosity and preference for its own answers. HELM (Liang et al., 2022) proposed a holistic evaluation by seven metrics: accuracy, calibration, robustness, fairness, bias, toxicity and efficiency. The OpenAI Evals framework lets you build private sets on your own data. Runs happen offline before a release and online on live traffic.
Example
Before launching an AI support assistant, the team collects three hundred real client questions with correct answers. Each new version of the instructions runs through this set, and only a version that answers at least nine questions out of ten correctly goes live.
Related terms
Questions and answers
How do you evaluate the quality of AI answers?
Sources
- Zheng et al., Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, 2023arxiv.org
- Liang et al., Holistic Evaluation of Language Models, Stanford CRFM, 2022arxiv.org
- OpenAI, Evals: a framework and registry of evaluations, GitHubgithub.com
- Anthropic, Define success criteria and build evaluations, documentationplatform.claude.com
Where companies use it
Not sure whether your company needs this?
AI solutions map