Glossary
What is an AI audit
In short
An AI audit is a check of how artificial intelligence works in a company: how accurate and safe the systems are, by what rules they act and who is responsible for them.
Business and adoption · AI audit
MMOOZZGG.com editorsChecked: 1 min read
For a child
Once a year, inspectors come to a school. They look at whether the desks are sturdy, the fire extinguishers work and the class registers are filled in correctly. Nobody gets scolded; they just look for what to fix before something bad happens. Smart programs at grown-up work get checked too: do they make mistakes, do they treat anyone unfairly, and do they keep other people’s secrets safe.
In plain words
The word covers two similar things. The first is a check of systems that already work. People look at how often the model makes mistakes, whether it answers different groups of clients differently, where the information goes, who can see it and who is responsible for errors. The second is an audit of the company before AI is introduced, to find the processes where AI will help. Both are like a medical checkup: you go while nothing hurts, to catch a problem early.
For business
A check is needed when AI already makes decisions that affect clients and money: it approves applications, answers in support, calculates prices. An error in such a system repeats thousands of times before anyone notices. An AI audit for business covers answer quality on real examples, data leaks, compliance with data protection law (in Russia, 152-FZ), access rights, dependence on a supplier and a human in the loop for disputed cases. The cost depends on the number of systems and the depth of testing. The result is a list of risks with ratings and a plan to fix them. If the company has not launched AI yet, a similar review of processes and data is done as a full check before implementation.
For a specialist
An AI audit (algorithmic audit) is a systematic assessment of an AI system against criteria of accuracy, reliability, fairness, safety, transparency and compliance. There is an external audit, run by an independent party after launch, and an internal one, built into the development life cycle. Raji and co-authors, in the paper Closing the AI Accountability Gap (ACM FAT* 2020), proposed a framework for end-to-end internal auditing: each development stage leaves documents that together make up a report and are checked against the organization’s stated principles. The practical base for procedures is the NIST AI RMF (2023) with the functions Govern, Map, Measure and Manage. Tools include sets of test scenarios, bias checks by group, prompt injection tests and analysis of the model’s work logs.
Example
A leasing company checked its application scoring model and found that it rejected clients from small towns more often. This feature was removed from the data, the model was retrained, and an employee now reviews disputed applications.
Related terms
Questions and answers
Sources
Not sure whether your company needs this?
AI solutions map