12 criteria
The criteria fall into three groups: data, answer quality and governance. The first five decide whether you can upload client documents into the system at all.
1. Place of processing: Russia or abroad. For a foreign server, Part 3 of Article 12 of 152-FZ, Russia’s personal data law, requires a notice to Roskomnadzor, the Russian data regulator, before personal data is transferred.
2. Storage of Russian citizens’ data in databases inside Russia under Part 5 of Article 18 of 152-FZ; the fine for a company under Part 8 of Article 13.11 of the Russian Code of Administrative Offenses is up to $72,000.
3. A data processing agreement for personal data processed on the firm’s behalf.
4. Model training on your requests: whether it is on by default and how to turn it off.
5. How long requests are stored and how they are deleted, including backups.
6. A source for every citation of a rule or a court decision.
7. A flag on a citation that is not in the database, instead of a confident answer.
8. Answers based on the firm’s archive and templates, not only on a general legal database.
9. The error rate on your cases, measured in a test, not claimed in a presentation.
10. Roles and access rights by case and practice area.
11. An activity log that can be exported for audit.
12. Exit: what happens to data, templates and access when the contract ends.
Stanford RegLab and HAI give a benchmark for point 9: in May 2024, Lexis+ AI and Ask Practical Law AI gave wrong answers in more than 17% of cases, and Westlaw AI-Assisted Research in more than 34%.
The ranking of AI for lawyers, with scores on each criterion, shows how 12 foreign services and models compare on data, access from Russia and citations.
Questions for the supplier
Ask the questions in writing and request a link to the clause of the contract or the policy, not a verbal answer.
1. In which country and in which data center are requests and files stored?
2. Are the firm’s requests used to train the model, and where does the contract say so?
3. How long are requests and deleted chats kept?
4. In which cases does the supplier disclose conversations to a court or third parties?
5. How will the firm get its data and templates back when the contract ends?
The third and fourth questions are not a formality. In The New York Times lawsuit against OpenAI, the court ordered OpenAI on 13 May 2025 to keep ChatGPT output logs, including chats deleted by users; the duty ended on 26 September 2025, and on 2 December 2025 the court upheld the handover of a sample of 20 million de-identified logs.
In the Heppner case in February 2026, a US court relied on the policy of a public service under which requests are collected and may be disclosed to third parties, and it did not find the conversations with it protected by attorney-client privilege.
A one-week test on your cases
A test shows what a supplier’s presentation does not. The order for five working days:
1. Monday: collect a set of typical tasks from closed cases where the right answer is already known: a contract review, a case law question, a draft claim letter. Remove personal data from the set.
2. Tuesday: run the tasks through the system and save the answers without edits.
3. Wednesday: a senior lawyer checks every citation and marks invented decisions, wrong quotes and gaps.
4. Thursday: count the share of answers with errors and the lawyer’s time to check one answer.
5. Friday: compare the time per task with AI and without it, and decide on which process the system will pay off.
If the share of answers with errors is close to 17% or higher, the system needs a citation check built into the process. Otherwise the lawyer will spend the saved time on rechecking.
Red flags in the contract
A red flag in the contract means the risk stays with the firm, even if the service works well:
1. The supplier may use requests and files to train the model or develop the service without separate consent.
2. The place of data processing is not stated or is stated as “at the supplier’s choice”.
3. There is no section on processing personal data on the firm’s behalf.
4. The supplier may change the data processing terms on its own.
5. There is no procedure for returning and deleting data when the contract ends.
6. The supplier’s liability is capped at the cost of one month’s subscription.
Since 1 September 2026, 243-FZ, a Russian AI law, has been in force: data of sovereign and national models is processed in Russia, and the Russian government decides when such models must be used. Check that the contract lets you change the model if the requirements touch your work.
A full review of the terms is on the page about the contract with an AI supplier.
Your own model or a service
A service fits when the tasks do not involve client information: general legal questions, drafts with no names or amounts, training lawyers to work with AI.
You need your own model when the work involves client secrets and personal data. It runs on the firm’s server or in a cloud in Russia, so there is no cross-border transfer and no notice to Roskomnadzor.
Many firms combine both: a service for general tasks and their own model for client documents. The rule for what goes where is written into the AI use policy.
If the firm already uses AI, start by checking what works now: AI due diligence, a full check of a company’s AI solutions, shows which services lawyers use, what goes into them and where the risk is.
A comparison of neural network types for lawyers, from a public chatbot to a closed model, is on a separate page.