MMOOZZGG.com

Basics

Local LLM vs cloud LLM: five questions that decide the choice

3 min read16 sources

The local LLM vs cloud LLM debate is decided not by ideology but by five questions. Below, each one comes with a number and an example of a company that chose one side.

On this page · 6 sections

1. May the data leave the company

If the data holds source code, medical records, credit documents or personal data with no processing contract with the provider, the answer is no. A telecom operator required in a tender that code must not leave its data center. Moscow health care deployed a language model locally for medical data, and a developer and a bank put an AI controller of credit documents into an isolated network. According to an estimate cited by an industry publication, most foreign providers have no processing contract under 152-FZ, Russia’s personal data law.

2. How many tokens a month

At 60 million tokens a month, cloud models cost from $12 to $1,200 a month at prices as of 29 September 2026. A 32 GB graphics card cost $9,000 at one seller on 30 September 2026. By our calculation, a workstation with this card, the platform, electricity and an engineer pays back in 17 months against flagship models at $1,200, and never against mid-level models at $250: its costs are higher than the bill. Against mid-level models, by our calculation, the local option pays back at hundreds of millions of tokens a month; the calculator finds the exact point.

3. What delay and availability you need

For code completion, the telecom operator wrote a response time of no more than 400 ms into the tender; a local model gives it without depending on the internet connection or the provider’s limits. For a chatbot, where the customer is ready to wait a few seconds for an answer, the cloud is enough.

4. What quality you need

On short chains of 2–3 steps, a local model with 9 billion parameters did as well as cloud models in a 27-run test by the IT unit of a retail chain; on a 15-step chain, no model gave a fully verifiable result. In the same 15-step test, a large 235-billion cloud model gave partial success in 2 runs of 3, and the local 9-billion model in 1. On a long chain the large model coped more often in this test, and running large models in-house needs, by our calculation, a server from $30,000, or a cloud at USD 10–50 per million output tokens for flagship models.

5. Who will maintain it

A local server needs a person: the median salary of a DevOps engineer in Russia is $2,700–$3,000 a month, and a middle-level engineer in Moscow earns up to $3,800. By our calculation, the share of that salary spent on the AI server is the largest cost item after hardware. The cloud includes this in the token price.

Summary

Data decides first; if it may leave the company, the next questions are volume and people. A hybrid setup, a local model for sensitive data and the cloud for the rest, covers most cases.

Questions and answers

Can you start in the cloud and move to a local model later?

Yes, if the agents are built on a single model interface. That is how we design them: the model changes with a setting.

Sources and fact-checking

Checked by the editors on 29 September 2026.

  1. Bank of Russia, exchange rates as of 29 September 2026 (cbr.ru)
  2. Claude Platform, API prices (checked 29 September 2026) (platform.claude.com)
  3. Developer portal of a large Russian bank, API rates of its AI model for companies, 16 September 2026 (developers.sber.ru)
  4. kalinkindev.ru, API prices of a Russian tech company’s AI model in its cloud billing, 13 September 2026 (kalinkindev.ru)
  5. Regard, GeForce RTX 5090, prices as of 29 September 2026 (regard.ru)
  6. Habr (Slurm), DevOps engineer salaries, 3 September 2026 (habr.com)
  7. Anti-Malware, 152-FZ and neural networks, 17 June 2026 (anti-malware.ru)
  8. Tproger, cloud GPUs for ML in 2026, 21 May 2026 (tproger.ru)
  9. Digital Platforms (Tsifrovye platformy), case no. 3148, February 2026: T2 (Tele2) (platforms.su)
  10. Digital Platforms (Tsifrovye platformy), case no. 3570, 2 September 2026: Ministry of Justice of Russia (platforms.su)
  11. Digital Platforms (Tsifrovye platformy), case no. 3509, 2 September 2026: Moscow Department of Health (platforms.su)
  12. Digital Platforms (Tsifrovye platformy), case no. 3502, 2 September 2026: Pulkovo Airport (platforms.su)
  13. Digital Platforms (Tsifrovye platformy), case no. 3534, 2 September 2026: Step agricultural holding (platforms.su)
  14. ComNews, developer MR brought a large Russian bank’s AI model into checks of project finance terms, 15 September 2026: Developer MR (comnews.ru)
  15. Digital Platforms (Tsifrovye platformy), case no. 4027, 2 September 2026: Faberlic (platforms.su)
  16. Digital Platforms (Tsifrovye platformy), case no. 4085, 2 September 2026: Petrovich-Tech (platforms.su)

Found a mistake? Tell us

Publications

Cases on this topic: 8

  • Market case · Russia

    T2 (Tele2) tender for a self-hosted AI coding assistant in a closed data center: 15 developers, response within 400 ms

    In February 2026 the operator announced a tender for a self-hosted AI coding assistant that runs in the company’s closed data center in Rostov-on-Don, so source code never leaves its infrastructure. Requirements: 15 users, autocomplete with a response of at most 400 ms, six programming languages.

    Telecom

    2026 · source: platforms.su

    Read the case
  • Market case · Russia

    Ministry of Justice of Russia AI lawyer avatar with a base of federal laws; a local language model is being trained for it

    The country’s first AI lawyer avatar understands spoken language and advises on legal questions; its knowledge base holds federal laws, regulations and information from the ministry’s website. As of July 2026, specialists are training a local language model, which runs on the organization’s own servers, to speed up the work and improve the answers.

    Government

    2 September 2026 · source: platforms.su

    Read the case
  • Market case · Russia

    Moscow City Health Department AI assistant for hospital analytics on a local language model: data does not leave the network

    An AI assistant built on a locally deployed large language model sits inside the data marts of the city’s unified medical system. Local deployment was chosen to work with sensitive medical data without sending it outside.

    Government

    2 September 2026 · source: platforms.su

    Read the case
  • Market case · Russia

    Pulkovo Airport AI secure code review plugin in a closed network: over 90% accuracy on checking findings and up to 85% on fixes

    The airport’s information security team built code analysis into the development environment. A trained AI plugin reviews vulnerabilities and fixes the code inside a closed network, so no data goes outside. The model verifies findings with more than 90% accuracy and fixes vulnerabilities with up to 85% accuracy.

    Industry and infrastructure

    2 September 2026 · source: platforms.su

    Read the case
  • Market case · Russia

    Step agroholding its own AI ecosystem on local neural networks, so data never goes to public services

    The holding builds its AI ecosystem on local AI models for companies, agents and chatbots, so confidential data does not go to public services. It already syncs herd management data with a government system and runs legal information monitoring for the back office. Several analytics projects have started.

    Industry and infrastructure

    2 September 2026 · source: platforms.su

    Read the case
  • Market case · Russia

    Developer MR Project finance AI controller in an isolated environment: 80% less time on credit document analysis

    The developer and its bank built a project finance AI controller on a Russian language model. It checks the terms of project finance automatically and tracks loan obligations. According to the company, the time spent on credit document analysis fell by 80%. Each large project comes with hundreds of pages of credit documents and dozens of additional agreements; the system assembles a master version of each contract and keeps one calendar of obligations.

    Industry and infrastructure

    15 September 2026 · source: comnews.ru

    Read the case
  • Market case · Russia

    Faberlik neural catalog search on a local Qwen model and rented GPUs

    The cosmetics maker replaced classic catalog search with neural search: a local Qwen language model and vector semantic search run on the GPU infrastructure of a Russian cloud. The model works inside infrastructure the company controls, data is not sent to outside services, and the cost of handling requests stays under control.

    Retail and manufacturing

    2 September 2026 · source: platforms.su

    Read the case
  • Market case · Russia

    Petrovich-Tech a local model with 9 billion parameters matched cloud models on tasks of 2–3 steps: 27 runs

    The IT unit of a building materials store chain compared three models in one agent setup with 13 analytics dashboard widgets and about 30 tools: a Western cloud model, an open cloud model with 235 billion parameters and a local model with 9 billion parameters in 4-bit quantization on one graphics card. The design: 3 scenarios, 3 models, 3 runs, 27 launches.

    Retail and manufacturing

    2 September 2026 · source: platforms.su

    Read the case
Contact us