MMOOZZGG.com

Basics

Open source LLM for AI agents: what exists and how to choose

3 min read8 sources

Local AI runs on open language models, that is, models with open weights. An open source LLM for AI agents can be downloaded for free and run on your own hardware. Below are the ones that make sense for a Russian-speaking business, with sizes, licenses and release dates, and a rule for choosing by task and memory.

On this page · 4 sections

Russian open models

An open model of a large Russian bank has 32 billion parameters, the Apache 2.0 license and a hybrid reasoning mode, and was released on 18 July 2025; the developer claims higher quality on hard Russian-language requests with half the compute of open competitors. An open model of a large Russian technology company has 8 billion parameters and a context of 32,000 tokens, with open weights since 25 February 2025; the developer notes that it does not need much computing power. A large bank keeps its own model family inside its own perimeter; according to the case, it includes models with the open Apache 2.0 license.

Global open models

Russian companies use the Qwen family in local setups: a cosmetics maker runs Qwen for catalog search, a bank’s IT manager runs an agent on Qwen 3.6–35B-A3B at about 70 tokens per second on a Mac, and the IT unit of a store chain compared a local Qwen3.5–9B with cloud models. The open model gpt-oss 20b takes 13 GB and runs together with a vision model on a 24 GB card. The Swiss Apertus comes in 8 and 70 billion parameter sizes with fully open weights, data and training recipe, and its license allows commercial use.

How to choose

Memory first: a model with 8 billion parameters in 4-bit quantization takes about 5 GB, with 32 billion about 20 GB, with 70 billion about 40 GB. Then the task: in our estimate, 8–32 billion is enough for extracting data from documents and short chains, while long reasoning needs large models or the cloud. Then the language: test the model on 50 of your own requests, not on benchmarks. Then the license: Apache 2.0 and similar licenses allow commercial use without royalties.

Quantization

Quantization compresses weights from 16 bits to 8 or 4 bits: memory falls 2–4 times, and the loss of quality depends on the model and the task. In the store chain’s test, the local model ran in 4-bit quantization on one graphics card and was as good as cloud models on short tasks. For agents we take 4–8 bits and test on our own data.

Questions and answers

Do foreign open models understand Russian?

The large families do, but quality in Russian differs from model to model. Test on your own requests: 50 real dialogs will show more than a benchmark table.

Can you fine-tune an open model on your own data?

Yes: an IT holding spent more than a year fine-tuning a model on its own contracts. For most tasks, search over documents (RAG) is enough without fine-tuning.

Sources and fact-checking

Checked by the editors on 29 September 2026.

  1. iXBT, a large Russian bank released an open 32B model under Apache 2.0, 18 July 2025 (ixbt.com)
  2. A large Russian technology company, open weights of an 8B model, 25 February 2025 (yandex.ru)
  3. ETH Zurich, Apertus press release, 2 September 2025 (ethz.ch)
  4. Digital Platforms, case no. 4027, 2 September 2026: Faberlic (platforms.su)
  5. Digital Platforms, case no. 4085, 2 September 2026: Petrovich-Tech (platforms.su)
  6. Digital Platforms, case no. 2447, October 2025: a large Russian bank (platforms.su)
  7. Digital Platforms, case no. 3855, 2 September 2026: the technology group of a large Russian bank (platforms.su)
  8. Digital Platforms, case no. 4067, 2 September 2026: Alfa-Bank, IT manager (platforms.su)

Found a mistake? Tell us

Publications

Cases on this topic: 5

  • Market case · Russia

    Faberlik neural catalog search on a local Qwen model and rented GPUs

    The cosmetics maker replaced classic catalog search with neural search: a local Qwen language model and vector semantic search run on the GPU infrastructure of a Russian cloud. The model works inside infrastructure the company controls, data is not sent to outside services, and the cost of handling requests stays under control.

    Retail and manufacturing

    2 September 2026 · source: platforms.su

    Read the case
  • Market case · Russia

    Petrovich-Tech a local model with 9 billion parameters matched cloud models on tasks of 2–3 steps: 27 runs

    The IT unit of a building materials store chain compared three models in one agent setup with 13 analytics dashboard widgets and about 30 tools: a Western cloud model, an open cloud model with 235 billion parameters and a local model with 9 billion parameters in 4-bit quantization on one graphics card. The design: 3 scenarios, 3 models, 3 runs, 27 launches.

    Retail and manufacturing

    2 September 2026 · source: platforms.su

    Read the case
  • Market case · Russia

    A large Russian bank in-house LLMs solve 45% of requests; support costs estimated down at least 25% a year

    The bank put its own family of language models into its chatbot for LLM customer support: about 45% of customer requests are solved without an operator. In October 2025 it launched AI copilots for first-line operators, and more than 90% of support staff use them every day. The models run inside the bank’s network perimeter, so data does not leave the bank.

    Banks and finance

    source: platforms.su

    Read the case
  • Market case · Russia

    A large Russian financial group AI financial assistant on in-house language models, beta since 20 July 2026

    The group built an AI financial assistant for tasks linked to money and shopping, on its own family of large language models. To avoid made-up answers, the system relies on verified data sources, not only on what the model has learned. The beta was presented on 20 July 2026, and the full launch is planned for the autumn.

    Banks and finance

    2 September 2026 · source: platforms.su

    Read the case
  • Market case · Russia

    Alfa-Bank, IT head local agent on the Qwen 3.6–35B-A3B model: about 70 tokens per second on a Mac

    A local AI agent on Mac does autonomous work on a model with 35 billion parameters, about 3 billion of them active per token. It runs through LM Studio at about 70 tokens per second. The agent first ran in Docker on the Mac, then moved to a virtual machine, and the author calls that move a turning point for ease of use.

    Banks and finance

    2 September 2026 · source: platforms.su

    Read the case
Contact us