MMOOZZGG.com

Practice

How to set up self-hosted AI: eight steps with numbers

4 min read17 sources

The safest order to set up self-hosted AI in a company is: first the data rules, then one task, then the hardware. Below are eight steps with what to do, what it costs and how to know the step is done.

On this page · 9 sections

Step 1. A data policy

Split data into three classes: may go outside, only to a Russian cloud, only inside the company perimeter. An agricultural holding from this section’s case studies runs webinars for staff and will approve an internal AI policy by the end of 2026. Result of the step: one page of rules signed by the head of the company.

Step 2. One task with a measurable result

Take a task with a steady flow: sorting requests, checking contracts, answers from a knowledge base, document drafts. Write down the metric before the start: minutes per document, the share of requests handled without an operator, the number of errors. A telecom operator put a response time of up to 400 ms and refactoring accuracy of at least 85% into its tender: this is what a measurable task looks like.

Step 3. 50 test requests

Collect 50 real requests with expected answers. This is your exam for any model, cloud or local, and it matters more than any ranking. A retail chain ran 3 scenarios on 3 models 3 times each and got its answer in 27 runs.

Step 4. A trial on a rented card

Rent an RTX 4090 from about $0.50 an hour or an H100 from $3 an hour in a Russian cloud with hourly billing, install Ollama or vLLM and run the 50 requests on two or three open models: 8–9 billion, 30 billion and 70 billion parameters. Measure quality and speed. A week of trials costs from $87 to $500 and replaces a blind purchase of $12,000.

Step 5. The model and the engine

Choose the smallest model that passed the exam: it needs less memory and runs faster. Choose the engine by the number of users: LM Studio or Ollama for one person, vLLM for a department. Pin the versions of the model and the engine: an update is a separate event with a repeat exam.

Step 6. Hardware by calculator

Enter the model, the context and the number of users into the memory calculator and get a list of suitable hardware. Then use the monthly cost calculator and the break-even point against the cloud. Benchmarks as of 29 September 2026, by our calculation: a Mac Studio with 128 GB at $11,000 for 1–3 users, a workstation with an RTX 5090 at about $13,000, and a server with two 96 GB cards at about $30,000 for a department. If there is no payback and the data may go outside, stay with the cloud or rented hardware.

Step 7. Agents through one interface

Agents connect to the model through one interface, so you can change the model with a setting. Move heavy deterministic processing into code, as a bank’s IT manager did: he moved deterministic processing into separate applications and left the rest to the agent. Keep chains short: in the retail chain’s test, all three models solved a two-step task in every run, but on 15 steps none of them, local or cloud, gave a fully verifiable result.

Step 8. Monitoring and maintenance

Monitor temperature, memory errors and response time from day one. Back up the settings and the document base. Have a failure plan: a spare card, a rental contract or a switch to the cloud for data that is not restricted. A responsible person at 10–20% of a full-time role for a department server, by our estimate. A repeat exam on the 50 requests after every update.

Timeline and our role

By our estimate, steps 1–5 take 2–4 weeks, step 6 depends on hardware delivery, and steps 7–8 take another 2–4 weeks. We do steps 2–5 and 7 in a pilot of 14 days from $1,499 on your own or rented hardware, and an AI department of 3–5 agents on a local model in 4–6 weeks from $4,999; you buy the hardware based on our calculation.

Questions and answers

Can you skip the trial on rented hardware and buy right away?

Yes, if the task is typical and the model has up to 32 billion parameters: a workstation or a Mac Studio will almost certainly do. For a server from $24,000, a trial is a must.

How long does it take to install a model?

By our estimate, 10–30 minutes on a Mac or a laptop. On a server with vLLM, one or two days with setup and tests, if the engineer has worked with graphics cards before.

Sources and fact-checking

Checked by the editors on 29 September 2026.

  1. Tproger, cloud GPUs for ML in 2026, 21 May 2026 (tproger.ru)
  2. Intelion Cloud, H100/H200 rental (checked 29 September 2026) (intelion.cloud)
  3. store-apple.msk.ru, Mac Studio M4 Max 128 GB, 29 September 2026 (store-apple.msk.ru)
  4. Regard, GeForce RTX 5090, prices as of 29 September 2026 (regard.ru)
  5. iXBT, RTX PRO 6000 Blackwell 96 GB for $13,000 at DNS, 2 February 2026 (ixbt.com)
  6. Habr (Slurm), DevOps engineer salaries, 3 September 2026 (habr.com)
  7. GPU Index, RTX 4090 rental in Russia: 173 offers, from about $0.40 an hour (checked 30 September 2026) (gpuindex.ru)
  8. GPU Index, H100 rental in Russia: 41 offers, from $3 an hour (checked 30 September 2026) (gpuindex.ru)
  9. Digital Platforms catalog, case no. 3502, 2 September 2026: Pulkovo Airport (platforms.su)
  10. Digital Platforms catalog, case no. 3534, 2 September 2026: Step agricultural holding (platforms.su)
  11. ComNews, developer MR added a Russian bank’s AI assistant to project finance term checks, 15 September 2026: Developer MR (comnews.ru)
  12. Digital Platforms catalog, case no. 3148, February 2026: T2 (Tele2) (platforms.su)
  13. Digital Platforms catalog, case no. 4027, 2 September 2026: Faberlic (platforms.su)
  14. Digital Platforms catalog, case no. 4085, 2 September 2026: Petrovich-Tech (platforms.su)
  15. Digital Platforms catalog, case no. 2447, October 2025: a large Russian bank (platforms.su)
  16. Digital Platforms catalog, case no. 3855, 2 September 2026: the technology group of a large Russian bank (platforms.su)
  17. Digital Platforms catalog, case no. 4067, 2 September 2026: Alfa-Bank, IT manager (platforms.su)

Found a mistake? Tell us

Publications

Mike Mozg

Agentic AI use cases at AT&T: nearly 500 processes now run on agents

AT&T no longer uses AI only to help staff work faster: it is rebuilding whole business processes around agents. SiliconANGLE reported on 14 November 2026 that nearly 500 agentic AI use cases at AT&T are already in production. Here is how the company rebuilds them, where a human still checks the agent, and what other large companies can take from it.

News4 min

Cases on this topic: 9

  • Market case · Russia

    Pulkovo Airport AI secure code review plugin in a closed network: over 90% accuracy on checking findings and up to 85% on fixes

    The airport’s information security team built code analysis into the development environment. A trained AI plugin reviews vulnerabilities and fixes the code inside a closed network, so no data goes outside. The model verifies findings with more than 90% accuracy and fixes vulnerabilities with up to 85% accuracy.

    Industry and infrastructure

    2 September 2026 · source: platforms.su

    Read the case
  • Market case · Russia

    Step agroholding its own AI ecosystem on local neural networks, so data never goes to public services

    The holding builds its AI ecosystem on local AI models for companies, agents and chatbots, so confidential data does not go to public services. It already syncs herd management data with a government system and runs legal information monitoring for the back office. Several analytics projects have started.

    Industry and infrastructure

    2 September 2026 · source: platforms.su

    Read the case
  • Market case · Russia

    Developer MR Project finance AI controller in an isolated environment: 80% less time on credit document analysis

    The developer and its bank built a project finance AI controller on a Russian language model. It checks the terms of project finance automatically and tracks loan obligations. According to the company, the time spent on credit document analysis fell by 80%. Each large project comes with hundreds of pages of credit documents and dozens of additional agreements; the system assembles a master version of each contract and keeps one calendar of obligations.

    Industry and infrastructure

    15 September 2026 · source: comnews.ru

    Read the case
  • Market case · Russia

    T2 (Tele2) tender for a self-hosted AI coding assistant in a closed data center: 15 developers, response within 400 ms

    In February 2026 the operator announced a tender for a self-hosted AI coding assistant that runs in the company’s closed data center in Rostov-on-Don, so source code never leaves its infrastructure. Requirements: 15 users, autocomplete with a response of at most 400 ms, six programming languages.

    Telecom

    2026 · source: platforms.su

    Read the case
  • Market case · Russia

    Faberlik neural catalog search on a local Qwen model and rented GPUs

    The cosmetics maker replaced classic catalog search with neural search: a local Qwen language model and vector semantic search run on the GPU infrastructure of a Russian cloud. The model works inside infrastructure the company controls, data is not sent to outside services, and the cost of handling requests stays under control.

    Retail and manufacturing

    2 September 2026 · source: platforms.su

    Read the case
  • Market case · Russia

    Petrovich-Tech a local model with 9 billion parameters matched cloud models on tasks of 2–3 steps: 27 runs

    The IT unit of a building materials store chain compared three models in one agent setup with 13 analytics dashboard widgets and about 30 tools: a Western cloud model, an open cloud model with 235 billion parameters and a local model with 9 billion parameters in 4-bit quantization on one graphics card. The design: 3 scenarios, 3 models, 3 runs, 27 launches.

    Retail and manufacturing

    2 September 2026 · source: platforms.su

    Read the case
  • Market case · Russia

    A large Russian bank in-house LLMs solve 45% of requests; support costs estimated down at least 25% a year

    The bank put its own family of language models into its chatbot for LLM customer support: about 45% of customer requests are solved without an operator. In October 2025 it launched AI copilots for first-line operators, and more than 90% of support staff use them every day. The models run inside the bank’s network perimeter, so data does not leave the bank.

    Banks and finance

    source: platforms.su

    Read the case
  • Market case · Russia

    A large Russian financial group AI financial assistant on in-house language models, beta since 20 July 2026

    The group built an AI financial assistant for tasks linked to money and shopping, on its own family of large language models. To avoid made-up answers, the system relies on verified data sources, not only on what the model has learned. The beta was presented on 20 July 2026, and the full launch is planned for the autumn.

    Banks and finance

    2 September 2026 · source: platforms.su

    Read the case
  • Market case · Russia

    Alfa-Bank, IT head local agent on the Qwen 3.6–35B-A3B model: about 70 tokens per second on a Mac

    A local AI agent on Mac does autonomous work on a model with 35 billion parameters, about 3 billion of them active per token. It runs through LM Studio at about 70 tokens per second. The agent first ran in Docker on the Mac, then moved to a virtual machine, and the author calls that move a turning point for ease of use.

    Banks and finance

    2 September 2026 · source: platforms.su

    Read the case
Contact us