How it differs
Versus an air-gapped setup: there is internet, so updates are easier, and you can go hybrid and use the cloud for anonymized tasks; security demands are softer. Versus GPU rental: the hardware is yours, and against rental it pays back only at high, steady load; the payback calculator shows the term, and you need an engineer and space. Versus a Russian cloud: the environment is yours, there are no third parties, and no data processing agreement is needed.
Level 1: one computer, 1–3 users
A Mac Studio M4 Max with 128 GB for $11,000, or a workstation with an RTX 5090 for about $13,000. Models up to 70 billion parameters on the Mac and up to 32 billion on the RTX 5090, with LM Studio or Ollama, at about $400–$800 a month in full cost by our calculation. The IT director of a bank runs a personal agent on a Mac at about 70 tokens per second.
Level 2: a team server, 10–20 users
A server with two RTX PRO 6000 cards of 96 GB each costs about $30,000, runs vLLM, and costs about $2,100 a month with hosting and an engineer by our calculation. It runs 70-billion-parameter models in 16 bits or 235-billion models in 4 bits, with a queue of agents. The legal department of an IT holding cut contract approval from two weeks to three days with its own model.
Level 3: server cards, hundreds of users
A server with four H100 cards from $120,000, about $6,100 a month by our calculation, or several such servers. A bank running its own models resolves about 45% of requests with no operator and estimates that support costs fall by at least 25%. According to an industry publication, a server for 1,000 simultaneous users costs about $660,000 and pays back at 80–90% load.
Platform or custom build
Integrators sell on premise AI as a platform with an interface, pipelines, and access rights; this is how the LANIT group presented a platform for deploying models on the client’s infrastructure. A build from open components costs less in licenses and needs an engineer. We build from open components and hand the result to your engineer with documentation.