MMOOZZGG.com

Hardware

Server for running an LLM: three typical configurations

3 min read10 sources

To run a local model, you do not need to design a cluster: a server for running an LLM can follow a typical configuration. Below are three typical configurations with component prices as of 29 September 2026, what they can run, and what they cost per month with power and hosting. The totals are our estimates from component prices.

On this page · 4 sections

Configuration 1: a workstation with one card, about $13,000

An RTX 5090 with 32 GB cost $9,000 on 30 September 2026, plus a platform with 128 GB of RAM and a power supply from 1,000 W, which we estimate at $3,600–$4,200. It runs models up to 32B parameters for 1–3 users, and 70B models slowly, with part of the layers offloaded. Power under load is, by our estimate, about 800 W: 576 kWh a month, about $54 at the Moscow household rate of about $0.10. It sits in the office and is noisy.

Configuration 2: two 96 GB cards, about $30,000

Two RTX PRO 6000 cards at $13,000 each, plus a server platform that we estimate at $3,600–$4,800. By our calculation, with an 8,192-token context, 192 GB of video memory is enough for a 70B model in 8-bit for 10–20 users with batched output. In 16-bit, the same model fits up to 3 users, and a 235B model in 4-bit up to 7. Power is up to 1,200 W for the cards plus 300 W for the platform: about 1,080 kWh a month, $100 at the about $0.10 rate. Hosting in a data center costs from $90 a month for a server up to 300 W, plus $24 for each extra 100 W: about $400 a month for 1.5 kW.

Configuration 3: four H100 cards, from $120,000

Four H100 SXM5 cards at $28,000 each, plus a chassis that links the cards: from $120,000 per server. 320 GB of video memory runs models up to 400B parameters in 4-bit, about 260 GB with cache and headroom, or 70–235B models for dozens of parallel dialogs. By our calculation with an 8,192-token context, that is about 80 for a 70B model in 4-bit and about 45 for a 235B model; 671B models no longer fit. Power is up to 2,800 W for the cards and about 1,000 W for the rest: 2,736 kWh a month, $250 of electricity at the about $0.10 rate, and about $1,000 for hosting 4 kW. An industry publication estimates that a server for 1,000 concurrent users costs about $660,000, and to keep the token price reasonable, GPU load must stay at 80–90% or higher.

What else goes into the bill

Backup disks and a second power supply, a UPS, monitoring system licenses, a spare card, and the administrator’s time. A rule from practice: a server without a person to run it costs more than a server with one, because downtime costs more than a salary.

Questions and answers

Can you build a server from several gaming cards?

Yes, for one team: in a 2024 test, two RTX 4090 cards gave 19 tokens per second on a 70B model in 4-bit. For a round-the-clock service, gaming cards are risky, since the maker does not state memory error correction for them.

How long does a server last?

Count on 3 years of depreciation: in that time both models and cards change. The calculator uses 36 months by default.

Sources and fact-checking

Checked by the editors on 29 September 2026.

  1. Regard, GeForce RTX 5090, prices as of 29 September 2026 (regard.ru)
  2. iXBT, RTX PRO 6000 Blackwell 96 GB for $13,000 at DNS, 2 February 2026 (ixbt.com)
  3. Servermall, NVIDIA H100 SXM5 80 GB, price as of 29 September 2026 (servermall.ru)
  4. NVIDIA, GeForce RTX 5090: 575 W, 32 GB GDDR7 (nvidia.com)
  5. NVIDIA, RTX PRO 6000 Blackwell: 96 GB, 600 W (nvidia.com)
  6. NVIDIA, H100: SXM 80 GB up to 700 W, NVL 94 GB 350–400 W (nvidia.com)
  7. RusCable, electricity rates in Moscow from 1 January 2026 (ruscable.ru)
  8. HOSTKEY, colocation in Moscow, 2026 (hostkey.ru)
  9. ComNews, local AI and the cost of automation, 19 February 2026 (comnews.ru)
  10. GPU-Benchmarks-on-LLM-Inference, Llama 3 8B and 70B, May 2024 (github.com)

Found a mistake? Tell us

Publications

Contact us