Task and model
1. Which model passed the exam on your 50 requests, and at what quantization. 2. How many users and agents work at the same time at peak. 3. What context length you need: a 10-page document of 1,800 characters a page is about 4,500–6,000 tokens for a Russian-language model at 3–4 characters per token; for other models, measure with the provider’s counter. 4. Whether a vision or speech model runs next to it: that adds memory.
Memory and speed
5. How much memory the calculator shows with a 15% reserve and a cache for all users: a 32-billion-parameter model for 10 dialogs of 8,000 tokens needs about 43 GB, not 22. 6. Will next year’s model fit: take memory one step higher. 7. Did you measure speed on a rented card, at about $0.50 to $7 an hour, with 1, 4 and 16 streams.
Hardware and location
8. Power: RTX 5090 575 W, RTX PRO 6000 600 W, H100 up to 700 W, plus the platform; is the power supply and the socket enough, and do you need a three-phase line. 9. Heat and noise: a workstation in an office can be heard, and server cards live in a data center, from $90 a month for a server of up to 300 W. 10. Warranty and replacement: the warranty term on the invoice, replacement outside warranty at the price of a new card, $6,500 to $28,000; does the seller have spare cards.
People and processes
11. Who runs the server: by our estimate 10–20% of an engineer’s time at a median salary of $2,700 to $3,000, or a contractor. 12. A failure plan: a spare card, a rental contract, a switch to the cloud for non-sensitive data. 13. An update procedure with a repeat exam. 14. Monitoring of temperature, memory errors and response time from day one.
Money
15. The payback point against the cloud and against rental, from the calculator: if it is not within 36 months and the data may leave the company, do not pay the invoice. By our calculation, a Mac Studio at $11,000 pays back in 71 months against a cloud bill of $250 a month, and in 10 months against $1,200 a month.