How the calculator counts
Weights: the number of parameters in billions, multiplied by bytes per parameter: 2 in FP16, 1 in 8-bit quantization and 0.55 in 4-bit, which includes the format overhead. KV cache: the context length in tokens, multiplied by the memory per token, which depends on the architecture: about 0.13 MB for 8-billion models, 0.25 MB for 30 billion and 0.33 MB for 70 billion and above. Reserve: 15% for working buffers and the system; you can change the field. For several users at once, the cache is multiplied by their number.
Worked examples
An 8-billion model, 4 bits, a context of 8,192 tokens: weights 4.4 GB, cache 1.0 GB, 6.3 GB with the reserve: a laptop with 16 GB. The same model with a context of 32,768: 9.8 GB. A 32-billion model, 4 bits, 8,192 tokens: 22.5 GB: a 24 GB graphics card is a tight fit, a 32 GB card has room, or a Mac with 36 GB. A 70-billion model, 4 bits: 47.3 GB, and 56.4 GB with a context of 32,768: a Mac with 64 GB fits a context of 8,192 but not 32,768, while a Mac with 128 GB has room, as do two 32 GB cards or one 96 GB card. A 70-billion model in FP16: 164 GB, two 80 GB server cards with some layers offloaded, two 96 GB cards or a Mac Studio with 512 GB. A 671-billion model, 4 bits: 427 GB: only a Mac Studio with 512 GB, where it really runs at 6.21 tokens per second, or a server with six 80 GB cards.
What it compares with
The hardware list in the calculator: MacBook Pro with 36, 64 and 128 GB; Mac Studio with 96, 128 and 512 GB; RTX 4090 with 24 GB; RTX 5090 with 32 GB and a pair of these cards; RTX PRO 6000 with 96 GB and a pair of these cards; A100 with 40 and 80 GB; H100 with 80 GB and a server with four H100 cards. For a Mac, the calculator counts unified memory minus 8 GB for the system; for graphics cards, the full video memory. Hardware prices as of 29 September 2026 come from our database; there is no price for the MacBook Pro with 36 and 64 GB. Price sources are listed on the hardware pages.
A check on a real case
A cloud provider’s team fit a text model and a vision model on a 24 GB card with a context of 65,536 tokens: weights of 13.0 and 6.1 GB, cache of 0.9 and 2.4 GB, 22.4 GB in total. If you enter the same parts and remove the reserve, the calculator shows 22.4 GB and marks the 24 GB card as a tight fit, meaning a margin under 10%. With the standard 15% reserve the total is 25.8 GB, and the 24 GB card does not pass.
Limits of the calculation
Models with few active parameters, for example 30 billion in total and 3 billion active, take memory by the full number of parameters but behave like small models in speed: count by the full size. Memory per token depends strongly on the architecture, so look for the exact value in the model card. The calculator does not count vision and speech models loaded alongside: add their weights as a separate line.