LM Studio: one person, a Mac or a laptop
A desktop app: pick a model from the catalog, press download, start a chat. It shows speed in tokens per second after every answer and starts a local API compatible with common libraries. It fits testing models before you buy hardware and running a personal agent. This is how the personal agent of a bank’s IT executive works on a Mac.
Ollama: one or two people, a developer team
A command-line tool and a background service: one command installs a model, another starts the API. It fits integrations and scripts better than LM Studio, runs on Mac, Linux and Windows, and works on graphics cards and Apple Silicon. The limit is the same: single requests are fast, but with many at once, some wait in a queue.
llama.cpp: the base and the benchmarks
The library on which LM Studio, Ollama and dozens of other programs are built. Its benchmark table for Apple Silicon is the source of the figures in this section: M4 Max 83.1 tokens per second and M5 Max 119.9 on the Llama 2 7B model in 4 bits. Use it directly when you need full control over parameters and minimal overhead.
vLLM: a server for a department
A batch serving engine for graphics cards: it handles dozens of requests at once, sharing cache memory between them, and by our estimate gives a total speed of thousands of tokens per second on a server card. By our estimate, it turns a server with two 96 GB cards into a service for 10–20 users, and four H100s into a service for hundreds. It is built for Linux servers with graphics cards and needs an engineer.
How to choose
One user on a Mac or a laptop: LM Studio. A developer or an integration on one computer: Ollama. A department and agents with a request queue on a server with graphics cards: vLLM. Our agents connect to any of them through one model interface, so moving from the cloud to a local engine and back is a setting, not a rewrite.