Why a local agent
A cloud model receives every agent request on its developer’s server. A local model runs at your site, and the Ollama documentation says it plainly: when you run locally, the company does not see your requests and data.
For customer data this is a legal question. Under 152-FZ, Russia’s personal data law, personal data of Russian citizens is collected and stored in databases on Russian territory, and before transferring data abroad the operator files a notice with Roskomnadzor, the data protection regulator. How we handle this in projects is described on the page about data security.
Open models for this exist. DeepSeek released R1 with open weights under the MIT license on 20 January 2025, and the Qwen 3 family came out on 29 April 2025 under Apache 2.0. Open weights mean you can download the trained model itself and run it at your site.
There is one limit: a small local model may not manage a task that a large cloud model solves. Test this on your own examples before you build an agent.
Ollama and the model
Ollama is a free program under the MIT license that downloads open models and runs them on a computer or server. It has versions for macOS, Windows and Linux. The steps below were checked against the Ollama documentation.
1. Download Ollama from ollama.com/download for your system and open the app. On Linux, if the server did not start by itself, start it with the command ollama serve.
2. Open a terminal, the window for text commands, and download the model with the command ollama pull qwen3:8b. The file weighs 5.2 GB.
3. Start a chat with the command ollama run qwen3:8b and ask the model a few questions from your work in your language. The command /bye closes the chat.
4. While Ollama runs, it opens an API, a programming entry point for other applications, at http://localhost:11434. The agent will connect to this address.
How Ollama differs from other programs for running models is covered in a separate comparison. For a first agent it is enough.
What computer you need
The main limit is memory. The whole model loads into the graphics card’s video memory or the Mac’s unified memory, so the model file must fit there with room left for work.
For the starter model Gemma 4 E2B, with a file of about 7.2 GB, the Ollama documentation advises 8 GB of free video memory or Mac memory. This is a guide for a first try.
Qwen 3 comes in several sizes in the Ollama library: 4b weighs 2.5 GB, 8b weighs 5.2 GB, 14b weighs 9.3 GB, 30b weighs 19 GB. The number before the letter b means billions of parameters. The more parameters, the more accurate the answers usually are and the more memory you need.
Start with a model that fits your memory and test it on your own questions. If the answers are weak, move to the next size.
The memory calculator shows how much memory a specific model needs. It tells you whether your computer is enough or you need a graphics card.
An agent on top of a local model
In the terminal, a model only answers questions. It becomes an agent when it gets tools: access to a spreadsheet, email or a document base. In the Ollama library, Qwen 3 is marked as a model that can call tools.
The no-code path goes through n8n, an automation builder that you can also run on your own computer. The steps below were checked against the n8n documentation.
1. Start n8n with the command npx n8n. You need Node.js version 20.19 to 24.x. After it starts, open the address http://localhost:5678.
2. Create a workflow and add the “AI Agent” node. This is the base of an agent in n8n.
3. Connect the “Ollama Chat Model” to the agent. In the Ollama credentials, set the “Base URL” (the server address) to http://localhost:11434. If n8n cannot connect, replace localhost with 127.0.0.1. In the “Model” field, choose the downloaded model.
4. Connect at least one tool to the agent: the n8n documentation requires this for the “AI Agent” node. Start with a safe one, such as reading a spreadsheet.
The full build of an agent in n8n, with a channel and memory, is covered in the guide to n8n.
When you need a server
A computer suits one person. By default, Ollama handles 1 request at a time per model, and the others wait in a queue of up to 512 requests. For an agent that customers or a whole department write to, this is not enough.
You need a server when the agent works around the clock, serves several employees or customers at once and stores personal data. Under 152-FZ, data of Russian citizens is stored in databases on Russian territory, so the server goes inside your own network or with a Russian provider.
A server gets an engine for parallel work and a larger model, and that is already an engineer’s job. How we build such agents is described on the page about AI agents on a local model.
Before you buy hardware, work out the load: how many requests an hour, what model size, how many users at once. Without these numbers it is easy to buy a server that stands idle.