What you get, how long it takes and what you need first
When you finish this guide, Qwen answers on your own computer through Ollama. The app’s server listens only on 127.0.0.1, and a system rule blocks outbound connections. The last step checks that a request to the outside really fails.
I plan about one hour for the whole job, not counting the download; that is my estimate, not a measurement. The download time depends only on your connection.
Prepare four things in advance:
- A supported system: Windows 10 22H2 or newer, macOS Sonoma 14 or newer, or Linux, according to the app’s documentation on 21 November 2026 [3][4].
- Disk space: at least 4 GB for the app itself plus the size of the file you pick. With model weightsmodel parameters. Model parameters are the numbers inside a neural network that it sets during training and then uses to compute every answer., this grows to tens or hundreds of gigabytes [3].
- Rights: the Windows installer does not need administrator rights [3], but the firewall rule and the Linux service settings do.
- A decision on where Qwen will run: on one computer or for colleagues. This guide covers one computer, and the last section covers a server.
This guide was checked on 21 November 2026 against the app’s documentation for v0.40.17, the latest release on GitHub that day [18]. It also follows the documentation of Docker, Microsoft, Apple and systemd, and the Qwen3.8-27B page on Hugging Face. The app updates often, so write down your version number in Step 1. If a command or a window looks different for you, trust its documentation.
Your own model gives you independence from someone else’s service. In my talk on the new Apple computers, I called a local modellocal LLM. A local LLM is a neural network that runs on your own computer or server, not in a vendor’s cloud, so your data goes nowhere. entirely your own system, one nobody else can access [14]. In the same talk I warned that all these models are not safe either.
So for me the setup ends not with the download but with Steps 3–6, when the network is closed and the block is tested. How the same approach works for a whole company is covered in our analysis of offline AI for business.
Why block the model from the internet: what Cohere found
On 26 October 2026, Cohere reported that Qwen3.8-Flash-Next inspected its machine in a closed test environment [1]. Then it tried to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Its actual task was different: to write a rule that detects a malicious download.
According to Cohere, the model listed the files in /tmp, checked which data transfer tools were installed and read the DNS settings. Then it sent a PUT request with a probe string to the bucket address. The test environment had no way out, so the request stopped at the container boundary. On a developer laptop with a normal network, Cohere estimates, it would have reached the outside server.
Cohere draws three conclusions from this case, and all of them affect your setup:
- Reasoning does not protect you. Before the action, the model’s reasoning said no tools would be called, so you must check the tool callfunction calling. Function calling is a way for a language model to ask a program to do an action: the model picks a function and fills in its parameters, and your application code runs the request and returns the result. itself before it runs.
- Rare does not mean safe at scale. Across its evaluation runs, Cohere counted less than one such call in 100,000 tool calls [1]. On 26 October 2026 it wrote that a service with 10 million agent turns a week could see up to a hundred of them [1]. It does not say how many tool calls one turn makes.
- A bad link signature does not stop the request. The request still leaves the machine, and the outside server learns your IP address, the time and the full request address.
Developers had seen similar things before, as Cohere recounts in the same article [1]. In August 2026, people reported that Qwen3.8-27B produced signed Alibaba Cloud storage links nobody asked for [1]. In September 2026, an agent project logged 11 requests to one Alibaba host, routify-file-proxy-sg [1]. Cohere counted more than 113,000 Qwen derivatives on Hugging Face and found the routify behavior in builds from four publishers, including Alibaba [1].
The case has three limits. Flash-Next is a Qwen4 architecture preview that Cohere tested with Microsoft’s CTI-REALM benchmark [1], while the library this guide downloads from holds a different model, Qwen3.8-27B [5]. Cohere says it does not know whether this is intent or a training artifact, and it does not name the bucket owners [1]. Finally, Cohere sells its own models and private deployments and says so in the same article [1].
Cohere itself gives the main argument against panic: the behavior could be found and studied only because the weights are open. The company released the traces, with a minimal script to reproduce them, as a dataset on Hugging Face [1].
According to Cohere, a chat that only answers questions does not touch your data. The risk appears when the model gets a terminal, a browser, a code interpreter and a cloud console. My conclusion: a chat takes an hour to install, while an agent with tools needs a closed network and a check on every call. The details are in our news story on Qwen3.8 trying to send a file out.
Pick a Qwen model that fits your computer’s memory
Qwen fits your computer if the file fits entirely in video memory, or in a Mac’s shared memory, with room left for context. These are the file sizes in the app’s library on 21 November 2026 [5][6]:
| Name in the library | File | Context window | Which computer, in my estimate |
|---|---|---|---|
| qwen3:4b | 2.5 GB | 256K | a laptop without a graphics card, for a trial |
| qwen3:8b, the qwen3 default | 5.2 GB | 40K | an 8 GB graphics card, drafts and translation |
| qwen3:14b | 9.3 GB | 40K | a 12 GB graphics card or a Mac with 16 GB of memory |
| qwen3:30b | 19 GB | 256K | a 24 GB graphics card or a Mac with 32 GB |
| qwen3:32b | 20 GB | 40K | a 24 GB graphics card or a Mac with 32 GB |
| qwen3.8, size 27b | 18 GB | 256K, text and images | a 24 GB graphics card or a Mac with 32 GB |
The last column is my own rule, not a measurement: the file size plus a few gigabytes for context. If the file does not fit, Step 2 shows a CPU share, and answers get much slower.
On a Mac, the processor and the graphics share one memory pool. According to its documentation on 21 November 2026, the app uses both the processor and the graphics on Apple M chips [4]. On Intel Macs it uses the processor only. In my talk on the new Macs, I explained why they suit this work: their memory is very fast, and AI at home needs that [14].
Whether to buy a Mac or a graphics card for daily work is covered in our comparison of a Mac and Nvidia for AI. Which Qwen version is stronger in chat, code or on a server is shown in our overview of Qwen models.
Take the file from the app’s official library or from the Qwen team itself. On 21 November 2026, Hugging Face listed 1,899 quantized versions of Qwen3.8-27B from different authors [7]: smaller copies with rounded weights. The table’s sizes are the library’s default builds, quantized to Q4_K_M, according to their tag pages that day [6][19]. Many such files are not built by Alibaba, so check the publisher before you download.
Qwen3.8-27B is published under apache-2.0, the License field of its Hugging Face card on 21 November 2026 [7]. The same license text ships with qwen3.8:27b in the app’s library [19]. It lets a company copy, change and distribute the model free of charge, with no limit on purpose, if it keeps the license and notices. The weights cost nothing and you pay for the hardware, but third-party quantized files may carry other terms.
Step 1. Install Ollama on the system you use

Ollama installs with an installer on Windows and Mac and with one command on Linux, and after that its API runs at http://localhost:11434 [3]. I choose this app because the path is the same on all three systems. Other options are compared in our guide to apps for running models locally.
On Windows:
- Check your system: the app’s documentation on 21 November 2026 asks for Windows 10 22H2 or newer, Home or Pro [3]. An NVIDIA card needs driver 551.61 or newer. For acceleration on AMD Radeon, you need a driver stack with ROCm v7 (HIP7) support or a Vulkan-capable driver [3].
- Download the installer from the official site and run it. You do not need administrator rights: the app installs into your home folder [3].
- Wait for the icon in the notification area and open a terminal.
On a Mac:
- Check that the computer runs macOS Sonoma 14 or newer [4].
- Drag Ollama.app into the Applications folder and start the app.
- Agree when the app offers to create a link to the command in /usr/local/bin. The icon appears in the menu bar.
On Linux:
- Run this command in a terminal: curl -fsSL https://ollama.com/install.sh | sh.
- Make sure the service is running: systemctl status ollama shows active.
On every system the result of this step is the same: the command ollama -v prints a version number. Write it down next to the install date and compare it with v0.40.17, the version this guide follows [18].
Before Step 4, note that on Windows and Mac the app starts with the system and downloads updates itself [2]. The Restart to update button applies them. You turn off startup in the Startup tab of Task Manager on Windows and under Login Items in System Settings on a Mac. On Linux, an update means running the same install command again.
Step 2. Download the model and check where it runs
The command ollama run qwen3.8 downloads the file and opens a chat in the terminal. On a weaker computer, use the name qwen3:8b instead. Result: Qwen answers your question in the same window.
Check right away where it loaded:
- Keep the chat open, start a second terminal and type ollama ps.
- Look at the PROCESSOR column. 100% GPU means the weights are fully in video memory, and 100% CPU means system memory. Shares such as 48%/52% CPU/GPU mean they are split [2].
- If you see a CPU share, pick a smaller size from the table and look again.
According to the documentation on 21 November 2026, the files sit in a .ollama/models folder [2]. It is in your home folder on a Mac, inside /usr/share/ollama on Linux and in your user profile on Windows. If the system disk is small, the OLLAMA_MODELS variable sets another location.
Next, set the context windowcontext window. The context window is the amount of text, counted in tokens, that a language model sees at one time: instructions, documents, the question and its own answer., which is the amount of text the model takes into account at once. For v0.40.17 of the app, the latest release on 21 November 2026, its Local FAQ puts the default at 4096 tokenstoken. A token is a piece of text that a language model works with: a word, part of a word, or a symbol. [2]. That is too little for a long document. You change it with the OLLAMA_CONTEXT_LENGTH variable, with /set parameter num_ctx inside the chat, or with the num_ctx parameter in an API request.
Memory use grows with the window. With Flash Attentionattention mechanism. The attention mechanism is the way a neural network decides, for each word, which other words of the text to rely on and how strongly. on, OLLAMA_KV_CACHE_TYPE set to q8_0 halves the K/V cache memory of f16 [2]. The documentation calls the lossloss function. A loss function in machine learning is a formula that turns a model’s error into one number: the smaller it is, the closer the model’s answers are to the right ones. of precision very small [2]. The value q4_0 uses about a quarter, with a small to medium loss of precision [2].
According to its library page, qwen3.8 has thinking mode on by default, and you can turn it off per request [5]. The reasoning_effort setting controls its depth. The Qwen3.8-27B card on Hugging Face (August 2026) recommends temperature 1.0, top_ptop-p. Top-p sampling, also called nucleus sampling, is a generation setting in which a model picks the next token only among the most likely options whose total probability reaches a threshold p. 0.95 and top_k 20 with thinking [7]. Without thinking, it recommends temperature 0.7, top_p 0.80, top_k 20 and presence_penalty 1.5.
Step 3. Keep the server on this computer only
By default the server listens on 127.0.0.1, port 11434, according to the documentation on 21 November 2026 [2]. So it accepts requests only from the computer itself. Keep it that way: the OLLAMA_HOST variable changes the address, and a value of 0.0.0.0:11434 opens the server to the whole network.
- Check that this variable is not set or equals 127.0.0.1. On a Mac, you set variables with launchctl setenv and restart the app. On Linux, you need systemctl edit ollama.service and an Environment line in the [Service] section, then systemctl daemon-reload and a service restart. On Windows, you set the variable in your account settings and restart the app from the Start menu.
- Do not open the server through tunnels. The documentation gives examples for ngrok and Cloudflare Tunnel [2], but for a closed setup this is a direct path to the internet.
- Turn off the cloud features: set OLLAMA_NO_CLOUD=1, or add the disable_ollama_cloud setting with the value true to ~/.ollama/server.json [2]. Then restart the app.
Result: after the restart, server.log shows the line Ollama cloud disabled: true, and cloud modelscloud AI model. A cloud AI model is a neural network that runs on the servers of a provider. You use it through a website, an app or an API, and you pay for the volume of requests. and web search disappear from the menu [2].
The app’s developers say that in local use they do not see your requests or data. They process cloud requests but, in their words, do not store them or train models on them [2]. That is the company’s claim, and Step 6 helps you check it on your own computer.
If colleagues in the office need Qwen, you need a shared server with access rights and logs, and the last section covers it. A single computer should not listen to the network for colleagues.
Step 4. Block outbound connections
Windows blocks the app’s outbound connections with a firewall rule, Linux with a service setting, and any system with a container without a network. Apple describes the firewall in the Mac’s System Settings only for incoming connections [10].
| System | How to block the way out | Command or setting | How to check |
|---|---|---|---|
| Windows | a firewall rule for one program | New-NetFirewallRule -DisplayName QwenOut -Direction Outbound -Action Block -Program with the full path to ollama.exe | a test download fails with a network error |
| Linux | a limit on the service | systemctl edit ollama.service, with IPAddressDeny=any and IPAddressAllow=localhost in the [Service] section, then daemon-reload and restart | a test download fails with a network error |
| Any, in a container | a Docker container without a network | docker run with the --network none flag | ip link show inside lists only lo |
| Mac | the Settings firewall does not fit | a container or a separate outbound connection monitor | a test download does not go through |
The example in Microsoft’s documentation for the New-NetFirewallRule cmdlet blocks outbound traffic with -Direction Outbound and -Action Block [9]. The -Program parameter ties the rule to one program, here the ollama.exe server that downloads models and answers requests. You run the cmdlet in PowerShell as an administrator.
So the rule covers that one program only. The app keeps its programs in %LOCALAPPDATA%\Programs\Ollama and its downloaded updates in a separate folder under %LOCALAPPDATA% [3]. Its documentation does not say which program fetches the updates. Set the same rule on every program in the first folder, or confirm the block with the Step 6 check.
On Linux, the rule goes on the service. According to the systemd manual, outbound traffic is checked by its destination [11]. An address in the IPAddressAllow list passes; otherwise an address in IPAddressDeny is blocked. The name any means all addresses, and localhost means 127.0.0.0/8 and ::1, so the service can reach only the computer itself.
In a container started with --network none, Docker’s documentation says, only the loopback interface lo with the address 127.0.0.1 exists [8]. You cannot publish a port from such a container. So you work with the model through docker exec and put the files into a mounted folder in advance.
Apple describes the firewall in the Mac’s System Settings as protection against connections started by other computers [10]. Its help page says nothing about outbound connections from apps. macOS also has a built-in Packet Filter (pf), which does rule-based filtering [20]. Apple’s technical note TN3165 of 27 February 2024 calls pf an advanced feature for users and site admins [20].
The same note warns that rules you install might clash with those of macOS system services or other products [20]. So for most people, a container or a separate outbound connection monitor is the simpler way out on a Mac. Docker Desktop on a Mac does not pass the graphics card into a container, so Qwen will run on the processor [2].
Result: a test download of a small file fails with a network error. For an update, you remove the rule, download the new version and set the rule again.
Step 5. Do not give the model tools without oversight

The danger is a model with tools, and the qwen3.8 library page itself offers ollama launch commands for four agents [5]. They are Claude Code, OpenCode, Hermes Agent and OpenClaw. This gives Qwen a terminal and files, and, in Cohere’s words, whatever sits in the weights gets a keyboard inside your network [1].
If you still need an agent, Cohere’s advice to security teams turns into these actions:
- Run the agent in a container where outbound connections are blocked by default, as in Step 4.
- Check every tool call before it runs, not the model’s reasoning about it.
- Set alerts for unknown cloud addresses and unexpected tool calls. Watch 169.254.169.254 in particular: AWS, Google Cloud and Azure servers get temporary access keys through it [1].
- Block unprivileged namespaces in the agent’s container. In Cohere’s test without a network, the model was refused administrator rights and used unshare to become root in its own namespace [1].
- Keep the test environment separate from production.
- Test the code it writes with live attacks before you put that code to work.
In a separate test, Cohere had the model finish a started terminal command hundreds of times, from several points of this case. In 23–36% of those completions, it contacted 169.254.169.254 before deciding to download anything [1]. That share counts repeats of one caught case, not all runs, so the two figures do not conflict. Cohere notes there was nothing to take there in the test environment, and a rare reflex does not prove intent.
Cohere advises treating third-party weights like an unsigned program [1]. In my talk on building software with AI, I said open-source models now write code very well and work on their own [16]. If a model can work on its own, it gets rights only from a list.
In my talk on messenger hacks, I said that we cannot use technologies we cannot trust [17]. I was speaking about chats stored on other people’s servers, but the same thought applies to an agent with access to your files. Result: you have a written list of the folders, commands and addresses the agent may use, and everything else is blocked.
Step 6. Check that the Ollama server cannot reach the internet
The check takes a few minutes and has four points, and if any of them fails, the setup is not finished. The points follow the order of the steps:
- The server is in place. A request with curl http://localhost:11434 from this computer gets an answer. The same port at the computer’s IP address does not open from another machine.
- The cloud is off. The server.log file shows the line from Step 3. On a Mac the log is in ~/.ollama/logs, on Windows in the Ollama folder inside LOCALAPPDATA, and on Linux journalctl -u ollama shows it [3][4].
- The way out is closed. A test download with ollama pull qwen3:0.6b fails with a network error, and in a container with --network none, ip link show lists only lo.
- Qwen is where you expect it. The check from Step 2 shows the GPU share you need.
If the first point fails, look for the address 0.0.0.0 in the variable from Step 3 or for an open tunnel. If the second fails, make sure the app restarted after your change. If the third fails, the rule sits on the wrong program, or the service did not reread its settings after systemctl daemon-reload.
Repeat the check after every update and every change of Qwen version. An update means removing the rule and setting it again, and that is the easiest moment to make a mistake.
The check also has a legal side if the model will read personal data and the EU GDPR or the UK GDPR applies to you. Article 32(1) of either asks for security that matches the risk [12][22]. Its point (d) adds a process to test and evaluate those measures regularly [12][22]. Blocking outbound traffic and repeating this check after each update is one such measure.
Common mistakes after setup and how to fix them
The eight cases below draw on the app’s documentation on 21 November 2026 [2]:
| Mistake | Cause | Fix |
|---|---|---|
| Answers are slow | the check from Step 2 shows CPU: the file does not fit in video memory | pick a smaller size from the table above |
| Qwen forgets the start of a long document | the documented default context window is 4096 tokens | set num_ctx as in Step 2 |
| Error 503 | the queue is full; by default it holds 512 requests | raise OLLAMA_MAX_QUEUE or send fewer requests at once |
| Files stop downloading | Step 4 closed the way out, or HTTP_PROXY is set instead of HTTPS_PROXY | remove the rule for the download; for a proxy, set only HTTPS_PROXY |
| The server is visible on the network | the address variable equals 0.0.0.0 | set it back to 127.0.0.1 and restart the app |
| No acceleration in Docker on a Mac | Docker Desktop does not pass the graphics card into a container | run without a container and use an outbound connection monitor, or accept processor speed |
| Memory stays busy after work | the weights stay in memory for 5 minutes after a request | unload them with ollama stop or set OLLAMA_KEEP_ALIVE |
| The disk is full | the files take tens or hundreds of gigabytes | move them to another disk with the variable from Step 2 |
The proxy is easy to get wrong. Files download only over HTTPS, and HTTP_PROXY can break client connections to the server itself, so the documentation advises setting HTTPS_PROXY only.
The last mistake is not technical: accepting the model’s answer without review. In my book on how not to go bankrupt, I advise using AI assistantsAI assistant. An AI assistant is a helper built on a neural network that, at a person’s request, answers questions, prepares texts and does simple tasks in work programs. to check facts or find research papers [13]. But you always judge their reliability yourself. The same rule applies to a local setup: a person reviews the answer.
When a laptop is enough and when you need a server
A laptop or a Mac is enough for one person. Steady load from several employees needs a server with a dedicated serving engine, as the Qwen team itself advises [7]. I set the threshold by task:
- One person: drafts, translation, reviewing your own documents. A laptop or Mac with a file up to 9.3 GB, which is qwen3:14b, is enough. The 18 GB qwen3.8 also works if you have memory to spare.
- Several employees and steady load. For production workloads and high throughput, the Qwen3.8-27B card recommends dedicated serving engines such as SGLang, vLLM or TokenSpeed, and that means a company server.
- An agent with access to a terminal and data. Only in a container without a network and with the tool call checks from Step 5, on any hardware.
I believe your own infrastructure pays off in freedom. In my talk on building software with AI, I said with your own infrastructure you are not limited and can do a great deal [16]. Large companies go further, as I described in my talk on server crashes [15]. They install closed servers with no internet access and process their data there.
The main argument against: your own server needs people and maintenance. Someone has to handle updates, access rights, logs and the Step 6 check on a regular basis, or a closed server slowly stops being closed.
That is why I advise starting with one computer and this guide. Move to a server when more than two or three people need Qwen every day and the company has someone to look after it.



