Put open-source AI on your own computer
Specs → install → one command to run it → works offline ｜ tested myself
黃志弘 Leo
Published 2026-09-11 · Updated 2026-10-07 · 1 readers
Why you want a local AI setup
Cloud AI is great, but it rests on three things: the platform is still running, your account still has access, and the network is up.
If any one of those breaks, you are left with nothing.
Put open-source AI on your own computer and your data stays with you, with some of it still working offline.
Most of all, this set of tools is genuinely yours.
I tested this whole flow on my own machine. Following it takes about ten minutes.
Step one: check your computer can handle it
Only one number matters: memory (RAM). To check it, on Mac open the Apple menu → About This Mac; on Windows go to Settings → System → About.
| Your memory | Model to use (checked current as of October 2026) | What to expect |
|---|---|---|
| 8 GB | Qwen3.5 2B or Gemma 4 E2B | Small models, fine for everyday questions |
| 16 GB | Qwen3.5 4B (recommended starting point) | Good with Chinese; move up to 9B once you are not running much else |
| 32 GB and up | Qwen3.8 27B (latest) ／ Gemma 4 12B | Clearly smarter, still leave memory for your other apps |
Download the names given here and you get the standard compressed build (beginners do not need to study what quantization is). The same model also comes in a heavier high-precision build, skip that one as a beginner. Also leave 10-20 GB of disk space for model files. (Updated 2026-08-18: Qwen3.8 is out but only in the large 27B size, so for the 8 GB and 16 GB tiers the small Qwen3.5 models are still the best pick.)
A graphics card is not required: Macs with M-series chips run this as is; on Windows an NVIDIA card is much faster, and without one your CPU can still run the small models (Windows needs version 10 22H2 or later, or Windows 11).
Step two: pick one of three tools
| Tool | Who it suits | What stands out |
|---|---|---|
| LM Studio | Complete beginners | Graphical interface with a built-in model store, click around and chat; free, though the interface app itself is not open source |
| Ollama | People who want to plug it into other tools | Most integrations and tutorials; one command to run it, and there is a basic window interface too (core is MIT open source) |
| Jan | People who want it fully open source | Interface like ChatGPT, code fully open source (Apache-2.0), the closest fit to keeping it in your own hands |
If you cannot decide: pick LM Studio if you would rather not touch the command line, Ollama if you want to follow most of the tutorials online, Jan if open source matters to you. All three are free; you can switch any time, but each tool manages its own model files, so switching usually means downloading again. Licences: the current Qwen3.5 and Gemma 4 are both Apache-2.0 open source, fine for personal and commercial use. One more thing: when the makers stop publishing an older model (Gemma 3 lost support once Gemma 4 shipped), the file already on your drive does not disappear and keeps working, which is the whole point of local AI.
Step three: install it and get it running
If you picked LM Studio or Jan (graphical interface)
Download and install from the official site (lmstudio.ai ／ jan.ai), open it, search the built-in model page for 「Qwen3.5 4B」 (with 8 GB of memory, search 「Qwen3.5 2B」 instead), hit download, and start chatting when it finishes. All of it with the mouse.
If you picked Ollama (command line)
Download and install from ollama.com, open a terminal (Mac: Terminal; Windows: PowerShell) and type:
ollama run qwen3.5:4bThe first run downloads the model for you (about 3 to 4 GB, stored on your computer, downloaded once). When the prompt appears you can start chatting. On a computer with 8 GB of memory, swap the command for qwen3.5:2b.
For reference, from my own run (M4 chip, 16 GB memory, one test, speed varies with output length and heat):
From install to chatting, under ten minutes.
On a clean machine, an 8B-class model answers in about 15-20 seconds. With other apps piled up and memory tight, the same class drops to over a minute per answer.
So close the memory-hungry apps before you run local AI. It is several times faster that way.
It also leans to Simplified Chinese by default. If you want Traditional, say 「answer in Traditional Chinese」 in your question.
Step four: confirm it really runs offline
Turn off Wi-Fi, then ask it a question.
It answers anyway, because the whole model is on your drive and needs no server. That is what keeping it in your own hands means.
That holds while you stay on a local model and leave the tool's built-in web search and cloud model features switched off. Those go online the moment you turn them on, and they are optional extras.
Common questions
Is local AI slow?
Match the model size to your memory. It only drags when you cram in something too big. Pick from the table above and you get the 「good enough for everyday questions」 level. Do not compare the speed with the cloud.
How is local AI different from ChatGPT?
A small local model has limited knowledge and reasoning. It suits everyday questions, translation, summaries and first drafts. Deep work still goes to the cloud. Local AI is your backup, and your option when privacy matters.
The model files are big and my connection is slow
You download a model once, then it lives on your drive for offline use. In a hurry, grab a small one first (2B class is about 3 GB) and add a bigger one later.
The thing most worth building right now is still your own thinking and skill. Use AI, but real stability has two layers: knowing how to think and act without AI, and having a set of tools you control when the cloud is unavailable. This page is the start of the second layer.
Contact / Follow
Want to turn AI into a system that actually works for you? See my services, or start with a free 30-minute chat.