Local AI models, a beginner's guide
Ollama, Gemma, censored and uncensored, all in one go
黃志弘 Leo
Published 2026-08-27 · Updated 2026-08-29 · 1 readers
You need no coding background for this one; it covers 7 core terms and ends with the latest model ranking as of Aug 2026.
Why local models are worth understanding
ChatGPT, Claude and Gemini are all cloud models.
The computing happens on someone else's servers and you connect to it.
But an AI model can also run right on your own computer. That is a local model, sometimes called an on-device model.
This page clears up the basics in one pass: cloud vs local, what Ollama is, what Gemma is, and how censored differs from uncensored,
so the words stop sounding like fog next time you meet them.
1One idea first: cloud vs local
Cloud model = ordering delivery, local model = cooking at home
delivery
cooking at home
Cloud model: like ordering delivery.
The chef is good (the model is strong) and the menu is long (all the features are there),
but you pay, you wait (queues or rate limits), and your food passes through someone else's kitchen (your data goes to their servers).
Local model: like cooking at home.
Free, cook whenever you feel like it, ingredients (your data) never leave the house,
but you are limited by your kitchen equipment (your hardware), so the food usually does not match a five-star chef.
It takes the work that is high volume, does not need top quality, saves money, or has to run offline,
and the important work still goes to a top cloud model.
2The terms you keep seeing
It is a model family, not a tool and not a company.
Llama comes from Meta, Qwen from Alibaba, and Gemma is Google's line.
7B means 7 billion parameters, 32B means 32 billion.
A bigger number usually means smarter, and also hungrier for your computer's resources (memory, speed).
(commercial models like ChatGPT and Claude all have this layer).
Uncensored means that layer was removed or reduced, so the model refuses less often.
Uncensored does not mean it will go out of its way to teach people harm. It means fewer restrictions, and it still has limits.
The community takes an existing model (Gemma, Llama) and operates on it, removing the machinery that handles refusals,
and the result is usually labelled abliterated.
3Three steps from install to daily use
Check your memory first, it decides which tier you can run
Local models live mainly on memory (RAM). Check how much you have, then find your row:
| Your memory | Tier you can run | What that means |
|---|---|---|
| 8GB | 7B class | Entry level on a small machine, good for trying it out |
| 16GB | 12B class | The sweet spot most people are on, mainstream laptops land here |
| 32GB and up | 35B class | Starts to come close to cloud models |
Mac → Apple menu in the top left →「About This Mac」
Windows → Settings → System → About
The GB figure in the memory row is the one you want.
Do not agonize over it, on 16GB just take Gemma 4 12B
There are plenty of models, but your first one does not need a comparison spreadsheet:
on a computer with 16GB of memory, download Gemma 4 12B (from Google) and pick the standard build.
On 8GB take Qwen3.5 2B, on 32GB and up take Qwen3.8 27B or Qwen3.6-35B.
Once you are comfortable and have a specific need (writing without getting refused, say), come back to the full comparison table below.
Hand it these four jobs first
Local models suit work that is high volume, repetitive and does not need top quality. That part is free from now on:
- Translating documents, throw foreign-language articles and emails at it
- Summarizing long text, pull the key points out of long reports and video transcripts
- Report drafts, let it write the first version and you edit
- Sorting, hand it a pile of notes and data to categorize
A local model makes the high-volume chores free. It does not replace everything.
Why do people call AI an 「AI employee」?
Because the division of labour looks a lot like hiring:
it is on call 24 hours a day, runs several things at once, shows up instantly, never takes leave or quits,
and the cost is a subscription or an electricity bill instead of a salary.
The difference: creativity, reading people, and carrying final responsibility still need a human.
So the sensible setup is to hand the repetitive chores to AI (a free local model already carries a big chunk of that),
and keep human time for the work only humans can do.
4Aug 2026 local model comparison
Compiled from published sources, not from my own benchmarks
Download ranking in the official Ollama library
These are actual download counts from the official Ollama model library (checked Aug 2026), so they show the local models the most people are really running:
| Rank | Model | Family／source | Downloads |
|---|---|---|---|
| 1 | Llama 3.1 | Meta | about 118 million |
| 2 | DeepSeek-R1 | DeepSeek | about 79 million |
| 5 | Gemma3 | about 39.6 million |
High download counts usually come from shipping early (more time to accumulate), name recognition and plenty of beginner tutorials.
That means 「most mainstream, easiest to find fixes for」, not strongest right now.
As of August 2026, newer models have passed Llama 3.1.
For real current ability, look at the recommendation table below, sorted by what you want to do.
Source: download statistics from the official Ollama model library (checked Aug 2026), see public write-ups such as Morph and BenchLM.
Current picks, by what you want to do
This table is compiled from public sources and community reviews. These are not numbers I benchmarked myself, so treat them as direction only.
| Model | Family／source | Parameters | Censorship | Best for |
|---|---|---|---|---|
| Qwen3.6-35B-A3B | Alibaba Qwen | 35B (only 3B active) | standard | The best all-rounder at the 32GB memory tier, fast like a small model, capable like a big one |
| Gemma 4 12B beginners pick this | Google Gemma | 12B | standard | A capable little assistant at the 16GB memory tier, the right first model |
| Qwen3.5 9B | Alibaba Qwen | 9B | standard | The largest of the small Qwen3.5 sizes. On a computer with less memory, use the 2B or 4B from the same family |
| Dolphin 3.0 Mistral | Community fine-tune (based on Mistral) | depends on the build | uncensored | Low refusal rate, small footprint, well regarded for writing and creative work |
| Hermes 3 (Llama 3.3) | Community fine-tune (based on Llama) | depends on the build | uncensored | Low refusal rate, strong at following instructions |
| Gemma Abliterated 9B | Community de-censored build (based on Gemma) | 9B | uncensored | Reasoning barely drops once the refusal layer is removed, runs at the 12GB tier |
Compiled from public write-ups on Hugging Face, Morph, InsiderLLM and Apidog (checked Aug 2026). What counts is how it feels when you run it yourself.
in the July and August 2026 roundups from r/LocalLLaMA (740,000+ members), the biggest local-model community in the world,
the two families mentioned most are Qwen3.6 and Gemma 4, which matches this table.
People with more memory (24GB and up) mostly go for Qwen3.6-27B or the larger Gemma 4 builds.
Start from your memory tier to get the matching parameter count, and take the standard build for general use;
only consider an uncensored build if you write, role-play, or keep running into refusals.
5Common questions
Will local models replace ChatGPT and Claude?
No, not at this stage.
A local model is the cost-saving workhorse, handling volume that does not need top quality;
anything important, anything you deliver, anything needing the strongest reasoning is safer on a top cloud model.
Do I need to code to install a local model?
No. With a tool like Ollama you open the app, pick a model to download, and start chatting. It is about the same as installing any app.
Are uncensored models legal? Are they safe?
Uncensored models are public open-source technology, and downloading and using them is not illegal.
But once the refusal layer is gone, more of the responsibility for the output sits with you,
so use your own judgement about what you do with it.
Do experts all use Ollama? How do LM Studio and llama.cpp differ?
The three are different wrappers around the same core:
underneath it is all the llama.cpp open-source engine, and Ollama and LM Studio are friendlier shells on top of it.
For beginners, Ollama (simplest commands) or LM Studio (full graphical interface) are both the mainstream answer;
people chasing maximum performance drive llama.cpp directly, and advanced Mac users add Apple's MLX framework for another step up.
Start with whatever is easy. Go a layer down when it stops being enough.
My computer is not powerful. Can I still run a local model?
Yes, just pick a smaller model (7B class, for example),
and the more memory you have, the bigger and smarter the model you can run.
Any computer with 16GB of memory or more is enough to start.
Want to learn alongside other people?
I run a free LINE learning community where you can ask AI tool questions directly
Join the LINE community