Yes, you can have an AI at ChatGPT's level running inside your own computer, with no internet, no monthly fee and without sending a single piece of your company's data to a third-party server. What almost nobody tells you is that choosing the model is not a technology decision, it is back-of-the-napkin math: how much memory your machine has. Getting that math wrong is why so many people install a local AI, watch the computer freeze and conclude that "this doesn't work".
This post gives you the rule of thumb to know what runs on your machine, what to expect from each model size (in business tasks, not benchmark scores), and when local AI actually beats paying for an API (the per-message AI rental that ChatGPT, Claude and Gemini charge).
The only math that matters: memory
An AI model is a giant file that has to fit entirely in the computer's memory while it works. Think of an employee who can only produce if all their material is on the desk: if the desk is small, it doesn't matter how brilliant the employee is, there is nowhere to work. The desk is your machine's memory (RAM, or the graphics card's memory, which is faster).
The model size shows up in the name as a number followed by B, for billions of parameters: 4B, 8B, 32B, 70B. The rule of thumb, already accounting for the compression everyone uses today: a compressed model takes up roughly 60 to 70 percent of that number in GB. In practice:
- A 4B model: needs about 3 GB free.
- An 8B model: about 5 to 6 GB free.
- A 14B model: about 9 to 10 GB.
- A 32B model: about 20 GB.
- A 70B model: 40 GB or more. This is where the laptop ends; it is workstation or server territory.
Free is the key word. If the laptop has 8 GB of RAM, the operating system and the browser already eat 4 or 5. That leaves room for a 4B model, not an 8B. When the model doesn't fit, the computer doesn't warn you: it starts using the disk as improvised memory and the AI starts answering one word every two seconds. That is what freezes the machine of anyone who sizes it wrong.
Compression (quantization): what it is and why you should want it
Every local model you install today comes compressed; the technical term is quantization. The honest analogy is the WhatsApp photo: when you send a picture, the app compresses it, the image loses a bit of definition nobody notices and it now fits on any phone. AI models work the same way: the standard compressed version (called Q4 in the catalogs) delivers 95 percent or more of the quality using half the memory.
The practical decision: a bigger compressed model almost always beats a smaller uncompressed one. Between running an 8B at maximum quality and a compressed 14B in the same memory, take the 14B. Tools like Ollama and LM Studio (free programs that download and run the model in two clicks, no programming) already ship the compressed version by default, so in practice you don't configure anything; you just need to know that this is what is happening.
What runs on your machine, by hardware tier
Regular office laptop (8 GB of RAM, no graphics card)
Runs models with 3 to 4 billion parameters: Gemma 3 4B (from Google), Qwen3 4B or Phi-4 mini (from Microsoft). Good for summarizing documents, classifying email, drafting standard replies. Don't expect sophisticated analysis; expect a fast, tireless intern.
Good laptop or desktop (16 GB of RAM)
The 7 to 9 billion range: Llama 3.1 8B (from Meta), Qwen3 8B. This is the first step where local AI becomes a serious work tool: internal support, document triage, the first draft of a proposal. Without a graphics card the speed is a comfortable reading pace, around 5 to 10 words per second.
Desktop with a gaming graphics card (8 to 12 GB of video memory)
An RTX 3060 or 4060 Ti, a card many companies already have sitting in a designer's machine (or a partner's kid's PC), changes the game: the same 8B models answer almost instantly (30 to 60 words per second), and a 12 GB card runs a 14B like Qwen3 14B or Phi-4, which already sustains contract analysis and writing with context.
Mac with an M chip (16 to 32 GB)
M-chip Macs are the quiet shortcut in this story: their memory is unified (it serves the processor and the graphics at the same time), so a 32 GB MacBook runs a compressed 32B like Qwen3 32B, something that on a PC would require a 20 GB graphics card. If your team already uses Macs, you may already own the hardware.
Workstation or server (24 GB+ of video memory)
An RTX 3090 or 4090 (24 GB) runs 32B models with room to spare and at high speed. Two cards, or one professional card, put the 70B class on the table: Llama 3.3 70B and the distilled versions of DeepSeek R1, which reason at a level comparable to the paid ChatGPT in a good share of office tasks. This is the hardware that makes sense when the AI will serve the whole operation, not one person.
What to expect from each size (in tasks, not benchmarks)
Forget test scores; what matters is what the model can handle in your routine:
- 3 to 4B: summaries, classification, extracting simple data from documents. It will fail at multi-step reasoning. Use it as a filter and triage layer, with human review.
- 7 to 9B: consistent conversation, answering customers from your company's knowledge base, drafting business copy. The best cost-benefit to start with.
- 14 to 32B: contract analysis, reports with numbers, writing that goes to a client with little review. Here local AI stops being an experiment and replaces an API subscription for many tasks.
- 70B+: the ceiling of what runs in-house today without becoming an infrastructure project. For most business tasks the gap to the paid ChatGPT is small. The privacy gap is total.
Local or API: the business math
AI through an API is rent: you pay per message, forever, and every company document travels to the vendor's server. Local AI is a purchase: the cost is the hardware (which you often already own) and the electricity, and the data never leaves your network.
The decision rule we use: the API wins when volume is low, the task demands the absolute top of intelligence and the data is not sensitive. Local wins when there is high, repetitive volume (triage, classification, internal support), when the data is customer, financial or legal data, or when the industry has compliance rules (the norms that say where data can and cannot live). And it is not either-or: the most common architecture we build is local AI handling 80 percent of the cheap, sensitive volume, with the API stepping in only for the 20 percent that requires a frontier model.
The right question is not "which model is best"
It is "what is the smallest model that solves my task". A smaller model is faster, cheaper to run and fits on a machine you already own. Whoever starts with the 70B because it is "the best" spends on hardware before knowing whether the task needed all of that. Whoever starts with an 8B on an existing machine finds out in a week, spending zero, whether local AI solves the problem. Then they decide whether to scale.
Test it with a real case from your operation: take 20 real documents or 20 real support tickets, run them through the model that fits your current machine and compare with what your team produces. The result of that test is worth more than any benchmark table, because it is your task, on your data, on your hardware.
