Run an AI Model Locally on Your Laptop

Cloud AI tools like ChatGPT and Claude are powerful — but every prompt you type is sent to an external server. For sensitive business content like client contracts, financial data, or HR matters, that's a real exposure risk. Running an AI model locally keeps everything on your machine, with zero data leaving your device and no monthly subscription required.

What you need

• A Mac (Apple Silicon M1 or later) or Windows PC with 8 GB+ RAM
• Ollama — a free tool that runs open-source models locally
• About 5 GB of free disk space for the model files

Step-by-step setup

1. Go to ollama.com and download the installer for your operating system. The install takes about two minutes.
2. Open Terminal (Mac) or Command Prompt (Windows) and run:
   ollama pull gemma4
   This downloads Google's Gemma 4 model — roughly 5 GB. Go make a coffee.
3. Once the download completes, start a conversation:
   ollama run gemma4
4. Type your prompt and press Enter. Your conversation stays entirely on your machine.

Which model should you use?

• gemma4 — Google's latest model. Best all-rounder for writing, summarising, and Q&A. Start here.
• mistral-small — Mistral's efficient model. Faster on older hardware and still very capable for business tasks.
• qwen3:4b — Alibaba's compact model. Excellent if your machine has limited RAM.

Using the chat interface

The Ollama desktop app includes a built-in chat window — no extra tools required. Once Ollama is installed, open the app and you'll find a ChatGPT-style interface where you can type prompts, paste documents, and switch between models. Your conversations stay entirely on your machine.

Practical uses once you're up and running

• Summarise confidential documents — paste a contract or board report and ask for a plain-English summary
• Draft sensitive communications — HR letters, disciplinary notes, internal memos
• Review financial documents — flag risks or anomalies without sending data to a third party
• Quick Q&A — ask questions about a document you've pasted in, without it leaving your machine