Every message you send to ChatGPT, Gemini, or Claude travels across
the internet, lands on someone else’s server, gets logged, gets stored,
and — depending on the provider — might get fed back into a training
pipeline that makes the next model a little smarter using your
words. Your business plans, your medical questions, your private
thoughts: all of it, sitting on infrastructure you don’t control,
governed by terms of service that change whenever the company feels like
it.
If that makes you uncomfortable, good. You’re paying attention.
The good news is you don’t have to choose between having an AI
assistant and having privacy. You can run a private AI assistant
no cloud provider ever touches. Everything stays on your
hardware. No API calls leave your network. No corporation gets to peek
at your prompts. This guide shows you exactly how.
The Problem
With Cloud AI (It’s Worse Than You Think)
Let’s be specific about what happens when you use a cloud-hosted AI
assistant.
Your data gets stored. Most providers retain
conversation logs. OpenAI keeps them for 30 days minimum, longer if you
haven’t opted out. Google’s Gemini conversations can be reviewed by
human employees. Anthropic retains data for safety monitoring. Even
providers that promise “we don’t train on your data” still
store your data on their servers, behind their security
practices, subject to their jurisdictions.
Your data can be subpoenaed. If your conversations
live on a US server, they’re subject to US law enforcement requests.
CLOUD Act provisions mean this applies even if the data belongs to
someone in Europe or elsewhere. You have no visibility into whether your
data has been requested, disclosed, or compromised.
Your data might train future models. OpenAI’s
default setting trains on your conversations unless you explicitly opt
out. Even when you do opt out, the data still sits on their servers. The
opt-out is a policy promise, not a technical guarantee. Policies change.
Companies get acquired. Databases get breached.
Metadata is data too. Even if a provider encrypts
your messages, they still know when you’re using AI, how
often, what topics (based on content filters), and
what patterns your usage follows. For journalists, lawyers,
activists, or anyone handling sensitive information, this metadata alone
can be damaging.
The uncomfortable truth is that using cloud AI means trusting a
corporation with your inner monologue. Some people are fine with that.
If you’re reading this article, you probably aren’t.
The Fully
Local Stack: What “No Cloud” Actually Means
When we say private AI assistant no cloud, we mean
it literally. No data leaves your machine. No API keys phoning home. No
telemetry. No “anonymised” analytics. The stack looks like this:
Ollama handles the model layer. It runs large
language models locally on your hardware — your CPU, your GPU, your RAM.
Models download once and run entirely offline. Ollama exposes a local
API on localhost:11434 that never needs to touch the
internet after the initial model download.
OpenClaw sits on top as the orchestration layer. It
connects to Ollama’s local endpoint, manages conversations, runs
automations, handles tool integrations, and gives you an actual
assistant — not just a raw model you have to prompt-engineer
from scratch. OpenClaw can route to cloud providers, but it can also run
entirely against local models with zero outbound connections.
Local models — Llama, Mistral, DeepSeek, and others
— provide the actual intelligence. They run on your hardware, respond to
your prompts, and forget everything the moment you clear the
conversation. No training feedback loops. No server logs. Just weights
and biases executing on your own silicon.
This is genuinely self-hosted AI in the truest
sense. The entire pipeline — from your keyboard to the model’s response
— never leaves localhost.
What You Gain Going Fully
Local
Absolute data sovereignty. Your conversations exist
on your disk and nowhere else. When you delete them, they’re gone. No
retention policies, no 30-day windows, no “we keep backups for disaster
recovery” caveats.
Zero network exposure. With the right configuration,
your AI assistant makes no outbound connections whatsoever. There’s
nothing to intercept, nothing to man-in-the-middle, nothing for a
compromised DNS to redirect.
No account required. No email verification, no phone
number, no credit card. No identity linked to your usage. You’re a
person talking to software on your own computer, and that’s the end of
the story.
Regulatory simplicity. If you handle data subject to
GDPR, HIPAA, or similar regulations, keeping everything local means you
don’t need to worry about data processing agreements, cross-border
transfers, or third-party compliance audits. Your data processor is your
own machine.
Resilience. Cloud services go down. APIs get
rate-limited. Providers change pricing, deprecate models, or shut down
entirely. Your local stack works as long as your hardware does. No
subscription renewals, no surprise pricing changes, no “we’re pivoting
our business model” emails.
What You Lose (Honesty
Matters)
Going fully local isn’t free. Here’s what you’re giving up, and you
should know this before committing.
Raw capability. The best cloud models — GPT-4o,
Claude Opus, Gemini Ultra — are significantly more capable than anything
you can run locally, especially for complex reasoning, coding, and
nuanced writing. Local models are good and getting better fast, but
there’s still a gap. A 70B parameter model running on your Mac is
impressive; it’s not frontier-model impressive.
Speed on CPU-only hardware. If you don’t have a
decent GPU (or an Apple Silicon Mac with unified memory), inference can
be slow. A 13B model on a CPU-only machine might take several seconds
per response. Usable, but not snappy.
Multimodal capabilities. Most local models handle
text well. Vision, audio, and other modalities are more limited in the
local ecosystem, though this is changing rapidly with models like LLaVA
and Whisper.
Context window size. Cloud models routinely offer
128K+ token context windows. Local models typically max out at 8K-32K
tokens in practice, depending on your available RAM. Long document
analysis is harder locally.
For many privacy-conscious users, these trade-offs are entirely
acceptable. You’re not trying to replace a research lab; you’re trying
to have a capable assistant that respects your boundaries.
Setting Up Your
Local-Only Private AI Assistant
Here’s how to get the full stack running with no cloud
dependencies.
Install Ollama. Head to ollama.com and download the
installer for your platform. On macOS, it’s a single app. On Linux, a
one-line curl command. Once installed, Ollama runs a local server
automatically.
Pull your first model. Open a terminal and run:
ollama pull llama3.1:8bThis downloads Meta’s Llama 3.1 (8 billion parameter version) to your
machine. It’s roughly 4.7GB. After this download, you never need
internet access for this model again.
Install OpenClaw. Follow the installation at
openclaw.com. OpenClaw runs as a local daemon on your machine. During
setup, when it asks for model configuration, point it at your local
Ollama instance:
providers:
ollama:
baseUrl: http://localhost:11434
models:
- llama3.1:8bRemove all cloud provider keys. This is critical. If
you’ve previously configured OpenClaw with API keys for OpenAI,
Anthropic, or others, remove them entirely. Don’t just switch the
default model — delete the keys. What doesn’t exist can’t be
accidentally used.
# Remove or comment out all cloud providers
# openai:
# apiKey: sk-...
# anthropic:
# apiKey: sk-ant-...Test it. Send a message through OpenClaw. Watch the
response come back. Check your network monitor — you should see zero
outbound connections. The entire exchange happened on
localhost.
If you’re running OpenClaw with clawdbot on Telegram
or moltbot on Discord, those bot connections obviously
require internet for the messaging platform itself. But the AI inference
— the part where your actual prompt content gets processed — stays
entirely local. The messaging layer sees your messages (that’s how
messaging works), but the AI model processing them never leaves your
machine.
Locking Down
the Network (For the Truly Paranoid)
If “trust but verify” isn’t enough and you prefer “verify then verify
again,” here’s how to ensure your AI stack can’t phone home,
even if a future update tries to add telemetry.
Firewall Ollama’s outbound access. On macOS with the
built-in firewall, or on Linux with
iptables/nftables, block all outbound
connections from the Ollama process except to localhost. On
macOS, Little Snitch makes this trivial — you’ll see every connection
attempt and can deny them individually.
Run on an air-gapped machine. The ultimate privacy
setup: a machine with no network connection at all. Download your models
on a connected machine, transfer them via USB, and run everything
offline. Ollama stores models in ~/.ollama/models — just
copy that directory.
DNS sinkhole. If you run Pi-hole or AdGuard Home on
your network, add the domains for major AI providers to your blocklist.
This catches any accidental cloud API calls from misconfigured
tools.
Monitor with Little Snitch or tcpdump.
Periodically audit your machine’s outbound connections while using your
AI assistant. You should see exactly zero connections to external AI
APIs. If you see something unexpected, investigate immediately.
# Quick network audit — watch for any AI API connections
sudo tcpdump -i any host api.openai.com or host api.anthropic.com or host generativelanguage.googleapis.comIf that command shows any traffic while you’re using your
local assistant, something is misconfigured. Fix it before
continuing.
For a deeper dive into securing your OpenClaw installation beyond
just model privacy, the OpenClaw
security hardening guide covers encryption at rest, access controls,
and more.
Choosing the Right Local
Model
Not all local models are created equal, and your choice depends on
your hardware and your needs.
Llama 3.1 (8B and 70B) — Meta’s open-weight model is
the default recommendation. The 8B version runs comfortably on machines
with 8GB+ RAM. The 70B version needs 40GB+ RAM (or a high-VRAM GPU) but
delivers genuinely impressive results. Llama’s license is permissive for
personal and commercial use, and Meta has been transparent about
training data.
Mistral (7B) and Mixtral (8x7B) — Mistral’s models
punch above their weight class. The 7B model is remarkably capable for
its size, and Mixtral’s mixture-of-experts architecture gives you
near-large-model quality with lower resource requirements. Excellent for
users with mid-range hardware.
DeepSeek-R1 (distilled variants) — DeepSeek’s
reasoning-focused models are strong for analytical tasks. The distilled
versions run locally and handle logic, math, and structured thinking
well. Worth pulling if you need more than conversational ability.
Phi-3 (3.8B) — Microsoft’s small model is
surprisingly good for its size. If you’re running on limited hardware —
an older laptop, a Raspberry Pi 5, or a NAS box — Phi-3 gives you a
functional assistant in under 3GB of RAM.
Gemma 2 (9B and 27B) — Google’s open models offer
strong general performance. The 27B variant is a sweet spot between
capability and resource requirements.
Pull multiple models and switch between them based on the task:
ollama pull llama3.1:8b
ollama pull mistral:7b
ollama pull deepseek-r1:8b
ollama pull phi3:3.8bOpenClaw lets you route different tasks to different models, so you
could use a smaller model for quick questions and a larger one for
complex work — all locally.
The Privacy You Deserve
There’s a philosophical point buried in all this technical detail,
and it’s worth stating plainly: you shouldn’t have to trust a
corporation to think privately.
AI assistants are becoming integral to how people work, learn,
create, and organise their lives. The idea that all of that cognitive
activity should flow through corporate servers, get logged in databases,
and potentially feed the next generation of models trained on your
thought patterns — that should bother everyone, not just the
privacy-conscious.
Running a private AI assistant no cloud provider can
touch isn’t paranoia. It’s the same instinct that makes you close the
blinds at night. It’s the same reason you don’t CC a stranger on every
email. Some things are yours, and having an AI that processes them
should not require surrendering ownership of your inner dialogue.
The local AI ecosystem is maturing fast. Models that required data
centre GPUs two years ago now run on a laptop. Tools like OpenClaw make
the local experience genuinely usable — not a hobbyist curiosity, but a
real daily-driver assistant. Whether you’re running clawdbot for
personal use on Telegram or moltbot in a team Discord, the option to
keep inference fully local exists today.
If you’re still weighing whether OpenClaw is the right foundation for
your private setup, the safety and trust
overview covers the project’s approach to security, data handling,
and why the architecture was built with local-first as a core
principle.
Your data. Your hardware. Your assistant. No exceptions.