A Private Mind, Summoned

Companion deep-dive to the Lab Log entry. Written for someone who has never touched Ollama, a local model, or a GPU — every step explains why, not just what. All specifics are generic placeholders: swap IPs, names, and MACs for your own.

What This Actually Is (read this first)

You can run a genuinely capable AI chatbot entirely on your own computer — no cloud, no subscription, no per-message fee, and nothing you type ever leaves your house. This guide builds exactly that, and puts a clean chat window in front of it that you can reach from any device on your home network.

A few plain-English definitions so the rest makes sense:

• A "model" is the AI itself — a big file of trained weights. You download it once and run it locally.
• Ollama is the free program that runs a model on your machine. Think of it as the engine.
• Open WebUI is a browser page that looks like ChatGPT and talks to Ollama. It's the dashboard.
• Qwen & Gemma are two free, open models (from Alibaba and Google). Qwen is excellent at code; Gemma is warm and good at conversation. You can run both and switch between them.
• VRAM is the memory on your graphics card (GPU). It's the one number that decides which models you can run — bigger models need more VRAM.

The design in one sentence: a GPU computer does the thinking, an always-on low-power box (like a NAS) holds the chat window, and your phone wakes the GPU when you want it.

phone (a shortcut)
  └─ wake the GPU box  ── Wake-on-LAN magic packet
       └─ Open WebUI  ── always-on chat UI (on the NAS)
            └─ Ollama  ── serves the model on the GPU box
                 └─ a clean local name  ── ai.lan (DNS + reverse proxy)

The Setup at a Glance

What you need: a computer with a decent GPU (a gaming PC is perfect), a second always-on machine (a NAS, mini-PC, or Raspberry Pi), and both on the same home network. That's it.

Step 1 — Install Ollama and Download a Model

On the GPU computer, download Ollama from its website and install it (Windows, Mac, and Linux all supported). It runs quietly in the background.

Then open a terminal and pull a model. Start with one — the number after the name is roughly how big it is; a card with 16GB of VRAM comfortably runs a "14B" model.

# A conversational model (warm, good at chat):
ollama pull gemma3:12b

# A coding model (precise, good at code):
ollama pull qwen2.5-coder:14b

# Talk to it right in the terminal to confirm it works:
ollama run gemma3:12b

If it answers you in the terminal, the engine works. Everything else is just making it nicer to use.

Step 2 — Let Other Devices Reach It

By default Ollama only listens to the computer it's on. One setting opens it to your home network so the chat page (next step) can connect from the always-on box.

# On the GPU computer, then fully restart Ollama:
setx OLLAMA_HOST "0.0.0.0"      # "listen on the whole network, not just myself"

Test from another device's browser — visit http://<GPU-box-IP>:11434. If it says "Ollama is running," the network can reach it.

Step 3 — Put a Real Chat Window in Front of It

Typing in a terminal is no way to live. Open WebUI gives you a proper ChatGPT-style page — and the smart move is to run it on your always-on box (a NAS), so the chat page is always there even when the GPU computer is asleep. It runs in Docker (a way to run a program in a tidy, self-contained box).

# docker-compose for Open WebUI on the always-on box:
services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    ports:
      - "3000:8080"                                  # reach it at  <box-IP>:3000
    environment:
      - OLLAMA_BASE_URL=http://10.0.0.11:11434       # <-- the GPU computer's address
    volumes:
      - /volume1/docker/open-webui:/app/backend/data # remembers your chats & settings
    restart: unless-stopped

Start it, then open http://<always-on-box-IP>:3000 in any browser. The first screen asks you to create an account — this is local, stored on your own box, nothing sent anywhere. The first account becomes the admin. Your models appear in a dropdown, ready to chat.

Step 4 — Give It a Friendly Name

http://10.0.0.25:3000 is hard to remember. Two small pieces fix that: a local DNS record (maps a name to the box) and a reverse proxy (drops the :3000). The result is a clean address like http://ai.lan.

# 1) Local DNS record (on your network's DNS resolver):
#      ai.lan   ->   10.0.0.25
# 2) Reverse proxy (on the always-on box):
#      http://ai.lan:80   ->   http://localhost:3000

Step 5 — Wake It With a Tap (so the GPU can sleep)

A gaming PC left on 24/7 burns real power. Instead, let it sleep, and wake it on demand with a "magic packet" — a tiny network message that tells a sleeping machine to boot. This is Wake-on-LAN (WoL).

Enable it (one-time)

# In the GPU computer's BIOS:  turn ON "Wake on LAN" / "Power on by PCI-E";  turn OFF "ErP".
# In Windows:  disable "Fast Startup";  in the network card's settings, enable
#              "Wake on Magic Packet".   (Sleep state S3 wakes fastest & most reliably.)

Wake it from anywhere on the network

Any always-on Linux box (your NAS or a Pi) can send the packet with one command:

wakeonlan c8:7f:54:00:00:00     # <-- the GPU computer's network (MAC) address

Make it one tap: a phone shortcut can SSH into the always-on box, run that wakeonlan command, wait ~25 seconds, then open http://ai.lan. Now "summon the AI" is a single button — or a voice command. The GPU sleeps until you need it, wakes in seconds, and drops you straight into the chat.

The One Gotcha That Stumps Everyone — "Tools"

If you pick a model and it either errors with "does not support tools" or spits out weird {"name": ...} code instead of answering, here's the cause: Open WebUI tries to hand every model a set of "tools" (functions it can call). Some models can't use tools and reject the whole message; others try to call the tools instead of just talking.

The fix is not in the chat settings. It's a per-model switch:

# In Open WebUI:  Settings > Models > [your model] > Capabilities
#   -> UNCHECK "Builtin Tools"

Optional Polish

• Personas: give each model a "system prompt" (Settings > Models > [model]) — e.g. a terse "lead with the code, no preamble" prompt for the coder, and a warm, direct one for the chat model.
• Context length: the num_ctx setting is the model's short-term memory. The default is small; raising it (e.g. 16384) lets it hold much longer conversations. Bigger uses more VRAM.
• Long-context speed: if a big context spills to the CPU and slows to a crawl, two env vars on the GPU box keep it on the card — OLLAMA_KV_CACHE_TYPE=q8_0 and OLLAMA_FLASH_ATTENTION=1. (On one 16GB card at 32K context that cut the spill from ~23% down to ~8%.)
• Multiple users: in Admin settings, enable sign-ups so family can make their own accounts — each gets private chat history, all sharing the one GPU.

Result

• ✅ A capable AI running entirely on your own hardware
• ✅ Private — nothing you type ever leaves your network
• ✅ Free per use — no subscription, no per-message cost
• ✅ Reachable from any device at a clean name like http://ai.lan
• ✅ One-tap "summon" — the GPU sleeps to save power, wakes on demand
• ✅ Two models on tap: one for code, one for conversation
• ✅ Total ongoing cost: $0 (hardware you already own)