Skip to content
Wonderthrough Request early access

Local LLM Setup

If you want Wonderthrough’s AI features to run entirely on your own machine — with nothing sent to any cloud provider — you can connect a local model through Ollama. This is the Local-only privacy preset, and it’s available in the desktop app. For how this fits among the privacy choices, see Privacy & Data.

What “local-only” means

Section titled “What “local-only” means”

On the Local-only preset, AI calls go to models running on your computer instead of to Anthropic, OpenAI, Google, or Wonderthrough’s servers. Your search index stays on the device, and nothing about your queries is logged. Everything stays on your machine.

It’s desktop-only by design: a browser can’t safely run or hold local models, so this lives in the desktop app. It works best on Apple Silicon or a machine with a discrete GPU, since the models run on your own hardware.

Setup steps

Section titled “Setup steps”
  1. Install Ollama. Download it from ollama.ai, install it, and let it run. It serves models locally on your machine.
  2. Pull an embedding model. This is what powers World Search. In a terminal, run: 
    ollama pull nomic-embed-text
  3. Pull a generation model of your choice. This is what answers and analyzes. Any chat-capable model Ollama supports will work, for example: 
    ollama pull llama3.2
  4. Choose the Local-only preset. In Wonderthrough, go to Settings → Privacy & Data and select Local-only.

That’s it. With Ollama running and a model pulled, Wonderthrough routes its AI work to your local models.

What to expect

Section titled “What to expect”
  • World Search builds its index locally the first time you use it. Because the local index is built on this device, it doesn’t sync to the cloud — opening the same project on another computer builds a fresh index there.
  • Speed depends on your hardware. On a capable machine this is quick; on a CPU-only machine, indexing a large manuscript can take a couple of minutes.
  • Ollama needs to be running. If it isn’t, Wonderthrough will let you know the local model is unavailable; start Ollama and it’ll reconnect.

How this compares to the other presets

Section titled “How this compares to the other presets”

Local-only is one of three privacy presets:

  • Standard uses Wonderthrough’s keys through our servers — the simplest setup, nothing to install.
  • Private AI uses your own provider keys (your OpenAI and Anthropic accounts) directly from the desktop app, so prompts skip our servers but still reach those providers.
  • Local-only contacts no cloud provider at all.

Choose Local-only when keeping everything on your own machine matters most and you’re comfortable installing and running Ollama.

Section titled “Related help”