Local Models

Crush can auto-discovers models from local providers. Add a custom provider with type set to llamacpp, omlx, lmstudio, litellm, or ollama and leave out the models list. Crush will populate the model list automatically.

# Piece of cake.
provider add ollama \
  --name Ollama \
  --type ollama \
  --base-url "http://localhost:11434/v1/"

For llama.cpp (llama-server), point at the server's base URL:

provider add llamacpp \
  --name "llama.cpp" \
  --type llamacpp \
  --base-url "http://localhost:2222"

Manual Model Configuration

You can still list models explicitly. User-defined models always take precedence over discovered ones, and any fields you set won't be overwritten by auto-discovery. Auto discovery will run if the model list is empty for any openai-compat provider or if you pass "discover_models": true it will merge the found models with your hand configured ones.

# crushrc
provider add ollama \
  --name Ollama \
  --type ollama \
  --base-url "http://localhost:11434/v1/" \
  --discover-models true

model add ollama/qwen3:30b \
  --name "Qwen 3 30B" \
  --context-window 256000 \
  --default-max-tokens 20000

The --discover-models true flag merges discovered models with the one above; your explicit model fields win on conflicts.