Point Kno at your own provider

Run the valuation loop against any OpenAI-compatible endpoint, a hosted API, or a local runtime like vLLM, Ollama, or llama.cpp.

This tutorial follows the repository recipe Point Kno at your own provider.

What you will learn

The openai: agent reference and --base-url
How local runtimes need no key at all
How to switch the same evals and pool between providers

Hosted OpenAI-compatible endpoint

export OPENAI_API_KEY=sk-...
kno baseline --evals cases.jsonl --agent openai:gpt-4.1 \
  --base-url https://your-endpoint.example/v1 \
  --max-cost-usd 2.00 --yes

Local runtime, no key

kno baseline --evals cases.jsonl --agent openai:llama-3.1-8b \
  --base-url http://localhost:11434/v1 \
  --max-cost-usd 2.00 --yes

vLLM, Ollama, and llama.cpp are all OpenAI-compatible. Nothing leaves your machine, and the local runtime costs whatever your hardware costs.

The same pool, any provider

The evals and pool do not change between providers, only --agent and --base-url do. That is what makes a provider comparison honest: the data is held fixed while the model changes.

kno value --evals cases.jsonl --pool pool.jsonl \
  --baseline-run-id <run id> --agent openai:gpt-4.1 \
  --base-url https://your-endpoint.example/v1 --max-cost-usd 5.00 --yes

Full recipe with troubleshooting: the cookbook entry.