Menu

Tutorial

Score your agent against Claude

Point Kno at the Anthropic API: configure the key, run a baseline and a valuation, and read real numbers instead of the free fake agent's zeros.

This tutorial follows the repository recipe Score your agent against Claude. You will take the loop you saw in Getting started and run it against the Anthropic API.

What you will learn

  • How Kno reads API keys (environment only, never a flag, never a file)
  • How to budget a real run before it makes a single call
  • How the output changes when the agent is real

1. Export the key

export ANTHROPIC_API_KEY=sk-ant-...

2. Baseline with a budget

kno baseline --evals cases.jsonl --agent anthropic:claude-sonnet-5 \
  --max-output-tokens 1024 --max-cost-usd 2.00 --yes

--max-output-tokens is required for the Anthropic adapter. The budget guard prices each case from its own table, so the cap binds before the first call. --yes consents to the printed estimate.

3. Value the pool

kno value --evals cases.jsonl --pool pool.jsonl \
  --baseline-run-id <run id> --agent anthropic:claude-sonnet-5 \
  --max-output-tokens 1024 --max-cost-usd 5.00 --yes

This time the deltas are real: assets that help show positive deltas with intervals; assets that do nothing show underpowered blanks or near-zero intervals; harmful assets show up as regressions in the controls.

4. Read the numbers

Before you quote anything, read What the numbers mean. The holdout number is the one you may put in a slide.

Full recipe with troubleshooting: the Anthropic cookbook entry.

← All docs