Explanation

The mental model

How Kno thinks about data as an experimental variable, the one idea that makes every stage and number fall into place.

Traditional evals answer one question: did my agent get better? They vary the model, the prompt, or the tools, and watch a score.

Kno asks a different question: which data made it better?

Traditional eval

Prompt ─────┐
Model ──────┼──► Agent ───► Score
Tools ──────┘


Kno

                    ┌─ Document A ──► +12%
                    ├─ Example B ───►  +8%
Agent + Evals ──────┼─ Policy C ────►   0%
                    └─ Asset D ─────►  -3%


              impact × cost × destination

Data is the variable

A document, an example, a policy, a conversation, each is a candidate asset. Kno treats each one as a variable in a controlled experiment: measure the agent without it, inject it, measure again, compare against fresh controls, and report the delta with its uncertainty.

The agent, the evals, and the goal stay fixed. Only the data changes. That is what makes the result attributable.

The stages

Stage Question Status
Init How is this run set up? Shipped
Mine Where do the evals come from? Shipped
Baseline How good is the agent now? Shipped
Value Which assets improve it? Shipped
Select Which combination should I keep? Shipped
Validate Does the combination still work on untouched evals? Planned
Export Where should each asset go? Shipped
Report What does the whole run say? Shipped

The holdout is the discipline

A slice of your cases is sealed at baseline and nothing reads it until validate. Without that separation, selection could silently overfit your dev set, and every number after it would be inflated.

The full argument, including what is deliberately out of scope: DESIGN.md and the mental model page.

← All docs