Evidence before explanation

A plausible story is not a diagnosis

“The deployment caused the errors” may be true. It may also be a very tidy story told by a tired human at 2am.

The Workbench starts by separating what it observed from what it infers. An alert saying that checkout errors rose is evidence. A deployment timestamp is evidence. “The new release broke checkout” is a hypothesis until we test it.

Give every fact a home

For this course, each investigation record has four small buckets:

This feels almost too simple. That is the point. It stops a system from turning its own generated text into a fact.

Why this matters before models

Later, a model will help us summarise logs and suggest useful checks. It must not get to quietly invent a source for itself.

The harness will therefore keep evidence separate from model output. The model can propose a hypothesis. The system records the proposal as a proposal. Only a successful check adds new evidence.

model/provider  → proposes a hypothesis
harness         → labels the proposal correctly
tool/runtime    → performs a permitted check
state/events    → record the tool result as evidence

Try it

Open the Workbench and choose Checkout regression. First look at What the system observed: the payment-provider health snapshot reports 0.0% errors. Then find What we still need to test and record “The payment provider may be failing” as disproved.

The hypothesis remains in the panel and in the event timeline as a dead end, with the evidence that ruled it out. Notice what does not happen: “The deployment may have changed checkout validation” remains unproven. We have a reasonable next check, not a conclusion.

Keep this with you

When the answer matters, ask: “What did we actually observe, and where did it come from?” That is useful with a spreadsheet, a human on-call engineer, or a model that writes beautifully.