✦ AI Explained
Emerging area 14 min lesson

Trace the whole decision

A trace should let an operator follow one request through context selection, model calls, tools, policy decisions and final outcome without exposing secrets or private content.

One request, one story

An OpenTelemetry trace represents work triggered by a request. Spans inside it show operations and timing. For an agentic service, that can include context retrieval, a model call, schema validation, policy evaluation, tool execution, approval and the final response.

request
 ├─ retrieve approved context
 ├─ model proposal
 ├─ validate proposal
 ├─ policy decision
 └─ tool call or human approval

Across a queue or delegated task, propagate W3C Trace Context: the traceparent header carries the trace and parent-span identity, while tracestate can carry vendor-specific trace state. Create a new child span when the work is a continuation of the same causal path. When one operation is related to several independent traces, use span links instead of pretending it has only one parent.

A correlation ID is still useful as a business or incident key in logs and records. It is not a replacement for trace propagation: it cannot reconstruct timing, parentage or the distributed path by itself.

Telemetry is not a data exfiltration channel

Do not copy full prompts, secrets, tokens, customer records or raw model output into traces by default. Capture bounded metadata: provider, model identifier, duration, result class, token count when appropriate, tool name, policy outcome and stable resource identifiers. Decide retention and access before data reaches your telemetry backend.

Logs explain discrete events. Metrics show patterns. Traces connect a request path. You usually need all three.

Try it

Run docker compose up --build, exercise the Workbench once, then open Grafana at http://localhost:3000. The Workbench exports the workbench.investigation, workbench.context-selection and workbench.policy spans through the OpenTelemetry Collector to Tempo. Draw the same path from click to final route, then circle the fields that would be unsafe to put into telemetry unchanged.

Keep this with you

If a request cannot be followed safely, it cannot be operated confidently.

Check the source

These are the primary sources behind the moving parts of this lesson. Check them again when you make an implementation decision.

✏ Suggest an edit on GitHub

Trace the whole decision

Question 1: What should a request trace connect?
Question 2: What is a safe default for prompt content in telemetry?