Harness-capability matrix#
The signals a feature can rely on depend on which runtime/harness the kit is hooked into. State the coverage; never assume it, never fabricate the gap.
Generated by
npm run capability-matrixfromsrc/lib/capability-declarations.ts— the single source of truth. Do NOT edit this file by hand; change the declarations and regenerate. CI fails on drift (npm run capability-matrix -- --check).
This is a cross-cutting contract: any feature whose data comes from telemetry (economics, delegation efficiency, liveness, learning) must declare which signals it needs, which class each falls in, and how it degrades when the current runtime does not expose one. A missing signal is rendered as unavailable, distinct from a real zero — so a consumer never reads "harness-blind" as "measured 0."
The load-bearing distinction: orchestrator-observable vs sub-agent-internal#
Not all "we can't see inside the sub-agent" gaps are equal. Two classes:
- Orchestrator-observable — what our hooks see the main agent (the orchestrator) do. The orchestrator knows the prompt it dispatched, how many times it re-dispatched/re-prompted the same sub-agent, its own iteration/route-back count, and when it superseded or corrected a result. Our telemetry observes the orchestrator directly, so these hold on any runtime that runs our hooks — no sub-agent introspection required. This is where per-delegation outcome comes from.
- Sub-agent-internal — signals that live inside the sub-agent's own turn: its token usage, its internal reasoning, a verdict it computed but never surfaced to the orchestrator. Capturing these requires the harness to expose per-sub-agent instrumentation. Most harnesses do not today, so features that need them (e.g. per-delegation cost) must declare the signal unavailable and fall back to a coarser, honest proxy.
The practical upshot for #415: we can attribute outcome (accepted / rework / diverged / failed) from orchestrator-observable behavior even though we cannot attribute cost per delegation — because tokens are sub-agent-internal and outcome is not.
The signals block#
Every kontour.console.economics record carries a signals object declaring what the emitting runtime
exposed:
| Field | Class | Meaning |
|---|---|---|
runtime |
— | the runtime that produced the record (claude-code, kiro-cli, codex, …) |
per_delegation_tokens |
sub-agent-internal | true iff the runtime isolates per-sub-agent token usage. false on every runtime today → per-delegation cost is unavailable; the console attributes cost at (role, model) granularity via cost.by_model instead. Derived from this matrix's per_delegation_tokens capability declaration, not a hardcoded literal. |
per_delegation_outcome |
orchestrator-observable | coverage of the outcome signal on this run: full (every delegation resolved to a real outcome), partial (some), none (delegations exist but none had a verdict/escalation/re-dispatch), n/a (no delegations observed). |
Consumers (console panels, learning-review) MUST read signals before rendering a metric: if the
signal a metric needs is unavailable, show "not measurable on this harness," not a misleading number.
Per-runtime capability coverage (generated, honest snapshot)#
✓ exposed · partial best-effort/orchestrator-derived · ✗ not exposed. This table is generated
from src/lib/capability-declarations.ts; update the declarations there as runtimes add
instrumentation — it is the single source of truth the signals values are stamped from. The
kiro-cli runtime value normalizes to the canonical kiro column before lookup.
| Capability | claude-code |
codex |
kiro |
opencode |
pi |
codex-local |
strands-local |
|---|---|---|---|---|---|---|---|
turn_id |
✓ | ✓ | ✓ | partial | ✓ | ✗ | ✗ |
transcript_path |
✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ |
intent_annotation |
✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
per_delegation_trace_context |
✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
per_delegation_tokens |
✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
terminal_verdict |
partial | ✗ | partial | ✗ | ✗ | ✗ | ✗ |
Declared notes and reasons#
Every partial / ✗ cell above, with its declared justification:
turn_id·opencode(partial): session.start telemetry is unavailable in opencode run (non-interactive) mode; tool events still carry a turn context.turn_id·codex-local(unsupported): codex-local is a kit-activation adapter, not an agent-runtime telemetry source; it emits no per-turn agent-runtime telemetry.turn_id·strands-local(unsupported): strands-local is a kit-activation adapter, not an agent-runtime telemetry source; it emits no per-turn agent-runtime telemetry.transcript_path·opencode(unsupported): opencode does not expose a per-session transcript path to hooks.transcript_path·pi(unsupported): pi does not expose a per-session transcript path to hooks.transcript_path·codex-local(unsupported): codex-local is a kit-activation adapter and does not expose a per-session transcript path.transcript_path·strands-local(unsupported): strands-local is a kit-activation adapter and does not expose a per-session transcript path.intent_annotation·claude-code(unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).intent_annotation·codex(unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).intent_annotation·kiro(unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).intent_annotation·opencode(unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).intent_annotation·pi(unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).intent_annotation·codex-local(unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).intent_annotation·strands-local(unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).per_delegation_trace_context·claude-code(unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).per_delegation_trace_context·codex(unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).per_delegation_trace_context·kiro(unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).per_delegation_trace_context·opencode(unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).per_delegation_trace_context·pi(unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).per_delegation_trace_context·codex-local(unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).per_delegation_trace_context·strands-local(unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).per_delegation_tokens·claude-code(unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.per_delegation_tokens·codex(unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.per_delegation_tokens·kiro(unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.per_delegation_tokens·opencode(unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.per_delegation_tokens·pi(unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.per_delegation_tokens·codex-local(unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.per_delegation_tokens·strands-local(unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.terminal_verdict·claude-code(partial): Captured only when the workflow records an evidence/verdict event for the sub-agent — not guaranteed every run.terminal_verdict·codex(unsupported): Codex does not surface a per-sub-agent terminal verdict to the orchestrator.terminal_verdict·kiro(partial): Captured only when the workflow records an evidence/verdict event for the sub-agent — not guaranteed every run.terminal_verdict·opencode(unsupported): opencode does not surface per-sub-agent terminal verdicts.terminal_verdict·pi(unsupported): pi has no named-subagent registry, so per-sub-agent verdicts are not observable.terminal_verdict·codex-local(unsupported): codex-local is a kit-activation adapter and does not surface per-sub-agent terminal verdicts.terminal_verdict·strands-local(unsupported): strands-local is a kit-activation adapter and does not surface per-sub-agent terminal verdicts.
The doctrine (applies to any telemetry-dependent feature)#
- Classify each signal as run-level, orchestrator-observable, or sub-agent-internal. Prefer orchestrator-observable derivations — they are the most portable across harnesses.
- Declare coverage in-band (a
signals-style block) so a consumer can distinguish real-zero from harness-blind at read time. - Degrade to an explicit
unavailable, never to a fabricated value. A cost you cannot attribute is not$0; an outcome you cannot observe is notaccepted. - Fall back to the coarsest honest proxy when the fine signal is missing (per-delegation cost →
(role, model)viaby_model), and label the proxy as such. - Keep these declarations current. When a harness starts exposing a signal, flip the declaration in
src/lib/capability-declarations.tsand regenerate — thesignalsstamp follows and the feature "lights up" without a schema change.