Harness-capability matrix#

The signals a feature can rely on depend on which runtime/harness the kit is hooked into. State the coverage; never assume it, never fabricate the gap.

Generated by npm run capability-matrix from src/lib/capability-declarations.ts — the single source of truth. Do NOT edit this file by hand; change the declarations and regenerate. CI fails on drift (npm run capability-matrix -- --check).

This is a cross-cutting contract: any feature whose data comes from telemetry (economics, delegation efficiency, liveness, learning) must declare which signals it needs, which class each falls in, and how it degrades when the current runtime does not expose one. A missing signal is rendered as unavailable, distinct from a real zero — so a consumer never reads "harness-blind" as "measured 0."

The load-bearing distinction: orchestrator-observable vs sub-agent-internal#

Not all "we can't see inside the sub-agent" gaps are equal. Two classes:

  • Orchestrator-observable — what our hooks see the main agent (the orchestrator) do. The orchestrator knows the prompt it dispatched, how many times it re-dispatched/re-prompted the same sub-agent, its own iteration/route-back count, and when it superseded or corrected a result. Our telemetry observes the orchestrator directly, so these hold on any runtime that runs our hooks — no sub-agent introspection required. This is where per-delegation outcome comes from.
  • Sub-agent-internal — signals that live inside the sub-agent's own turn: its token usage, its internal reasoning, a verdict it computed but never surfaced to the orchestrator. Capturing these requires the harness to expose per-sub-agent instrumentation. Most harnesses do not today, so features that need them (e.g. per-delegation cost) must declare the signal unavailable and fall back to a coarser, honest proxy.

The practical upshot for #415: we can attribute outcome (accepted / rework / diverged / failed) from orchestrator-observable behavior even though we cannot attribute cost per delegation — because tokens are sub-agent-internal and outcome is not.

The signals block#

Every kontour.console.economics record carries a signals object declaring what the emitting runtime exposed:

Field Class Meaning
runtime — the runtime that produced the record (claude-code, kiro-cli, codex, …)
per_delegation_tokens sub-agent-internal true iff the runtime isolates per-sub-agent token usage. false on every runtime today → per-delegation cost is unavailable; the console attributes cost at (role, model) granularity via cost.by_model instead. Derived from this matrix's per_delegation_tokens capability declaration, not a hardcoded literal.
per_delegation_outcome orchestrator-observable coverage of the outcome signal on this run: full (every delegation resolved to a real outcome), partial (some), none (delegations exist but none had a verdict/escalation/re-dispatch), n/a (no delegations observed).

Consumers (console panels, learning-review) MUST read signals before rendering a metric: if the signal a metric needs is unavailable, show "not measurable on this harness," not a misleading number.

Per-runtime capability coverage (generated, honest snapshot)#

✓ exposed · partial best-effort/orchestrator-derived · ✗ not exposed. This table is generated from src/lib/capability-declarations.ts; update the declarations there as runtimes add instrumentation — it is the single source of truth the signals values are stamped from. The kiro-cli runtime value normalizes to the canonical kiro column before lookup.

Capability claude-code codex kiro opencode pi codex-local strands-local
turn_id ✓ ✓ ✓ partial ✓ ✗ ✗
transcript_path ✓ ✓ ✓ ✗ ✗ ✗ ✗
intent_annotation ✗ ✗ ✗ ✗ ✗ ✗ ✗
per_delegation_trace_context ✗ ✗ ✗ ✗ ✗ ✗ ✗
per_delegation_tokens ✗ ✗ ✗ ✗ ✗ ✗ ✗
terminal_verdict partial ✗ partial ✗ ✗ ✗ ✗

Declared notes and reasons#

Every partial / ✗ cell above, with its declared justification:

  • turn_id · opencode (partial): session.start telemetry is unavailable in opencode run (non-interactive) mode; tool events still carry a turn context.
  • turn_id · codex-local (unsupported): codex-local is a kit-activation adapter, not an agent-runtime telemetry source; it emits no per-turn agent-runtime telemetry.
  • turn_id · strands-local (unsupported): strands-local is a kit-activation adapter, not an agent-runtime telemetry source; it emits no per-turn agent-runtime telemetry.
  • transcript_path · opencode (unsupported): opencode does not expose a per-session transcript path to hooks.
  • transcript_path · pi (unsupported): pi does not expose a per-session transcript path to hooks.
  • transcript_path · codex-local (unsupported): codex-local is a kit-activation adapter and does not expose a per-session transcript path.
  • transcript_path · strands-local (unsupported): strands-local is a kit-activation adapter and does not expose a per-session transcript path.
  • intent_annotation · claude-code (unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).
  • intent_annotation · codex (unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).
  • intent_annotation · kiro (unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).
  • intent_annotation · opencode (unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).
  • intent_annotation · pi (unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).
  • intent_annotation · codex-local (unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).
  • intent_annotation · strands-local (unsupported): Intent-annotation capture is not implemented on any runtime yet (#622).
  • per_delegation_trace_context · claude-code (unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).
  • per_delegation_trace_context · codex (unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).
  • per_delegation_trace_context · kiro (unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).
  • per_delegation_trace_context · opencode (unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).
  • per_delegation_trace_context · pi (unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).
  • per_delegation_trace_context · codex-local (unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).
  • per_delegation_trace_context · strands-local (unsupported): Per-delegation trace-context propagation is not implemented on any runtime yet (#425).
  • per_delegation_tokens · claude-code (unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.
  • per_delegation_tokens · codex (unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.
  • per_delegation_tokens · kiro (unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.
  • per_delegation_tokens · opencode (unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.
  • per_delegation_tokens · pi (unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.
  • per_delegation_tokens · codex-local (unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.
  • per_delegation_tokens · strands-local (unsupported): No runtime isolates per-sub-agent token usage; per-delegation cost is attributed at (role, model) granularity instead.
  • terminal_verdict · claude-code (partial): Captured only when the workflow records an evidence/verdict event for the sub-agent — not guaranteed every run.
  • terminal_verdict · codex (unsupported): Codex does not surface a per-sub-agent terminal verdict to the orchestrator.
  • terminal_verdict · kiro (partial): Captured only when the workflow records an evidence/verdict event for the sub-agent — not guaranteed every run.
  • terminal_verdict · opencode (unsupported): opencode does not surface per-sub-agent terminal verdicts.
  • terminal_verdict · pi (unsupported): pi has no named-subagent registry, so per-sub-agent verdicts are not observable.
  • terminal_verdict · codex-local (unsupported): codex-local is a kit-activation adapter and does not surface per-sub-agent terminal verdicts.
  • terminal_verdict · strands-local (unsupported): strands-local is a kit-activation adapter and does not surface per-sub-agent terminal verdicts.

The doctrine (applies to any telemetry-dependent feature)#

  1. Classify each signal as run-level, orchestrator-observable, or sub-agent-internal. Prefer orchestrator-observable derivations — they are the most portable across harnesses.
  2. Declare coverage in-band (a signals-style block) so a consumer can distinguish real-zero from harness-blind at read time.
  3. Degrade to an explicit unavailable, never to a fabricated value. A cost you cannot attribute is not $0; an outcome you cannot observe is not accepted.
  4. Fall back to the coarsest honest proxy when the fine signal is missing (per-delegation cost → (role, model) via by_model), and label the proxy as such.
  5. Keep these declarations current. When a harness starts exposing a signal, flip the declaration in src/lib/capability-declarations.ts and regenerate — the signals stamp follows and the feature "lights up" without a schema change.

Source: docs/specs/harness-capability-matrix.md