Fixture Ownership#

evals/fixtures/ is canonical eval source, not generic sample data. Keep fixture groups only when an owning eval or validator names the behavior they protect. When adding, moving, or deleting a fixture directory, update this inventory and run npm run validate:source -- and npm run fixture:retirement-audit --.

Ownership Inventory#

Fixture directory	Classification	Owners	Cleanup rule
`evals/fixtures/ci-compute`	immutable reusable-workflow expansion snapshots for the participating CI compute budget	`scripts/ci/compute-policy.mjs`, `evals/ci/compute-policy.test.mjs`, `evals/static/test_ci_compute_policy.sh`	Keep while these immutable shared-workflow pins participate in the two-slot policy; replace with reviewed snapshots when pins change.
`evals/fixtures/assignment-provider`	AssignmentProvider local-file and GitHub render/status fixtures (#290); hostile-effective-state.json is the #291 ensure-session ownership guard's AC9 sanitization fixture	`evals/integration/test_assignment_provider_local_file.sh`, `evals/integration/test_assignment_provider_github.sh`, `evals/integration/test_pull_work_assignment_join.sh`, `evals/integration/test_ensure_session_ownership_guard.sh`	Keep while the assignment provider contract's claim/release/supersede/status/list operations and the assignment ⋈ liveness join are tested against local-file and GitHub fixture inputs.
`evals/fixtures/backlog-provider-settings`	settings precedence fixtures	`evals/integration/test_effective_backlog_settings.sh`	Keep while backlog provider settings resolution supports global defaults and project overrides.
`evals/fixtures/builder-kit-workflow-state`	Builder Kit workflow-state fixtures	`evals/static/test_workflow_skills.sh`	Keep while Builder Kit state contract and resume behavior are documented in workflow skill contracts.
`evals/fixtures/console-declared-projection`	delivery/DECLARED accepted-gap projection fixtures (#1267): mixed structured gaps[] / legacy / invalid entries proving the no-prose-scraping fold and its disclosed legacy count	`evals/integration/test_console_declared_projection.sh`	Keep while `console-declared-projection` projects only structured `gaps[]` entries and discloses legacy/invalid entry counts in its summary.
`evals/fixtures/delivery-bundle-corpus`	Session trust bundles once committed under delivery/, preserved as a status-function re-derivation corpus (#1422)	`src/cli/surface-status-function.test.mjs`, `evals/integration/test_trust_reconcile_negatives.sh`	Keep while historical bundles must keep re-deriving to their recorded status across Surface status-function changes.
`evals/fixtures/console-learning-projection`	console learning projection fixtures	`evals/integration/test_console_learning_projection.sh`	Keep while learning projection supports correction and open-route examples.
`evals/fixtures/console-process-projection`	console process-state projection fixtures (issue #778): blocked (handoff blockers), needs_decision (next_action.summary), a required-pending critique (review_pending), and a plain in_progress/running session	`evals/integration/test_console_process_projection.sh`	Keep while the workflow-state -> Console interactive-session process state mapping (needs_input/review_pending/blocked + blockedReason) is enforced.
`evals/fixtures/change-provider-settings`	ChangeProvider effective-settings fixtures for configured, malformed, and hostile secret-bearing inputs	`src/cli/effective-change-provider-settings.test.mjs`, `evals/integration/test_publish_change_operation.sh`	Keep while effective ChangeProvider settings distinguish configured, absent, malformed, and secret-bearing inputs without fallback or secret disclosure.
`evals/fixtures/codex-legacy-agents`	exact historical Flow Agents-generated Codex AGENTS.md migration fixtures (#563)	`evals/integration/test_install_merge.sh`	Keep while the Codex home installer recognizes audited historical payload hashes, refuses before mutation, and provides explicit operator remediation without fuzzy ownership matching.
`evals/fixtures/economics`	per-run kit-economics record fixtures (#349): a transcript with .message.usage blocks, state.json/acceptance.json/critique.json join sources, a session.usage event, and the golden expected kontour.console.economics record. Its `run-binding/` subdirectory (#922/#925 phase A, extended by the r1 review fix round) carries seven hand-built, `schemas/flow-run.schema.json`-validated Flow run stores -- `routeback-completed` (also carries a real-shaped `gate_outcome_history` mirroring production run `kontourai-flow-agents-1206`, exercising the PRIMARY gate-fire counting path), `active-mid-flight`, `canceled-with-pause` (lifecycle pause/resume), `paused-mid-execute` (mirrors the real, live `kontourai-flow-agents-944` production run: ~99.995% pause inside one phase), `zero-duration-boundary` (two transitions sharing one timestamp), `unknown-status` (a `status` value outside Flow's own enum), and `multi-cursor-active` (a non-empty `multi_cursor` ledger) -- each with a hand-computed `expected.json`, plus a synthetic transcript (incl. one `isSidechain:true` line) + phase-window JSON for the token-slicer	`evals/integration/test_economics_record.sh`, `evals/integration/test_economics_run_binding.sh`	Keep while the current `kontour.console.economics` v0.2 contract and historical v0.1 compatibility (docs/specs/economics-record-contract.md) are enforced; the golden proves cost-from-transcript, defects-from-critique, canonical Flow verdict authority, and the R7 co-required guard. `run-binding/` fixtures stay while `scripts/telemetry/flow-run-economics.mjs` and `scripts/telemetry/economics-enrich-tokens.mjs` (docs/specs/economics-record-contract.md "flow_run_record mode") derive phases/iterations/defects/terminal_status/human_wait_s from the canonical Flow run store and token attribution from a transcript, both against exact hand-computed values, incl. the pause-exclusion, null-cost-propagation, unrecognized-status-refusal, and multi_cursor-refusal fixes proven by RED/GREEN evidence in the r1 review fix round.
`evals/fixtures/flow-kit-repository`	Flow Kit repository contract fixtures	`evals/integration/test_published_codex_install.sh`, `evals/integration/test_flow_kit_repository.sh`, `evals/integration/test_local_flow_kit_install.sh`, `evals/integration/test_runtime_adapter_activation.sh`, `evals/integration/test_activate_npx_context.sh`, `evals/integration/test_flow_kit_install_git.sh`, `evals/static/test_validate_source_kit_asset_scope.sh`, `evals/static/test_workflow_skills.sh`	Keep valid and invalid cases paired with the Flow Kit repository contract.
`evals/fixtures/kit-conformance-levels`	K-level conformance and consumer-target derivation fixtures	`evals/integration/test_kit_conformance_levels.sh`	Keep while K-level derivation, degradation invariant, and consumer-target badge rules are tested.
`evals/fixtures/kit-gate-policies`	non-Builder kit fixture whose flow declares Work Item selection, a route-back map with no default route, and a tests-evidence gate under its own gate ids and claim types	`evals/integration/test_kit_gate_policies.sh`, `src/cli/builder-flow-runtime.test.mjs`, `src/cli/routeback-disclosure.test.mjs`	Keep while the engine derives the initial-visit selection skew, the reason-less failure refusal, and verified-tests trust from what a gate declares rather than from Builder's gate ids and claim types. The integration owner starts and loads the declared definition through public Flow, then checks selection evidence against the persisted visit through Flow Agents; it does not qualify full workflow admission.
`evals/fixtures/kit-observability`	host-neutral Kit observability descriptor and third-party compatibility fixtures (#911)	`evals/static/test_kit_observability_contract.sh`	Keep while Builder, Knowledge, absent/future-version, and third-party contribution behavior is validated without a host installation.
`evals/fixtures/hook-influence`	hook influence behavioral cases	`evals/integration/test_hook_influence_cases.sh`, `evals/static/test_workflow_skills.sh`, `scripts/validate-hook-influence-cases.js`	Keep while hook influence cases define agent guidance behavior.
`evals/fixtures/hachure-conformance`	unmodified hachure 0.16.0 npm tarball, integrity-pinned, whose status function 3 conformance vectors the writer suite runs through Surface (#1422)	`evals/integration/test_workflow_sidecar_writer.sh`	Keep while the writer derives under status function "3" and no hachure release with these vectors is reachable through a dependency. Replace, never edit: the suite refuses the tarball unless its sha512 matches the pinned registry integrity.
`evals/fixtures/learning-review-proposals`	learning-review kit/gate tuning proposal fixtures (#352): pattern-present (engineered cost-inflation + gate false-block-rate pattern, with sessions/ trust.bundle + gate-review.inquiries.json joins and a hand-computed expected-aggregates.json), balanced (proportional cost/findings movement -> zero proposals), under-threshold (below LR_MIN_WINDOW_SAMPLE), repeat-window (idempotency), and effect-follow-up (later-window effect-fill pass for a ratified proposal)	`evals/integration/test_learning_review_proposals.sh`	Keep while `scripts/telemetry/learning-review-proposals.sh`/`learning-review-decide.sh` (docs/specs/learning-review-proposals-contract.md) are enforced: hand-computed aggregates, evidence-cited proposals, zero-mutation, idempotency, and the ratify -> effect-fill trail.
`evals/fixtures/narrative-sources`	narrative source snapshot, offline resolution, integrity, completeness, fail-closed redaction, grounded runtime projection, grounded execution narrative, and trust-isolation fixtures (#613, #617, #618, #619)	`evals/integration/test_narrative_source_contract.sh`, `evals/integration/test_narrative_runtime_projection.sh`, `evals/integration/test_narrative_grounded_envelope.sh`, `evals/integration/test_narrative_redaction_failclosed.sh`, `evals/integration/test_narrative_trust_isolation.sh`	Keep while the narrative source manifest, content-addressed offline resolver, integrity verification, grounded runtime projection, grounded execution narrative composition, capture-completeness disclosure, lifecycle locality, fail-closed redaction, and narrative trust-isolation contracts are supported.
`evals/fixtures/narrative-grounding-validator`	narrative grounding validator adversarial fixtures (#623)	`evals/integration/test_narrative_grounding_validator.sh`	Keep while deterministic citation integrity, material-event coverage, epistemic-label integrity, and fail-closed narrative publication are enforced.
`evals/fixtures/narrative-prose-renderer`	model-assisted prose renderer adversarial fixtures (#614): provenance-subset (unsupported-summary), no-op-loop lack-of-progress wording, and prompt-injection fixtures reused from evals/fixtures/narrative-grounding-validator	`evals/integration/test_narrative_prose_renderer.sh`	Keep while the fail-closed prose renderer (D3 provenance-subset citation check, provider gating, economics-always, zero-prose-artifact-on-failure) is enforced.
`evals/fixtures/narrative-evals`	grounded narrative faithfulness eval corpus + result-schema package (#612): ten R1 case-class fixtures with frozen-manifest answer keys plus five R3 corruption fixtures whose named eval-check anchors are the scorer-teeth mutation targets	`evals/integration/test_narrative_evals.sh`	Keep while the versioned adversarial corpus, its frozen-manifest answer keys, the schema-valid narrative-eval-result output, the declared cross-runtime capability parity, and the R3 scorer-teeth mutation battery are enforced.
`evals/fixtures/pr-body-file-refs`	verbatim GitHub-stored pull request bodies for the #1375 PR-body file-reference contract, each paired with a hand-checked expected-paths answer key: #1366's fabricated body (the incident — it names `src/cli/trust-bundle-verifying-actor.test.mjs`, which lived on sibling branch #1368 and appeared 0 times in #1366's diff), #1366's corrected body, and three unrelated merged bodies (#1041, #1101, #1354) that must not red	`evals/static/test_pr_body_file_refs.sh`	Keep while `scripts/ci/validate-pr-body-file-refs.mjs` gates PR bodies. These are historical records, not synthesized inputs: regenerating them would destroy the only proof the check reds on the real failure rather than on a happy-path mock.
`evals/fixtures/pull-work-provider`	work item provider normalization fixtures	`evals/integration/test_pull_work_provider.sh`	Keep while provider normalization preserves blockers, artifact refs, board membership, and freshness metadata.
`evals/fixtures/pull-work-wip-shepherding`	WIP shepherding state fixtures	`evals/static/test_workflow_skills.sh`	Keep while pull-work documents personal versus global WIP behavior.
`evals/fixtures/reconcile-preflight`	#356 reconcile-preflight shape fixtures not already covered by trust-reconcile-exploits (un-superseded disputed critique, standalone disputed session-local claim)	`evals/integration/test_reconcile_preflight.sh`	Keep while the local `reconcile-preflight` subcommand (#356) is proven against the two shapes trust-reconcile-exploits does not already fixture (un-superseded disputed critique, standalone disputed session-local claim); the other four shapes reuse trust-reconcile-exploits/trust-reconcile-mixed-bundle directly rather than forking near-duplicates.
`evals/fixtures/reviewed-grounding-workflow`	deterministic reviewed-grounding reference telemetry fixture (#715)	`evals/integration/test_reviewed_grounding_workflow.sh`	Keep while the pinned public-product composition proves offline extraction, semantic review, portable evidence, action refusal, typed failures, and optional live telemetry.
`evals/fixtures/surface-trust`	Surface trust evidence fixtures	`evals/integration/test_workflow_sidecar_writer.sh`	Keep while sidecar writer maps Surface trust evidence into workflow records.
`evals/fixtures/telemetry`	hermetic Stop-hook usage transcript fixture (#defect-2026-07 telemetry-usage-cost-extraction): a multi-model JSONL transcript with .message.model + .message.usage token blocks proving real token/cost/model extraction end-to-end; also reused by the #568 slice 1 per-tool-call usage-enrichment eval (test_telemetry_tool_usage.sh), which adds usage-transcript-oversized-prefix.jsonl to prove the bounded tail-read	`evals/integration/test_usage_cost.sh`, `evals/integration/test_telemetry_usage_pipeline.sh`, `evals/integration/test_telemetry_tool_usage.sh`, `evals/integration/test_telemetry_tool_outcome.sh`, `evals/integration/test_installed_runtime_correlation.sh`	Keep while telemetry.sh's Stop-hook usage pipeline (usage_parse_transcript / usage_model_from_transcript_usage / usage_last_turn_usage), installed producer reconstruction, and idempotent privacy-safe retrospective observation compilation are proven against hermetic multi-model and oversized-prefix transcripts rather than relying on live runtime data.
`evals/fixtures/trust-reconcile-exploits`	WS8 trust-reconcile anti-gaming exploit fixtures (frozen negative regressions); also reused by the #356 local reconcile-preflight eval (same shapes, no forked copies)	`evals/integration/test_trust_reconcile_negatives.sh`, `evals/integration/test_reconcile_preflight.sh`	Keep while trust-reconcile.js enforces the WS8 iteration-2 soundness properties (no-label test_output, unwaived-assumed, status-misassertion, waiver-on-command); each fixture is a permanent negative regression.
`evals/fixtures/trust-reconcile-mixed-bundle`	WS8 trust-reconcile mixed-evidence end-to-end proof fixture; also reused by the #356 reconcile-preflight eval as its CLEAN-BUNDLE (AC4) case	`evals/integration/test_trust_reconcile_mixed_bundle.sh`, `evals/integration/test_reconcile_preflight.sh`	Keep while the trust-reconcile manifest/classification/waiver contract (ADR 0020) is enforced; proves a mixed test_output + session-local + waived bundle passes the CI anchor.
`evals/fixtures/trust-reconcile-ws3`	WS8 AC6 backward-compat fixture: real ws3-kit-dependencies-namespacing old-style bundle	`evals/integration/test_trust_reconcile_negatives.sh`	Keep while backward compatibility with pre-classification (all-test_output) bundles is asserted; proves an old-style bundle still FAILS the same way (no silent pass).
`evals/fixtures/usage-feedback`	usage feedback import/outcome fixtures	`evals/integration/test_usage_feedback_import.sh`, `evals/integration/test_usage_feedback_outcomes.sh`, `evals/integration/test_usage_feedback_report.sh`	Keep while usage feedback import, outcome, and report flows accept JSONL fixture input.

Cleanup Policy#

Do not delete a fixture group just because a filename has no obvious direct reference. Many fixture directories are consumed as directories by integration tests. A fixture deletion is safe only when:

The owning eval or validator no longer needs the behavior.
This inventory is updated in the same change.
npm run validate:source -- and the owning eval pass.
Any contract or docs that described the fixture behavior are updated.

Generated eval output belongs under ignored result directories, not under evals/fixtures/.

npm run fixture:retirement-audit -- is a read-only retirement pass. It reports fixture groups as kept when they have documented owners and live eval/script references. A nonzero retire-candidate count is not permission to delete by itself; it is a prompt to update or remove the owning eval and fixture together.

Packaged CLI fixture copy failures#

src/cli/fixture-kit-package.mjs copies the compiled CLI into an owned package fixture. A failed copy rethrows the original exception with fixtureCopy metadata serialized as JSON so test reporters retain the values: process/Node identity and before/after filesystem metadata for the source build, source src, fixture root, destination build, and destination src. It does not retry, read file contents, or turn setup failure into a contract-test result. Node's native copy path can name the containing destination directory for a source-file failure, so the exception path alone is not proof that cleanup removed the destination. Retain the complete failing test output; missing historical metadata leaves the historical cause unverified.