Reference#

This page lists the public surface of @kontourai/dispatch: its package entrypoints, the exports of each, the plan and receipt types, and the dispatch CLI. For how the pieces work together, read Concepts.

Entrypoints#

The package also exports ./package.json and installs one binary, dispatch.

Core API#

dispatch#

dispatch(plan: ExecutionPlan, runtimes: RuntimeRegistry, options?: DispatchOptions): Promise<DispatchOutcome>

Runs one plan. It resolves to { result, receipt } when the outcome is succeeded and to { receipt } otherwise. It throws TypeError for an invalid plan or authorization, and AuthorizationPersistenceError when an authorization reservation cannot be recorded safely.

DispatchOptions:

dispatchBatch#

dispatchBatch(plans: readonly ExecutionPlan[], runtimes: RuntimeRegistry, options?: DispatchOptions): Promise<readonly DispatchOutcome[]>

Runs the plans as one provider-native physical batch and returns one outcome per plan, in input order. It throws TypeError for an empty list, and a Relay ModelInvocationError with code INVALID_REQUEST when the plans' first eligible candidates do not share one runtime, or that runtime cannot run a physical batch of this size. See physical batches.

createDispatchRuntime#

createDispatchRuntime(options: DispatchRuntimeOptions): ModelRuntime

Returns a Relay ModelRuntime whose invoke() runs dispatch() with the request it receives.

DispatchRuntimeOptions:

invokeBatch() accepts between 1 and maxBatchSize requests and runs dispatchBatch(). Every receipt is delivered. In fail-closed mode, if any delivery fails, the first ReceiptDeliveryError is thrown after all of them have been attempted; in best-effort mode the outcomes are returned normally.

When an invocation ends without a result, invoke() throws DispatchRuntimeError. Its code and retryable depend on the receipt's outcome:

For exhausted, a failed attempt whose code is Dispatch's own, such as RUNTIME_NOT_FOUND, counts as non-retryable and surfaces as PROVIDER_UNAVAILABLE, because a missing runtime is a configuration error that an outer retry would meet again. The receipt records that attempt with retryable: false as well; dispatch() moves on to the next candidate after a missing runtime regardless of that flag. An exhausted receipt with no failed attempts maps to PROVIDER_UNAVAILABLE, non-retryable. Batch items rejected by Dispatch carry the same code and retryable.

Errors#

With receiptDeliveryFailureMode: "fail-closed", a failure in onReceipt throws ReceiptDeliveryError, even when the model call succeeded. With "best-effort", invoke() returns the model result, or throws the original DispatchRuntimeError, and the delivery failure is reported only through onReceiptDeliveryFailure. Text from the callback's error is never copied into the error or the receipt.

FileAuthorizationLedger#

new FileAuthorizationLedger(options: FileAuthorizationLedgerOptions)

A file-backed AuthorizationLedger that is safe across processes. Options are root (required), lockTimeoutMs (default 5000), and now. See Durable authorization.

withCapabilityEvidence#

withCapabilityEvidence(candidates: readonly ExecutionCandidate[], source: CapabilityEvidenceSource): Promise<readonly ExecutionCandidate[]>

Calls source.evidenceFor(candidate) for each candidate that has no evidence, and attaches what it returns. Candidates that already have evidence, or for which the source returns undefined, are returned unchanged. The order is preserved.

executionPlanDigest#

executionPlanDigest(plan: ExecutionPlan): string

Returns the hex SHA-256 used as the receipt's planDigest. It hashes the prefix dispatch:execution-plan:v1 and a NUL byte, followed by canonical JSON of schemaVersion, role, the request's digest (not the request itself), candidates, budget, policy, and authorization.

DISPATCH_RECEIPT_SCHEMA_VERSION#

The constant 1, the current DispatchReceipt schema version.

Types#

ExecutionPlan#

ExecutionCandidate#

ExecutionBudget#

Any other value, including NaN, Infinity, a string, or a negative number, makes dispatch() and dispatchBatch() throw TypeError("Invalid execution plan: budget.<field> must be ...") before any runtime is invoked. Omit a limit to leave it unbounded; Infinity, which earlier versions accepted as "unbounded", is now refused. See execution plans for migrating other values that earlier versions accepted.

ExecutionPolicy#

CapabilityEvidence#

A CapabilityEvidenceSource has one method, evidenceFor(candidate): Promise<CapabilityEvidence | undefined>.

ExecutionAuthorization#

DispatchReceipt#

DispatchAttemptReceipt#

AuthorizationLedger#

The port that FileAuthorizationLedger implements. A host can supply its own.

Dispatch invokes the provider only when reserve() resolves to reserved.

Datum entrypoint#

import { bindDatumResolvedRef } from "@kontourai/dispatch/datum";

bindDatumResolvedRef(role: string, resolved: ResolvedRef, options?: DatumCandidateOptions): DatumCandidateBinding

Maps the output of Datum's resolveRef() to { candidate, target }:

• candidate is an ExecutionCandidate whose id defaults to role.
• target describes the runtime for the host to build: runtimeId, provider, kind, model, baseUrl when set, and auth with the auth kind, ref, and available flag. It never holds a credential value.

The runtimeId defaults to datum:<provider>:<model>:<digest>, where <digest> is the first 12 hex characters of a SHA-256 of the baseUrl (or of default when there is none). options can set id, runtimeId, evidence, estimatedUsdPer1kTokens, and worstCaseUsage. Datum's ResolvedRef carries no token or cost limits, so worstCaseUsage is never derived; pass it when the candidate runs under an authorization with token or cost limits. Without it, dispatch() throws a TypeError that names the candidate and the missing worstCaseUsage field. An empty role throws TypeError. The function does not create or invoke a runtime.

Bearing entrypoint#

import { capabilityEvidenceFromBearing } from "@kontourai/dispatch/bearing";

capabilityEvidenceFromBearing(candidate: RankedCandidate | RankedCandidateV2, catalogDigest: string): CapabilityEvidence

Projects a Bearing ranked candidate into CapabilityEvidence:

• capabilities are the distinct, sorted measurementKeys of the candidate's reasons with code REQUIREMENT_MET.
• level is confirmed when there is at least one capability and each has an evidence entry with evidence ids, declared when there are capabilities but not all are backed that way, and unavailable when there are none.
• source is bearing:catalog:<catalogDigest>.

The projection does not set structuredToolsFidelity.

AI SDK entrypoint#

import { createAiSdkDispatchModel } from "@kontourai/dispatch/ai-sdk";

createAiSdkDispatchModel(options: AiSdkDispatchModelOptions): LanguageModelV3

Wraps AI SDK v3 language models as Relay runtimes, routes between them with createDispatchRuntime(), and returns one AI SDK v3 model.

If a plan names a runtimeId that is not a key of models, the invocation fails with a non-retryable INVALID_REQUEST error that lists the missing ids.

CLI#

dispatch <fixture|replay> --plan <file> (--fixtures <file>|--records <file>) [--authorization-root <dir>]

The CLI runs one plan without provider credentials. It does not build live runtimes.

--authorization-root <dir> runs with a FileAuthorizationLedger rooted at <dir>.

Each input file may be at most 2 MiB. The CLI writes one JSON document to standard output: { "schemaVersion": "dispatch.cli.result/v1", "result"?, "receipt" }.