Evidence Readiness Check

Before a full Agent Authorization Review, ActionBoundary can check whether one existing redacted trace or exported log is scoreable.

This is a small pre-pilot gate. It uses evidence the team already has whenever possible: LangSmith, Langfuse, OpenTelemetry, Datadog, CloudWatch, internal JSON logs, tool invocation tables, audit tables, or payment job logs. No new instrumentation and no new staging run are required for the first diagnostic.

If there is no existing trace, or the trace is not representative, the same check can use one synthetic staging run.

Goal

Decide whether one high-impact agent action can support a trace-backed authorization verdict.

The check does not decide that the system is safe. It decides whether the trace can prove what happened well enough to run the full pilot.

Input

Preferred input:

• one existing redacted trace, exported log, or structured table for a high-impact action.

Acceptable sources include LangSmith, Langfuse, OpenTelemetry/OTLP, Datadog, Honeycomb, New Relic, CloudWatch, internal JSON logs, tool invocation tables, approval audit tables, payment job logs, webhook logs, or sandbox ledger events.

The trace can come from an ordinary development, staging, demo, or sandbox run. It does not need to be a new test scenario for the first diagnostic.

If no useful existing trace is available, use one synthetic staging run for a high-impact action, such as:

• schedule a payment;
• submit or release a payment batch;
• update vendor remittance details;
• issue a refund;
• grant access;
• export customer or vendor data;
• submit a regulated or customer-visible record.

Use synthetic, redacted, de-identified, or harmless canary data. Do not send production credentials, real secrets, unrestricted admin accounts, raw customer data, payment card data, financial account data, PHI, or PII.

What ActionBoundary Checks

The readiness check looks for runtime evidence that can be correlated to the same action:

• acting identity or service account;
• target resource, such as invoice, vendor, tenant, account, or record;
• authorization source, approval lookup, policy decision, or permission check;
• request provenance and the exact action/target/parameters bound to approval;
• execution-time rereading of authoritative state;
• tool decision;
• tool result;
• business, ledger, sandbox, or audit-log outcome;
• independent postcondition or mutation record;
• trace ID, correlation ID, event IDs, or timestamps.

For multi-agent or externally orchestrated workflows, the check also maps any existing workflow phase, parent/delegation chain, tool grant, deterministic gate, state-artifact version, knowledge version, and fork/join evidence. These fields are optional unless the selected scenario depends on them.

Scenario setup is useful context, but it does not count as runtime evidence.

Output

The result is one of three readiness levels:

Machine-readable minimal event profile

If existing logs do not expose enough evidence, use evidence_event.schema.json as a vendor-neutral field target. It does not require an ActionBoundary SDK. Existing OpenTelemetry spans/logs, internal JSON events, audit tables, and job records can be exported or mapped into the same event names.

python -m actionboundary validate \
  --evidence-events examples/minimal_evidence_events.redacted.json
python -m actionboundary readiness \
  --evidence-events examples/minimal_evidence_events.redacted.json \
  --out tmp/evidence-readiness.json

The minimal profile covers workflow state, authorization decision, exact approval binding, harness gate, tool attempt, execution revalidation, execution receipt, and independently observed postcondition. Missing event classes are reported as a gap map rather than treated as a failed security control.

Engagement Ladder

Readiness is a path, not a rejection. Different teams can start at different levels:

Only Level 3 supports a full trace-backed authorization verdict. Lower levels still produce useful buyer-readiness work without pretending the system has passed or failed.

Example Finding

PARTIALLY_READY

Observed:
- tool calls
- tool results
- target invoice ID

Missing:
- acting principal or service account
- approval lookup source
- final sandbox ledger outcome

Strict verdict today:
INCONCLUSIVE

Recommended instrumentation:
1. Emit actor_id and tenant_id at the tool gateway.
2. Log approval lookup source, approval_id, and decision reason.
3. Emit payment_intent_id or business_action_key for retries.
4. Record final sandbox ledger outcome.

What This Is Not

The readiness check is not a full authorization review, penetration test, compliance certification, SOC report, or production security assessment.

It is an evidence-readiness diagnostic: it tells the team whether one high-impact action produces enough runtime evidence for an independent review.

When To Use It

Use this before the full pilot when:

• an existing trace or log may already show enough to diagnose scoreability;
• staging exists, but logging may be incomplete;
• tool calls are visible, but authorization decisions are not structured;
• side effects are created asynchronously through jobs, webhooks, or ledgers;
• the team is not sure whether current traces separate scenario setup from runtime evidence;
• a customer is likely to ask how the agent proves that money movement, data export, access grant, or record change was authorized.