Docs menu · Shadow report

The Shadow report

What the Shadow report measures, the explicit criteria an agent must meet before Enforce, and how to get there.

Updated

What it shows

In Shadow Mode nothing is blocked, so the question is what would have happened. The report (console → Shadow report, last 7, 14 or 30 days, per agent or all) answers it in the agent’s currency:

  • Requested spend, and how much was within authority.
  • What Neltava would have prevented — BLOCKs, and the part a LIMIT cuts. Deterministic authority only; a purpose assessment never counts here.
  • What would have gone to a person, split into policy review (a limit, budget, threshold or category rule asked) and purpose review (policy alone would have allowed it; the purpose is in doubt).
  • How much was actually spent, from reported outcomes.
  • Human disagreement, where your labels differ from the purpose assessment.

Console test decisions are listed apart and never counted.

Four stages

StageMeans
ConnectNo agent actions yet in this window.
ObserveDecisions are coming in; not enough time or volume yet.
ProveEnough to judge — coverage, availability and your agreement are being measured.
Ready for EnforceEvery criterion below is met.

Readiness criteria

Readiness is a list of explicit, deterministic criteria — not a confidence score. Each shows its value, what it needs, and whether it is met.

CriterionNeedsWhy
Observation period≥ 7 daysEnough time to see a real week of spending, not a demo.
Observed agent actions≥ 100Enough decisions for the other numbers to mean something.
Authority coverage≥ 95%Decisions made under a real, versioned authority.
Purpose provider configuredyesPurpose can be assessed at all.
Purpose evaluation coverage≥ 99%Almost every spend had its purpose assessed.
Purpose provider availability≥ 99%Assessments rarely failed.
Human purpose labels≥ 20You judged enough decisions yourself to compare against.
Human disagreement with purpose assessment≤ 10%Where you labeled, Neltava's purpose judgment matched yours.
Would-review decisions without a human review≤ 10%Your team keeps up with what would have been sent to it.

Getting to ready

  1. Route every spend the agent makes through Neltava — coverage is measured, and gaps show.
  2. Work the review queue (console → Review): allow or deny what would have been sent to you. Keyboard shortcuts make it quick — A allow, D deny, J skip.
  3. Label at least 20 decisions for purpose: aligned, misaligned or unclear.
  4. Report outcomes (SDK, MCP, API), so “actually spent” is real.
  5. If a rule is too strict or too loose, edit the authority. Preview impact replays the last 30 days of the agent’s real decisions against your edit before you save.

Switching to Enforce

Enforce is switched on per agent, by a person, on the agent’s page — you type the agent’s name to confirm. It needs a paid plan. You can switch before every criterion is met; the console says so and asks you to acknowledge it.

Before you do, put enforcement where money moves: a payment adapter that pays only with a consumed capability. See Enforce at the payment point.

The report over the API

GET /v1/reports/shadow
curl "https://api.neltava.com/v1/reports/shadow?agent=growth-agent" \
  -H "x-api-key: $NELTAVA_ADMIN_KEY"

Query: from and to (ISO-8601; default the last 14 days, at most 93), agent (slug or id), mode (SHADOW by default, ENFORCE or ALL). Needs a key with read.