Human-in-the-loop agent spending

Not every spend needs a person

“Human in the loop” is easy to say and expensive to do. If a person has to approve every agent purchase, you haven’t automated spending — you have added a queue. If no person ever looks, you have handed a budget to software and hoped.

The useful middle is narrow on purpose: people decide the spends that are doubtful, and nothing else. Clear cases are allowed, capped or blocked by the rules of the agent’s spend authority. The rest — and only the rest — wait for a person.

What sends a spend to a person

A spend becomes REVIEW for one of a small number of reasons, each recorded with the decision:

• It exceeds the per-transaction limit or the remaining budget, and the action can’t simply be capped.
• It tips a windowed total — more to one payee, one category or overall in 24 hours than the authority allows — and can’t be capped.
• It is above the review threshold — an approval ceiling that sits below the limits: allowed in principle, but big enough that someone should look.
• Its purpose is in doubt. The purpose assessment is uncertain or misaligned, or couldn’t be made at all — a failed check is never read as approval.
• Its category can’t be trusted: the agent declared a category that can’t be verified against a list it is restricted to, or contradicts the category Neltava trusts.

If the action is limitable — ad budget, credits, top-ups — amount problems cap the spend to the safe amount instead of asking anyone. That alone removes most of a typical review queue.

How a review works

1 · The agent asks, and waits

The agent requests a $620 conference booth. Its transaction authority is $500, and booths can’t be bought by the half:

curl -X POST https://api.neltava.com/v1/decisions \
  -H "x-api-key: $NELTAVA_AGENT_KEY" \
  -H "idempotency-key: run-7-step-3" \
  -H "content-type: application/json" \
  -d '{
    "task": "Q4 pipeline: enterprise developer leads",
    "action": {
      "type": "purchase",
      "merchant": "DevConf",
      "amount_minor": 62000,
      "currency": "USD",
      "description": "Booth at an enterprise developer conference"
    }
  }'

{
  "decision_id": "dec_7f3c…",
  "decision": "REVIEW",
  "effective_decision": "REVIEW",
  "reason_code": "TRANSACTION_LIMIT_EXCEEDED",
  "requested_amount_minor": 62000,
  "authorized_amount_minor": null,
  "currency": "USD",
  "mode": "ENFORCE",
  "explanation": "620.00 USD exceeds the 500.00 USD transaction authority; a person decides."
}

effective_decision is REVIEW: the agent must not spend. The decision waits in the console’s review queue.

2 · A person decides, with the evidence

The reviewer sees what the agent can’t argue with: the delegated purpose, the spend, the agent’s task and steps, every rule’s result and the purpose assessment — side by side. They allow or deny, with a reason:

The console’s review queue is keyboard-first. The same review through the API:

curl -X POST https://api.neltava.com/v1/decisions/dec_7f3c…/reviews \
  -H "x-api-key: $NELTAVA_REVIEWER_KEY" \
  -H "content-type: application/json" \
  -d '{
    "action": "ALLOW",
    "reason_code": "SERVES_PURPOSE",
    "comment": "Enterprise conference, planned in the Q4 budget."
  }'

3 · The decision takes effect

On allow, the amount is reserved against the agent’s budget at once, and the decision now carries a single-use capability for the requested amount — the agent picks it up with the decision, and its payment service consumes it before paying. On deny, nothing is spendable.

An enforced review is final: a second review is refused, and a person who wants to record a different judgment later does so as a label. Reviews happen within the decision’s review window; once it closes, the agent has to ask again.

Separation of duties

A review is worthless if the requester can approve itself. So:

• An agent’s key can never review or label anything — not its own decisions, not another agent’s.
• The credential that requested a decision can never review it.
• Where credentials carry a principal — a person, a team, a service — the same principal can’t review its own request through a different key either.
• Every review records who made it, with which credential and scopes, and it is chained into the workspace’s tamper-evident record.

In the console, workspace owners and members can review; nobody needs admin rights to do it.

Reviews in Shadow Mode

In Shadow Mode, nothing is waiting — the agent already proceeded. A review there is a counterfactual: I would have allowed this, or I would have denied it. It changes nothing, can be corrected later, and it is exactly the evidence you need to decide whether an agent’s thresholds are right before they bind.

Labels: the ground truth

A review answers “should this spend happen?”. A label answers a different question, for any decision: did the spend serve the purpose — aligned, misaligned or unclear? Labels are how the purpose assessment is measured. Before an agent is ready for Enforce, its readiness criteria ask for at least 20 labels on assessed decisions, and for human disagreement with the assessment at or below 10%.

Keeping the queue small

• Mark cappable spends limitable, so amount problems become a capped spend, not a question.
• Set the review threshold from data. The Shadow report shows how many spends each threshold would have sent to a person; replay the recorded decisions against a new threshold before you change it.
• Use allowlists where the vendor set is known: a known payee inside the limits needs no one.
• Write the purpose precisely. Vague objectives produce uncertain assessments, and uncertain assessments produce reviews.

When you don’t need it