Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 39 ยท Evaluation and Safety

Human-in-the-Loop and Permissions

Design approval gates for AI agents: classify actions by risk, pause for human confirmation, support edit and reject, log decisions and set autonomy levels, with runnable Python.

Autonomy is a dial, not a switch. At one end the agent only suggests; at the other it acts without asking. Most useful agents sit in the middle: they act freely on safe things (reading, searching, drafting) and pause for a human on risky ones (spending money, deleting data, sending messages to other people). This design is called human-in-the-loop (HITL).

In this lesson you will learn how to classify actions by risk, build an approval gate that pauses and resumes, let humans edit or reject a proposed action, keep an audit log, and choose an autonomy level. The examples run here with a simulated human.

Real approval flows use a UI, chat message or ticket. Here the "human" is a function so the control flow is easy to see. Framework features for pausing (often called interrupts) follow the same idea.

Why approvals matter

Even a good model makes mistakes: it misreads a request, hallucinates an argument or gets tricked by injected text. Approval gates make the worst outcomes recoverable. The cost is friction, so the skill is asking only when it matters. An agent that asks about everything gets ignored; users click "approve" on autopilot.

Classify actions by risk

Rate each tool by two questions: can it be undone, and who is affected?

Risk Examples Policy
Low search, read file, calculate Run automatically
Medium write a draft, create a ticket Run, then show the result
High send email, delete data, spend money, deploy Ask first
Forbidden access other users' data, disable logging Never allow

Store the classification in code next to the tool definition, so it cannot be argued away in conversation.

An approval gate

The gate sits between the agent's proposal and the actual execution.

Python Approve, edit or reject a risky action

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Three decisions are worth supporting. Approve runs the action as proposed. Edit lets the person fix an argument (very common: a wrong recipient or amount). Reject returns a reason to the agent, which should be told so it can try a different plan instead of retrying the same action.

Pause and resume

In a real system the human may answer minutes or days later, so the run must be saved at the pause point and resumed after the decision. That means keeping the agent's state (messages, pending tool call) in a database. The loop below separates the two phases.

Python Pause for approval and resume later

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The saved state must be self-contained. Store the exact proposed call, so what the person approved is exactly what runs, with no chance the agent changes the arguments in between.

Good approval requests

A person can only judge what they can see. Show:

  • what will happen, in plain words ("Send an email to [email protected]"),
  • the exact arguments (recipient, amount, file),
  • why the agent wants to do it,
  • what cannot be undone,
  • approve, edit and reject buttons.

Avoid walls of JSON and avoid asking for a blanket "allow everything". Per-action approval is safer than one big yes.

Autonomy levels

Offer users a clear setting, and let them change it as trust grows.

  1. Suggest only: the agent proposes; the human does everything.
  2. Ask for every change: read freely, approve every write.
  3. Ask for risky changes: the policy above.
  4. Autonomous within a sandbox: act freely, but only inside a restricted environment with a spending limit and rollback.

Combine levels with budgets: maximum spend, maximum emails per hour, maximum files touched. A budget turns "ask about everything" into "ask when over the limit".

Audit trails

Record every proposal, decision, who decided and when. Audit logs answer "who approved the refund?" and help you tune the policy: if people approve 99 percent of a certain action unchanged, it may deserve a lower risk class. If they edit it often, the agent needs improvement. See observability and tracing for recording this.

Permissions beyond approvals

Approvals are one control. Also use scoped credentials (a token that can only read one folder), per-user permissions (the agent can only touch what the current user may touch) and sandboxes for code execution. A permission the agent never had cannot be misused, whatever the prompt says.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which actions should usually require human approval?

    High-impact, hard-to-undo actions deserve a pause; low-risk ones should run automatically.
  2. 2. Why let a human edit a proposed action instead of only approve or reject?

    Editing keeps the useful part of the proposal while fixing the mistake.
  3. 3. What must be saved when a run pauses for approval?

    Resuming needs the full context, and storing the exact call guarantees what was approved is what runs.
  4. 4. Why not ask for approval on every action?

    Approval fatigue makes the gate meaningless, so ask only when it matters.

Summary

  • Autonomy is a dial; let agents act freely on safe tasks and pause on risky ones.
  • Classify tools by risk in code, and give a forbidden class that is never allowed.
  • Offer approve, edit and reject; pause by saving the exact pending call and resume after the decision.
  • Show what, why and consequences; avoid blanket approvals and approval fatigue.
  • Add budgets, scoped credentials, sandboxes and an audit log.

Next: Agent Observability and Tracing