Risk tiers for AI agent actions: let it run, ask, or never

Four questions for every action an agent can take, the tier each answer points to, fourteen worked examples, and the rule that holds each tier in place.

· 7 min read

Three piles, “let it run”, “ask” and “never”, are easy to agree on and hard to fill. Where does a €60 refund go? A bulk update to 40 records? This is a worksheet for deciding, with four questions per action, a tier for each answer, and the rule that holds each tier in place.

Four questions for each action

List every action the agent can take, with the agent that takes it, and answer four questions for each:

Undo
Can you undo it in minutes, with effort, or not at all? Money sent, email delivered and rows deleted are not at all.
Reach
Does it stay inside your systems, reach one person, or reach many people, the public or a regulator?
Size
One small thing, a handful, or a whole list? Money under a line you choose, or above it?
Input
Does the agent act on your own data, or on text anyone can write, such as emails, tickets and web pages?

The input question is the one people leave out. An agent reading support tickets takes instructions, in effect, from whoever writes the ticket. An action that is safe on your own data can be risky when a stranger can steer it.

From answers to tiers

Run
Easy to undo, inside your systems, small, on your own data. Or the worst case is capped by an amount limit and a budget, so it is a cost you accept.
Ask one person
Hard to undo, or reaches people outside, or large, or steered by text anyone can write.
Ask two people
Cannot be undone and large, or changes where money or access goes: the actions where one person's slip is the expensive kind.
Never
Cannot be undone and you could not recover from it, or it changes the controls themselves: permissions, backups, logs.

Caps are what make the run tier useful. A €50 refund cannot be undone, but with a limit per refund and a budget per day, the worst a manipulated agent can do is a known, small number. Without the cap, the same action belongs in ask.

Worked examples

Fourteen actions from common agents, with the tier each lands in. The names follow one convention, explained in naming agent actions.

crm.update
Run. One record, easy to change back, your own data.
deploy.staging
Run, for the pipeline's key only. The next deploy undoes it.
refund.create up to €50
Run, with a daily budget. It cannot be undone, but the worst day is a cost you chose.
ticket.reply
Ask at first: the customer's text steers it. An early candidate for run, since it reaches only the person who wrote.
refund.create above €50
Ask one person: the people who own refunds.
email.send_bulk
Ask one person, who sees the count and the list. Two if one bad send would make the news.
deploy.production
Ask one person: whoever is on call.
data.erase.customer
Ask one person: the privacy lead.
payment.* above €1,000
Ask two people: large and gone once sent.
payee.change_bank_details
Ask two people at any amount: it decides where every later payment goes.
infra.destroy.*
Ask two people: production data may not come back.
db.schema_change from the data agent
Never. Schema goes through reviewed migrations.
backup.delete
Never, and no credentials for it either.
iam.grant.admin
Never. An agent does not raise anyone's access.

Each tier is a rule

  • Run is an “always allow” rule for an exact action name and one agent key, with an amount limit and a budget where money moves.
  • Ask one person needs no rule: with no rule, a request waits for a person. Write an “always ask” rule only to send it to particular people, or to hold back amounts above a line.
  • Ask two people is an “always ask” rule that needs two approvals.
  • Never is a “never allow” rule, and credentials that cannot do it anyway.

Do not write “always ask *” to make asking the default. It already is, and an ask rule beats every allow rule, so the run tier would never work.

The same action, different tiers

Tiers belong to an action and an agent. A deletion the privacy agent prepares carefully is a different risk from the same deletion by a support bot that read an angry email. Give each agent its own key, write allow rules for one key at a time, and deny an action outright for the agents that have no business doing it.

Moving an action between tiers

  • Up to run, when the record shows a long run of approvals for the action, none of them edited or hesitated over.
  • Try it for a day first. Approving with “And for the next day” writes an allow rule for that action and key that ends on its own.
  • Down to ask, after any incident, or when denials for the action start to appear. Remove the rule; requests it already decided stay decided.
  • Never moves up only by a deliberate change to the rule, made by an owner. An agent key cannot write rules.

For how a tier set looks in full for one kind of agent, see the guides, such as support refunds or production deploys. The rule types themselves are on Rules.