Risk tiers for AI agent actions: let it run, ask, or never
Four questions for every action an agent can take, the tier each answer points to, fourteen worked examples, and the rule that holds each tier in place.
· 7 min read
Three piles, “let it run”, “ask” and “never”, are easy to agree on and hard to fill. Where does a €60 refund go? A bulk update to 40 records? This is a worksheet for deciding, with four questions per action, a tier for each answer, and the rule that holds each tier in place.
Four questions for each action
List every action the agent can take, with the agent that takes it, and answer four questions for each:
- Undo
- Can you undo it in minutes, with effort, or not at all? Money sent, email delivered and rows deleted are not at all.
- Reach
- Does it stay inside your systems, reach one person, or reach many people, the public or a regulator?
- Size
- One small thing, a handful, or a whole list? Money under a line you choose, or above it?
- Input
- Does the agent act on your own data, or on text anyone can write, such as emails, tickets and web pages?
The input question is the one people leave out. An agent reading support tickets takes instructions, in effect, from whoever writes the ticket. An action that is safe on your own data can be risky when a stranger can steer it.
From answers to tiers
- Run
- Easy to undo, inside your systems, small, on your own data. Or the worst case is capped by an amount limit and a budget, so it is a cost you accept.
- Ask one person
- Hard to undo, or reaches people outside, or large, or steered by text anyone can write.
- Ask two people
- Cannot be undone and large, or changes where money or access goes: the actions where one person's slip is the expensive kind.
- Never
- Cannot be undone and you could not recover from it, or it changes the controls themselves: permissions, backups, logs.
Caps are what make the run tier useful. A €50 refund cannot be undone, but with a limit per refund and a budget per day, the worst a manipulated agent can do is a known, small number. Without the cap, the same action belongs in ask.
Worked examples
Fourteen actions from common agents, with the tier each lands in. The names follow one convention, explained in naming agent actions.
- crm.update
- Run. One record, easy to change back, your own data.
- deploy.staging
- Run, for the pipeline's key only. The next deploy undoes it.
- refund.create up to €50
- Run, with a daily budget. It cannot be undone, but the worst day is a cost you chose.
- ticket.reply
- Ask at first: the customer's text steers it. An early candidate for run, since it reaches only the person who wrote.
- refund.create above €50
- Ask one person: the people who own refunds.
- email.send_bulk
- Ask one person, who sees the count and the list. Two if one bad send would make the news.
- deploy.production
- Ask one person: whoever is on call.
- data.erase.customer
- Ask one person: the privacy lead.
- payment.* above €1,000
- Ask two people: large and gone once sent.
- payee.change_bank_details
- Ask two people at any amount: it decides where every later payment goes.
- infra.destroy.*
- Ask two people: production data may not come back.
- db.schema_change from the data agent
- Never. Schema goes through reviewed migrations.
- backup.delete
- Never, and no credentials for it either.
- iam.grant.admin
- Never. An agent does not raise anyone's access.
Each tier is a rule
- Run is an “always allow” rule for an exact action name and one agent key, with an amount limit and a budget where money moves.
- Ask one person needs no rule: with no rule, a request waits for a person. Write an “always ask” rule only to send it to particular people, or to hold back amounts above a line.
- Ask two people is an “always ask” rule that needs two approvals.
- Never is a “never allow” rule, and credentials that cannot do it anyway.
Do not write “always ask *” to make asking the default. It already is, and an ask rule beats every allow rule, so the run tier would never work.
The same action, different tiers
Tiers belong to an action and an agent. A deletion the privacy agent prepares carefully is a different risk from the same deletion by a support bot that read an angry email. Give each agent its own key, write allow rules for one key at a time, and deny an action outright for the agents that have no business doing it.
Moving an action between tiers
- Up to run, when the record shows a long run of approvals for the action, none of them edited or hesitated over.
- Try it for a day first. Approving with “And for the next day” writes an allow rule for that action and key that ends on its own.
- Down to ask, after any incident, or when denials for the action start to appear. Remove the rule; requests it already decided stay decided.
- Never moves up only by a deliberate change to the rule, made by an owner. An agent key cannot write rules.
For how a tier set looks in full for one kind of agent, see the guides, such as support refunds or production deploys. The rule types themselves are on Rules.