OWASP ASI09 Human-Agent Trust Exploitation: when the approver is the target
A poisoned invoice, a confident explanation, a manager who approves. How to ask for approval so the person sees facts your code checked, not a story the model wrote.
· 8 min read
ASI09 Human-Agent Trust Exploitation is the ninth risk in OWASP’s Top 10 for Agentic Applications, and the one aimed at the person in the loop. The agent is fluent, confident and usually right, so people come to trust it. An attacker, or a badly designed agent, uses that trust to get a person to approve something harmful.
OWASP’s description has a line every team with an approval step should read twice: the agent “acts as an untraceable ‘bad influence,’ manipulating the human into performing the final, audited action, making the agent’s role in the compromise invisible to forensics.” The approval itself becomes the attack.
What it looks like
- Invoice fraud. A poisoned invoice reaches the finance copilot. It recommends an urgent payment to the attacker’s bank details, the manager trusts the explanation and approves.
- Made-up reasons. A hijacked agent writes a plausible rationale for deleting a production database or changing a configuration, and the reviewer approves on the strength of it.
- A preview that is not read-only. Opening a preview triggers a webhook, so “just looking” already did something.
OWASP separates this from ASI10: ASI09 is about people trusting too much, ASI10 about the agent drifting. It builds on Excessive Agency and is often set off by prompt injection. Its mitigations translate into five habits for how you ask.
1. Ask with facts, not with the model’s story
OWASP’s fourth mitigation asks for a “plain-language risk summary (not model-generated rationales)”. The easy way to build an approval step gets this backwards:
// The person reads the model's story, and the story is the attack.
await approvals.ask({
action: "payment.send",
summary: plan.explanation,
// "Urgent: the vendor moved banks. Pay today to avoid a late fee."
});Build the summary from the arguments your code is about to execute, and add the checks a careful person would make, worked out by code rather than asserted by the model:
// The person reads facts your code looked up and compared.
const vendor = await vendors.get(invoice.vendorId);
const ibanOnFile = vendor.iban === invoice.iban;
await approvals.guard(
{
action: "payment.send",
summary: `Pay €${invoice.total} to ${vendor.name}, invoice ${invoice.number}`,
amount: invoice.total,
currency: "EUR",
details: {
iban: invoice.iban,
iban_check: ibanOnFile
? "same as the vendor record"
: "NOT the IBAN in the vendor record",
vendor_paid_before: vendor.paymentsCount,
invoice_came_from: invoice.sender,
agent_says: plan.explanation.slice(0, 300),
},
},
() => bank.pay({ iban: invoice.iban, amount: invoice.total })
);The person deciding now sees the IBAN, whether it matches the vendor record, and where the invoice came from. The model’s explanation is still there, labelled as what the agent says, and it can no longer stand in for the facts. In the invoice scenario, “NOT the IBAN in the vendor record” is the line that saves the money.
2. Opening is not approving
The seventh mitigation is to “separate preview from effect”: nothing that only shows a request may change state. In VariantGate an Approve link in an email opens a page with the request and one button. Nothing is decided until that button is pressed, so a mail scanner that follows links, or a curious click, cannot approve. Apply the same rule to your own tools: a GET never acts.
3. Two people for the ones that cannot be undone
The first mitigation is explicit confirmation, with “multi-step approval” for risky actions. One trusting person is exactly what this attack needs, so make the largest actions need two: a rule that sends payments above an amount to two different people, where one denial stops it. See two-person approval.
4. Fewer, better questions
OWASP asks for “adaptive trust calibration”: oversight that scales with risk, and cues that make people question high-impact actions instead of approving on autopilot. A person who approves forty routine requests a day will approve the forty-first. Let standing rules answer the small, reversible ones, with an amount limit and a budget, so the questions that reach a person are the ones worth reading. The mechanics are in approval fatigue and standing rules.
When a request is almost right, the person can approve a narrower version instead: a lower amount or a reworded summary. The agent may then do only what was approved, and changed tells it so.
5. Keep the agent’s part in the record
The second mitigation is immutable logs, and the forensic worry above is that the agent’s role disappears. A record that keeps the request exactly as the agent asked, which agent key asked, who decided and with what note, keeps that role visible. VariantGate seals each decision into a chain, so an entry cannot be changed later without the check failing.
OWASP also asks for a way to flag suspicious behaviour that triggers “a temporary lockdown of agent capabilities”. Any member of a workspace can pause it, or pause one agent; while paused, nothing that agent asks for is allowed, whatever was approved before. Only an owner can resume.
An approval step does not make people immune to persuasion. It gives them the facts at the moment it matters, a second pair of eyes for the worst cases, and a record that shows who asked for what. Training people to read the facts first is still on you.
The other nine risks, and which ones an approval step helps with, are in the OWASP agentic Top 10 overview. To see what a person sees when an agent asks, try the playground; it runs in the browser without an account.