LlamaIndex
Make a LlamaIndex agent's tool wait for a person's approval with wait_for_event, inside the run or hours later with a saved context and a webhook.
A LlamaIndex agent can stop in the middle of a tool and wait for a person. The tool calls ctx.wait_for_event with an InputRequiredEvent, the event appears in the run’s event stream, and the code running the agent answers with a HumanResponseEvent. Who answers, and where, is up to you. This page makes each question a request in your VariantGate inbox, so the answer can come from a phone, an email link or Slack, a rule can settle the routine ones, and every decision is kept on the record.
1. Install the client
pip install https://variantgate.com/sdk/variantgate-latest.tar.gzIt has no dependencies beyond the standard library and works on Python 3.9 or newer. Make an agent key under Agents in your workspace and put it in VARIANTGATE_KEY. The client reads it from there.
2. Wait for a person inside the tool
Install LlamaIndex too: pip install llama-index-core llama-index-llms-openai. It needs Python 3.11 or newer, and this code ran against llama-index-core 0.14.26. The tool says what it wants to do and waits. The loop around the run asks in the inbox, waits for the answer and sends back only the request id. The tool then reads the request from the API with check and refunds only on an unchanged yes.
import asyncio
import contextlib
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.core.workflow import Context, HumanResponseEvent, InputRequiredEvent
from llama_index.llms.openai import OpenAI
from variantgate import (
ApprovalClient,
ChangedApprovalError,
NotApprovedError,
VariantGateError,
is_allowed,
is_changed,
)
import shop # your own refund code
approvals = ApprovalClient("support-agent") # reads VARIANTGATE_KEY
def refund_if_approved(request_id: str, order_id: str, amount_eur: str) -> str:
"""Read the person's answer from the API, and refund only on an unchanged yes."""
decision = approvals.check(request_id)
if not is_allowed(decision):
return str(NotApprovedError(decision))
if is_changed(decision): # a person approved less, or other wording
with contextlib.suppress(VariantGateError):
approvals.report_outcome(request_id, "failed", "not done: approved with changes")
return str(ChangedApprovalError(decision))
refund_id = shop.refund(order_id, amount_eur)
with contextlib.suppress(VariantGateError): # the money moved: never fail over the record
approvals.report_outcome(request_id, "succeeded", f"refund {refund_id}")
return f"Refunded €{amount_eur} for order {order_id}."
async def refund_order(ctx: Context, order_id: str, amount_eur: str) -> str:
"""Refund an order to the customer's original payment method."""
call = f"refund {order_id} {amount_eur}"
# LlamaIndex runs the tool again from the top when the answer arrives,
# so everything before this line must be safe to repeat.
answer = await ctx.wait_for_event(
HumanResponseEvent,
waiter_id=call,
waiter_event=InputRequiredEvent(
call=call,
approval={
"action": "payment.refund",
"summary": f"Refund €{amount_eur} for order {order_id}",
"amount": amount_eur,
"currency": "EUR",
"details": {"order": order_id},
},
),
requirements={"call": call},
timeout=None, # the request's own expiry ends the wait
)
return await asyncio.to_thread(refund_if_approved, answer.response, order_id, amount_eur)
agent = FunctionAgent(
tools=[refund_order],
llm=OpenAI(model="gpt-5"),
system_prompt="Help customers with their orders.",
allow_parallel_tool_calls=False, # one call at a time, each with its own request
)
async def ask_a_person(event: InputRequiredEvent) -> HumanResponseEvent:
"""Ask in the inbox and wait up to 15 minutes. The tool reads the answer itself."""
asked = await asyncio.to_thread(approvals.ask, **event.approval)
decision = await asyncio.to_thread(approvals.wait, asked, 900)
return HumanResponseEvent(response=decision["id"], call=event.call)
async def run(text: str) -> str:
handler = agent.run(user_msg=text)
async for event in handler.stream_events():
if isinstance(event, InputRequiredEvent):
try:
reply = await ask_a_person(event)
except VariantGateError:
await handler.cancel_run() # the API is unreachable: nothing is refunded
raise
handler.ctx.send_event(reply)
return str(await handler)
if __name__ == "__main__":
print(asyncio.run(run("Order 1182 arrived broken. Please refund the €49.")))- An approved call runs. A denied, expired or withdrawn one, or any call while the workspace is paused, returns a message such as “Not approved: the request is denied” as the tool’s result, so the model can tell the customer instead of trying again.
- If a person lowers the amount or rewords the request, nothing is refunded and the record says why. The model is told what was approved, such as “for at most 20.00 EUR”, and can ask again for that.
- When the answer arrives, LlamaIndex runs the tool again from its first line. That is why the tool does not ask the API itself: a request made before
wait_for_eventwould be made twice. The tool only describes the request, and the loop asks once perInputRequiredEvent. - LlamaIndex hands one answer to every waiting call that matches it, and a call is matched by its order and amount. So the agent makes one tool call at a time (
allow_parallel_tool_calls=False), and two identical refunds can never share one approval. - Once the money has moved, a failure to record the refund never fails the tool, so nothing tempts the agent into a second refund.
- If the API cannot be reached, the loop stops the run and raises, and nothing is refunded. A request still open after 900 seconds is withdrawn, so nobody can approve a refund that will never happen, and the tool reports it as not approved.
- The
approvaldict is what a person reads. Give each tool a clear action name and state the amount, so a rule such as “always allowpayment.refundup to €20” approves the small ones without anyone waiting.
When the answer may take hours
The loop above holds the run open while a person decides. For a decision that may take longer, stop the run while it waits. A run’s context turns into a plain dict with to_dict and back with Context.from_dict, so park saves it with the request id and stops the run. Call resume on every signed webhook about the request; webhooks need the Pro plan or higher. The webhook only says when: the same tool reads the request with check before it refunds. With the code above saved as support.py:
import asyncio
import contextlib
import json
import sqlite3
from collections.abc import Iterator
from llama_index.core.llms import ChatMessage
from llama_index.core.memory import ChatMemoryBuffer
from llama_index.core.workflow import Context, HumanResponseEvent, InputRequiredEvent
from llama_index.core.workflow.handler import WorkflowHandler
from variantgate import VariantGateError
from support import agent, approvals
CLAIM = "UPDATE runs SET claimed = 1 WHERE request_id = ? AND claimed = 0 RETURNING id, call, run"
@contextlib.contextmanager
def database() -> Iterator[sqlite3.Connection]:
"""Your database: a connection per call, so a webhook on any thread can use it."""
with contextlib.closing(sqlite3.connect("runs.db")) as db, db: # commits on the way out
yield db
with database() as db:
db.execute(
"CREATE TABLE IF NOT EXISTS runs"
" (id TEXT PRIMARY KEY, request_id TEXT UNIQUE, call TEXT, run TEXT, claimed INT)"
)
async def start(run_id: str, text: str) -> str | None:
"""Run until the agent needs a person. Returns its answer, or None while it waits."""
return await follow(run_id, agent.run(user_msg=text))
async def follow(run_id: str, handler: WorkflowHandler) -> str | None:
"""Run on until the agent answers, or park the run at its next question."""
async for event in handler.stream_events():
if isinstance(event, InputRequiredEvent):
return await park(run_id, handler, event)
with database() as db:
db.execute("DELETE FROM runs WHERE id = ?", (run_id,))
return str(await handler)
async def park(run_id: str, handler: WorkflowHandler, event: InputRequiredEvent) -> str | None:
"""Save the paused run, ask without waiting, and stop the run."""
try:
memory = await handler.ctx.store.get("memory") # to_dict leaves the chat out
chat = [message.model_dump(mode="json") for message in await memory.aget_all()]
run = json.dumps({"ctx": handler.ctx.to_dict(), "chat": chat})
asked = await asyncio.to_thread(approvals.ask, **event.approval, ttl_seconds=24 * 3600)
finally:
await handler.cancel_run()
with database() as db: # one transaction: the old request goes, the new one comes
db.execute("DELETE FROM runs WHERE id = ?", (run_id,))
db.execute("INSERT INTO runs VALUES (?, ?, ?, ?, 0)", (run_id, asked["id"], event.call, run))
return await resume(asked["id"]) # a rule may have decided before the row was saved
def release(run_id: str) -> None:
"""Hand the run back, so the next webhook or the sweep can take it."""
with database() as db:
db.execute("UPDATE runs SET claimed = 0 WHERE id = ?", (run_id,))
async def restore(run: str) -> WorkflowHandler:
"""Start the saved run again, with the chat that to_dict left out."""
saved = json.loads(run)
handler = agent.run(ctx=Context.from_dict(agent, saved["ctx"]))
memory = ChatMemoryBuffer.from_defaults()
await memory.aset([ChatMessage.model_validate(message) for message in saved["chat"]])
await handler.ctx.store.set("memory", memory)
return handler
async def resume(request_id: str) -> str | None:
"""Call on every signed webhook, and from a sweep. Goes on once the request is decided."""
with database() as db:
row = db.execute(CLAIM, (request_id,)).fetchone() # take the run in one step
if row is None:
return None # unknown, done already, or another delivery has it
run_id, call, run = row
try:
decision = await asyncio.to_thread(approvals.check, request_id)
except VariantGateError:
release(run_id)
raise
if decision["status"] == "pending":
release(run_id)
return None # the webhook for the decision tries again
handler = await restore(run)
handler.ctx.send_event(HumanResponseEvent(response=request_id, call=call)) # the tool reads it
return await follow(run_id, handler) # the agent may need a person again- Webhooks can arrive twice, so
resumefirst claims the saved run in a singleUPDATE … RETURNING(SQLite 3.35 or newer). Only the delivery that claims it goes on, so two deliveries at once refund the order once. - While the request is still pending,
resumeputs the run back and does nothing. A rule can decide before the run is saved, soparkcallsresumeonce itself after saving. Run a sweep every few minutes that callsresumewith the request id of each unclaimed run, for a webhook that never arrived or arrived while another delivery held the claim. - When the continued run asks for another approval,
parksaves it in place of the old one. When the agent is done, the run is removed andresumereturns its answer. to_dictleaves out the agent’s memory and warns that it did. Soparksaves the chat next to the context, andrestoreputs it back before the answer is sent. If you give the agent a memory of your own, save and restore that one instead.- If continuing itself fails, the run stays claimed rather than going back: the tool may already have refunded, so look at the request on the record before you set
claimedback to 0.
When the built-in approval is enough
If the person chatting with the agent is the one who should decide, answer the InputRequiredEvent in your own UI and leave VariantGate out. It earns its place when the agent runs with nobody watching, when someone else must decide, when rules should settle the routine calls, or when you need a record of who approved what. The framework comparison goes through that choice.