Community Article
Community articles are authored by SitePoint Premium contributors. Content is screened before publication, and SitePoint reserves the right to moderate or remove articles that violate our guidelines. Views expressed are those of the authors and do not necessarily reflect those of SitePoint.
AI Agents Need Idempotency: The Distributed Systems Problem Hiding Inside Agentic AI
ASAditya ShrivastavaPublished inOctober 23, 2026
·Updated:October 23, 2026
The AI briefing for Developers
Stay up to date with AI tools, model releases, and developer workflows that matter.
Weekly. Free. One click to leave.
SitePoint Premium
Stay Relevant and Grow Your Career in Tech
- Premium Results
- Publish articles on SitePoint
- Daily curated jobs
- Learning Paths
- Discounts to dev tools
7 Day Free Trial. Cancel Anytime.
A customer wants a $480 refund for a faulty part of a shipment he received. After waiting for 30 seconds for the payment API to process the refund, the customer support AI sends another request. It ultimately is fulfilled. However, as it turns out, the first request had also been fulfilled. The only thing missing was the notification of the refund for the customer. So the customer received two refunds of $480 each instead of the one he was asking for.
The AI was acting appropriately within its given constraints; the surrounding system couldn’t tell if the initial action had already been taken and failed to deliver the result. A well-known problem in distributed systems is gaining more notoriety as AI-powered agents increasingly perform actions beyond simply generating text. Idempotence is the solution to this problem.
Idempotence means an action can be performed multiple times with the same result as performing it once. This concept is particularly important for an AI agent that issues refunds, creates tickets, sends emails, or updates business applications.
AI agents are becoming distributed systems
AI agents as distributed software components have simple processes.
Think → choose a tool → call an API → receive a result → decide what to do next.
As we have outlined, many of the agent actions will utilize services and networks that can have issues from time to time. Consider the simple action of asking a payment provider to issue a refund. There are three possible outcomes:
The payment service never gets the request.
The refund went through, and the agent got the answer.
The refund went through, but the agent never received it back.
The most severe case, however, is when the time to complete an operation exceeds the timeout, but the operation does complete successfully. Since there is no answer, the software agent never learns of the operation’s successful completion and must attempt to complete the operation again.
Distributed systems have dealt with the issues of unreliable networks for a long time already (retries, duplicate requests, etc.). By introducing agents, we give the software more freedom to choose when and how to proceed.
Not every tool should be retried the same way
Consider an agent with these tools: search_customer(), check_inventory(), refund_payment(), send_email(), create_order(), delete_account()
Calling search_customer() twice will usually not cause much trouble.
Calling refund_payment() twice could.
In other words, an agent runtime should be able to tell if a tool only gets data or makes changes to the outside world. In the second group are things like sending money, getting money back, placing orders, writing messages, making accounts, and changing infrastructure.
There are side effects to these treatments. It’s risky to think that agents can just retry every failed tool call as they get access to more business systems.
Idempotency makes retries safer
One common way to handle the duplicate requests issue is to implement idempotency keys.
Idempotency-Key: refund-order-123-run-842
The service stores the idempotency key and the result of processing the associated request.
It allows the refund to be processed the first time the idempotency key is sent to the service by the agent. Thereafter, if the response to the refund is lost and the agent attempts to resubmit the refund with the same idempotency key, the service will recognize that the refund has already been processed and return the previous result to the agent instead of processing another refund.
Many of the world’s leading payment companies, including Stripe, PayPal, and Square, use idempotency keys to allow their customers to safely retry requests.
However, one important rule for agents is that the agent software or workflow system—not the AI model—creates the idempotency key, and it is up to date.
Operation ID: refund-order-123-run-842
The model should not be the system’s memory
Things could be easy if we could tell the model to ‘remember if it has already issued one of these before issuing another.’ But even in the production environment, this approach is not sufficient. The agent could crash. The context could go away. The workflow could be taken off to be executed on another machine. The problem is that the model lacks sufficient information about what has or has not happened in external systems. Instead, the system should keep a record of the execution state outside of the model. The model determines what should happen. The runtime determines safe execution.
Image 1: generated using ChatGPT
Tools need more than a name and description
Modern agent tools include a name, description, and input parameters for the production system. More importantly, the agent runtime also needs to know about the behavior of the individual tool. For example, ‘refund_order’ modifies the application outside this tool, so the operation must be re-executed with the same actionID used for the initial failure. Also, as an example, refunds larger than $5,000 require manual approval. On the other hand, a ‘search_customer’ action is only a read, so re-execution can occur without human intervention. Based on this information, the OS makes decisions about the danger of the individual tool call. Automatic re-execution is still possible, even if the search fails. However, refunds require a bit more caution to avoid double posting of money. Thus, in addition to the input models for the AI tools used in real life, so-called “execution rules” are also required. These rules state when re-execution, increased security, or human intervention is required.
Idempotency is only part of the solution
Suppose an AI agent were to perform the following three actions: 1) Make whole the customer, 2) Send a confirmation email with the order update and a proof, and 3) Check the status of the order in process. We had described how stopping the AI from paying out twice was only half the problem; the system also needed to know where the work had stopped. To that end, we’d like to introduce the concept of durable execution. For durable execution, the agent must do more than simply remember the actions it has taken. We need to keep track of the progress of the actions of the AI. The refund action was marked complete, but the email action was not sent yet. Thus, when the agent is restarted, it continues from where it left off rather than starting over. To create reliable AI agents, we need a few things: stable operation IDs, irrevocable actions, durable state, controlled retries, and logging. There are also cases where one wants to perform a recovery or compensating action (e.g., the agent makes a reservation but then fails to perform the subsequent action). The recovery action would be to cancel the reservation that had been made.
Human approval does not solve duplicate execution
While controls with a person in the loop are mostly suitable for fine-grained actions that cannot be undone, idempotence ensures that an action is not accidentally executed more than once, and human approval is required to decide whether the action should be executed in the first place. This point is illustrated by an example: An employee correctly approves a $10,000 refund. However, the payment request somehow times out and is automatically re-executed several times. Without adequate idempotency security, the customer might end up with even more refunds than intended. Note, however, that a person’s approval cannot prevent double killing in such high-risk agent processes. Therefore, additional decision controls and corresponding execution controls are usually required to ensure that all actions are executed safely and correctly.
Five questions to ask before deploying an agent
There are 5 simple questions that need to be answered before allowing an AI agent to take real-world actions.
- Which tools change external systems or create side effects?
- Which actions are safe to retry?
- How does the system detect multiple attempts to perform the same logical action?
- If the agent crashes in the middle of a workflow, will it continue from the point of failure or restart the workflow?
- Can we trace attempted actions, completed actions, retried actions, and failed actions?
Improving the model alone will not make the agent reliable unless we can answer all of these questions.
Reliable agents need more than better reasoning
Much work in AI today focuses on making agents smarter, better at planning, and better to use as tools. But running agents in production has its own problems. API calls time out, processes crash, networks fail, and responses get lost, and the smartest agent can try again and again until it works. These are not new problems. Engineers working on distributed systems have been struggling with these same issues for decades. So long as the AI agent is confined to a simple test environment, of course it can query payment systems, customer information, and corporate tools. But when it is turned loose on the production corporation’s infrastructure (with all the payment systems, customer information, and corporate tools that entails), it must also be reliable. A smart model can figure out that a customer is due a refund. But a strong system can only give that customer a refund once. So the important question to ask of an AI agent before turning loose on it is, “Some powerful new tool: what happens if the action worked?” The answer will tell you whether or not the AI agent is currently able to operate reliably in production.
