Webhook Idempotency: How to Handle Duplicate Events (2026)

Webhooks are the backbone of modern event-driven architecture, enabling real-time communication between disparate services. However, the distributed nature of the web means that "exactly-once" delivery is functionally impossible to guarantee.

Network latency, server timeouts, and retry logic often result in the same event being sent multiple times.

Whether you are building a custom listener or using an automation tool like Activepieces to connect your apps, failing to account for these duplicates can lead to corrupted data, double-charged customers, or redundant database entries.

Understanding why these duplicates occur is the first step toward implementing robust idempotency strategies that ensure your system remains consistent regardless of how many times a single webhook is fired.

Webhook idempotency is the practice of designing receiver-side systems to recognize and ignore redundant event notifications, ensuring that a single action is not processed multiple times due to network retries or delivery overlaps.

Webhook delivery guarantees prioritize reliability over uniqueness

When a sender prioritizes payload delivery over the risk of creating duplicate records, they follow an "at-least-once" system. This design choice shifts the burden of idempotency to the receiver, as network jitter or slow processing can easily trigger redundant transmissions.

Webhook delivery standards

Standard delivery protocols assume that a failure to receive an acknowledgment is a failure to deliver the message. To mitigate transient network issues, providers like ScaiLabs employ aggressive retry schedules.

60 seconds pass before the first and second retries occur, meaning a brief flap in your load balancer can result in three identical hits within two minutes.

The third retry jumps to 300 seconds, extending the window where your database must remain aware of the initial transaction ID.

The fourth retry occurs at 900 seconds, so a developer must account for a fifteen-minute latency spike before a failure is officially reported. A service recovering from a fifteen-minute outage will immediately face a surge of re-transmitted payloads.

Why senders prefer duplicates

Providers would rather flood your endpoint with duplicates than lose a single event. WebhookWhisper documents these maximum retry windows, which define the period during which your system must guard against re-processing old data.

Providers would rather flood your endpoint with duplicates than lose a single event.

Three days of idempotency logs are required for Stripe, which retries for 72 hours, meaning any data purged before that window expires will result in duplicate charges for the customer. Svix retries for 28 hours, requiring just over a full day of state retention.

Slack retries for 1 hour, so the system discards late-arriving retries relatively quickly. PagerDuty retries for 0.33 hours, so any downtime exceeding twenty minutes results in permanent data loss.

How 200 OK responses confirm webhook delivery

Only if the receiver returns a 200 OK status within a strict, provider-defined timeout window does a sender consider a delivery successful. Activepieces manages these logic paths by syncing flows to git and promoting them through Release Management across separate projects.

This ensures that deduplication rules are versioned and reviewed like software, rather than being accidental side effects of a "publish" button, providing a clear audit trail for how a 200 OK is triggered.

The following chart illustrates the disparity between a typical 100ms API response and the maximum seconds a provider will wait before timing out and retrying the request.

• Activepieces: 30 seconds, providing a generous buffer for complex workflows.
• Twilio: 15 seconds, so a slow third-party lookup will trigger a duplicate SMS event.
• GitHub: 10 seconds, meaning heavy CI/CD triggers must offload work immediately.
• Shopify: 5 seconds, so any synchronous database write that locks for five seconds or more generates a duplicate webhook.

Because these timeouts are so varied, engineers must treat the receipt of a webhook and the processing of its data as two separate, decoupled operations.

Aggressive retry policies trigger most duplicates

When a platform interprets a delayed server response as a failure and re-transmits the payload, duplicate events occur, forcing the receiver to reconcile the state.

Maximum webhook retry windows by provider

Because the duration and frequency of these retries vary wildly between services, a temporary outage on your end can result in a backlog of duplicate data that persists for days.

This disparity forces engineers to build for the longest possible window. A three-day retry cycle from a payment processor requires a much larger idempotency key cache than a one-hour window from a chat app.

ScaiSend webhook retry backoff

To prevent an accidental Distributed Denial of Service (DDoS) on your endpoint, the ScaiSend delivery engine uses an exponential backoff strategy where the interval between retries doubles after each failure.

If your server is struggling under high load, this backoff provides the necessary breathing room for your infrastructure to recover. By the time a retry succeeds, however, the delay means the data inside it may be minutes or hours old.

Distributed systems and the 'two generals' problem

Reliable webhook delivery is an implementation of the Two Generals' Problem. In this scenario, two parties cannot reach a state of absolute certainty that the receiver got a message over an unreliable link.

You cannot solve the physics of the network.

Your code must treat every incoming webhook as a potential duplicate that requires a server-side check against a unique event ID before any business logic executes.

Idempotency keys prevent duplicate processing at the database level

Whether a specific webhook delivery succeeds once or retries ten times, designing for idempotency ensures that your system produces the same outcome.

Defining the idempotency key strategy

Selecting a stable idempotency key requires identifying a value that remains constant across every retry. In most integrations, this is the unique event identifier provided in the payload. Common examples include the id field from a Stripe event or the X-GitHub-Delivery header.

If a provider does not supply a unique ID, engineers must generate a deterministic key by hashing a combination of the event type, the primary entity ID, and the timestamp.

Database unique constraints for webhook deduplication

A final line of defense is provided by enforcing uniqueness at the database level. By creating a dedicated processed_webhooks table with a unique index on the idempotency_key column, the database will naturally reject any insert statement that attempts to reuse a key.

This architectural pattern forces the following sequence:

1. The application starts a database transaction.
2. It attempts to insert the incoming idempotency key into the tracking table.
3. If the insert fails due to a unique constraint violation, the application catches the error and returns a success code to the sender without re-running the business logic.
4. If the insert succeeds, the application processes the payload and commits the transaction.

Setting cache TTLs for webhook idempotency keys

Maintaining an infinite history of every webhook ID is rarely sustainable. If a service like Shopify retries failed webhooks over forty-eight hours, the idempotency keys must persist in a fast-access store, such as Redis, for at least that long.

Setting a Time-To-Live (TTL) on these keys ensures the tracking table does not grow indefinitely and degrade query performance.

Managing webhook redundancy with Activepieces automation

Activepieces syncs flows to git and promotes them through Release Management, allowing teams like MoneyGram to manage deduplication logic as versioned code rather than hidden UI configurations.

Automating the deduplication check

Deduplication in Activepieces relies on the Integration Storage service to verify whether a specific event ID has been processed.

The system queries its internal database for a unique identifier by using the "Get Value" action at the start of a flow. It checks for a key such as the x-github-delivery header so that any incoming request with a known key is immediately terminated.

If the key is absent, the "Put Value" action records the ID with an expiration time.

Handling high-frequency events without race conditions

The platform mitigates race conditions by utilizing atomic storage operations that prevent two concurrent executions from claiming the same event ID simultaneously.

When a high-volume source like Stripe sends rapid-fire notifications, the Activepieces storage backend ensures that the first execution to write the key succeeds. Subsequent attempts receive the existing value. This atomicity means the engineer does not have to implement complex locking mechanisms manually.

Standardizing webhook response codes across providers

Activepieces allows for the creation of uniform HTTP response codes regardless of the specific requirements of the service sending the webhook.

The execution engine for these flows is MIT-licensed code, meaning every step of the deduplication logic is visible in the public monorepo rather than being a black-box decision.

Every tool call and state change appears in the run trace, allowing developers to verify exactly why a duplicate was caught or how a retry was handled.

This decoupling ensures that providers do not falsely flag the endpoint as timed out and trigger unnecessary retries.

A checklist for hardening your webhook consumers

Hardening a webhook consumer requires shifting from a "fire and forget" mindset to a defensive architecture that assumes every event will be delivered at least twice.

• Verify the provider sends a unique event ID to distinguish retries from new actions.
• Ensure the listener returns a 2xx status code before initiating long-running tasks to prevent the sender from timing out and retrying.
• Implement a 7-day retention window for event ID logs to catch delayed redeliveries during provider outages.
• Set a concurrency limit on the consumer to prevent a sudden burst of events from exhausting database connection pools.

Hardening a webhook consumer requires shifting from a "fire and forget" mindset to a defensive architecture that assumes every event will be delivered at least twice.

Auditing webhook endpoints for duplicate record creation

Every endpoint that triggers a resource creation must be audited. If a listener for a payment processor like Stripe or a CRM like HubSpot lacks a check for existing external IDs, a single network hiccup will generate duplicate records.

You must verify that your code queries for the existence of the provider’s unique identifier before executing any POST or INSERT command. If the record exists, the consumer should return a success code and terminate the execution immediately.

Using a Redis cache to deduplicate webhooks

A high-speed cache acts as the first line of defense. Storing the hash of an incoming webhook payload in an in-memory store like Redis for a five-minute window allows the system to drop redundant retries that occur in rapid succession.

This short-lived buffer prevents race conditions where two identical requests are processed by different worker threads simultaneously, ensuring that data integrity is maintained during high-concurrency operations, which means the system avoids corrupted database states under heavy load.

Logging unique event IDs for forensic audits

Standard application logs are insufficient for troubleshooting webhook failures. You must explicitly index the unique event ID provided in the webhook header.

Mapping your internal transaction IDs to the provider's event IDs in a searchable log allows an engineer to prove exactly when a specific signal was received and how the system responded.

Frequently asked questions about webhook duplicates

Do all webhooks send duplicates?

Every major event provider, from the payment processor Stripe to the version control platform GitHub, guarantees at least-once delivery.

This necessitates that your system must treat duplicate payloads as a standard operating condition. Because these services prioritize delivery over uniqueness, a network timeout or a 500-level response from your server triggers an automatic retry.

Your server might process the initial request but fail to send a 200 OK response before the provider’s timeout threshold.

How do I test my system against duplicate events?

You test for idempotency by manually replaying the exact same signed payload against your endpoint twice in rapid succession. This ensures the second request returns a success code without altering your database state.

Relying on a provider's dashboard to "resend" a hook is insufficient for testing race conditions, as those manual triggers often carry new timestamps or unique delivery headers.

Does webhook ordering matter if I have duplicates?

Ordering is critical because a delayed duplicate of an older "Update" event can overwrite the data from a more recent "Completed" event, leading to a permanent state of data corruption.

Even if you successfully filter out identical IDs, webhooks often arrive out of sequence due to concurrent processing at the source.

Your code must compare the timestamp or version number within the payload against the current record in your database to maintain data integrity. The system should reject any incoming data that is chronologically older than what you already have stored.

Should I use a message queue to handle webhooks?

To decouple the ingestion of a webhook from its execution, a message queue like Amazon SQS or RabbitMQ is the standard architectural choice. This prevents a sudden spike in traffic from overwhelming your primary application database.

This separation allows your ingestion endpoint to acknowledge receipt immediately, which prevents the sender from timing out and triggering unnecessary retries.

However, moving the payload to a queue does not solve the duplication problem; it merely shifts the responsibility of deduplication to the worker process that eventually consumes the message.

Related reading

• How to handle errors in production AI agents
• Designing Webhook Payloads: A 2026 Guide
• Webhook Retry Logic: Best Practices for 2026

References

• ScaiLabs
• Stripe
• WebhookWhisper