Building an Idempotent Webhook Receiver with PostgreSQL and Python

Webhooks are the connective tissue of modern web architecture. Whether you are processing payments through Stripe, syncing inventory from Shopify, or tracking deployments via GitHub, webhooks let external services notify your backend when something happens. However, distributed systems come with a fundamental reality: at-least-once delivery.

Because third-party providers prioritize delivery guarantees over single-delivery perfection, your server will eventually receive the same event twice. If your endpoint is not idempotent, a transient network timeout could lead to duplicate charges, multiple shipping labels, or corrupted state. Here is how to build an idempotent receiver using Python and PostgreSQL to guarantee event processing safety.

What Does Idempotency Mean in Webhooks?

An operation is idempotent if applying it multiple times produces the exact same system state as applying it once. In the context of a webhook receiver, if provider X sends event evt_12345 three times, your application should process the business logic exactly once and acknowledge receipt with a successful HTTP 200 response on all three attempts.

The Storage Architecture

The simplest and most resilient way to track processed webhooks is inside your primary relational database. By coupling event tracking with your database transactions, you eliminate race conditions that often plague in-memory caches like Redis.

A minimal PostgreSQL table schema looks like this:

CREATE TABLE webhook_events (
  id VARCHAR(255) PRIMARY KEY,
  provider VARCHAR(64) NOT NULL,
  status VARCHAR(32) NOT NULL DEFAULT 'processing',
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  processed_at TIMESTAMPTZ
);

The primary key constraint on id ensures that the database engine itself rejects duplicate entries at the disk level.

Implementing the Receiver Flow in Python

Using a modern asynchronous framework like FastAPI or standard Flask with an ORM/query builder, you can handle incoming payloads through atomic database locks or conditional inserts. The process follows a straightforward state machine:

• Extract the Identifier: Most providers supply a unique identifier in the payload or headers (e.g., Stripe's id field).
• Attempt Atomic Registration: Execute an INSERT ... ON CONFLICT DO NOTHING statement. This establishes whether your system has ever seen this event before.
• Handle Existing Records: If the insert returned zero rows, check the existing record. If it has already completed, immediately return an HTTP 200. If it is currently processing, return an HTTP 200 or 409 depending on whether you want the provider to back off.
• Execute Business Logic: If the record was inserted successfully, run the internal domain logic within a transactional boundary.
• Finalize State: Update the status column to completed and record the timestamp.

Key Considerations for Production

While database constraints protect your state, real-world webhook handling demands a few extra precautions:

• Verify Signatures First: Never write an event to your database before validating the cryptographic signature header provided by the vendor. This prevents denial-of-service attempts aimed at filling your database with junk records.
• Acknowledge Rapidly: Webhook senders typically enforce strict timeout thresholds (often 5 to 10 seconds). If your downstream logic involves heavy I/O or external APIs, commit the incoming event record, respond with HTTP 200, and offload the actual workload to a background task queue like Celery or RQ.
• Handle Failures Intentionally: If your application throws an unhandled exception before finishing the business logic, ensure you do not mark the record as completed. Either roll back the transaction or flag the record as failed so retries can be evaluated cleanly.

Conclusion

Idempotency is not an optimization; it is a prerequisite for reliable backend services. By shifting the deduplication burden to PostgreSQL's battle-tested transactional engine, you can build webhook consumers in Python that remain rock-solid regardless of network instability or aggressive provider retry policies.