This pack has not completed a verification run yet, so nothing is published here. When the harness runs it, this space becomes the measured record: pass rate as M of N clean-scaffold runs on a pinned model, per-run token cost and wall time, per-check results, locked dependency versions, and whether the scaffold was left pristine. No pass rate, token cost, or deploy is claimed until then.
Webhook Durability & Replay
complex-universaldraftThe webhook ingestion pipeline every provider assumes you built and none of them ship — idempotency, ordering, dead-letter, backfill and provably exactly-once effects across Stripe, Shopify, Apple, Google Play and Resend — as a receipt-backed build pack for your own coding agent.
How install works
- Accounts
- Tools
Tool install and prerequisite check: ~15 minutes (less if you already
Full setup walkthrough ships in the zip after purchase.
Every provider you take money through assumes you built a durable webhook
ingestion pipeline, and none of them ship one — Shopify retries 8 times over 4
hours, then drops the event permanently, offers no replay API, and tells you in
its own docs to build reconciliation jobs instead. This pack builds the layer
underneath your handlers for Stripe, Shopify, Apple, Google Play and Resend: two
distinct dedupe keys doing two different jobs, a per-resource ordering
watermark, a real dead-letter, and backfill for the drops each provider
documents — with the acceptance suite and a chaos simulator already written.
**Status: draft. There is no published receipt yet**, so the only proof
currently on offer is one you run yourself: `verify.sh` drops, duplicates,
reorders and delays a multi-provider event history against a real Postgres, then
recovers from a simulated three-day outage, and asserts the final state matches
a clean-delivery oracle exactly.
Example use cases
- **An indie SaaS on Stripe that has been double-crediting accounts and can't
reproduce it.** Stripe documents that it doesn't guarantee event order and
that endpoints may receive the same event more than once. The pack's effect
ledger makes the second delivery free and the reordered one harmless, and the
chaos test is where you find out whether your version actually does that.
- **A Shopify app developer who has been quietly losing orders.** Shopify's
retry budget is 4 hours and there is no replay API; developers have publicly
reported missing `orders/create` webhooks and 20-minute-plus `orders/paid`
delays. This pack ships the reconciliation job Shopify's own best-practices
page tells you to write, keyed so it can never double-count an order you
already received.
- **A mobile team shipping on both stores.** Apple App Store Server
Notifications and Google Play RTDN have completely different failure shapes —
Apple retries 5 times over ~3 days and never retries at all in the sandbox;
Play arrives through Pub/Sub, unordered, carrying a signal rather than state.
One pipeline, one ledger, one dead-letter for both, with the per-provider
differences pinned in a table instead of discovered in production.
- **A team that already bought one of the other packs in this catalogue.** The
billing, email and App Store packs each ship a correct handler for their own
domain. This is the ingestion layer under all of them — see "How it composes"
below, including what it deliberately does not do for you.
- **Anyone who has ever written "TODO: make this idempotent" above a webhook
route.** That TODO is four separate mechanisms, and the pack's whole argument
is that collapsing them into one is the bug.
Why this pack
No "always" claims — agents are stochastic, and every ingestion design trades
something. Here is what the pack pre-solves, what it costs, and what you give
up.
| Bare agent, no pack | Hosted webhook gateway (Hookdeck / Svix-class) | Durable-execution platform (Inngest / Trigger.dev-class) | Hand-rolled in-house | This pack | |
|---|---|---|---|---|---|
| **Cost** | "Free" (agent time only) | Recurring subscription, priced by event volume — check their current pricing | Recurring subscription, priced by run volume | Engineering time — realistically weeks, because the bugs surface as incidents | $149 one-time |
| **What it actually solves** | A handler | **Delivery** durability: getting the event to you, retrying, replaying, a hosted DLQ. Genuinely good at this | Execution durability: retries and step-level resumption around your own code | Whatever you got to before shipping | **Effect** durability: what happens in *your* database when the same fact arrives twice, out of order, or from a different path entirely |
| **Still yours afterwards** | everything | the effect key, the ordering watermark, the reconciliation job for a provider with no replay | the dedupe key, the ordering model, the backfill | — | the domain meaning of your events (deliberately — see non-goals) |
| **Correctness risk** | High and silent. The failure mode is a number that's wrong by one, months later | Low on delivery; **unchanged** on double-application inside your DB | Low on execution; unchanged on cross-path dedupe | High until you've independently rediscovered the bug classes below | Bounded by 17 checks including a seeded chaos run and an outage simulation — **and currently unproven by a receipt** |
| **Lock-in** | none | Real: your ingestion path runs through their infrastructure and their event ids | Real: your handlers are written to their runtime | none | none — plain Postgres and TypeScript in your repo, no runtime call home |
**What a bare agent typically ships that this pack pre-solves** — specific bug
classes, not a claim that every agent hits every one:
- *It writes a handler, not a pipeline.* No dead-letter, so a poisoned event
retries forever or vanishes into a log line. No replay, so recovery means a
manual SQL session. No backfill, so a provider's **documented** permanent drop
is simply data you never had. Shopify is the case that makes this
non-theoretical: 8 retries, 4 hours, no replay API, and Shopify's own
guidance is "your app shouldn't rely on receiving data from Shopify
webhooks."
- *It dedupes on the wrong key.* This catalogue's own `email-deliverability`
pack documents exactly this bug: Resend's `data.email_id` identifies the
*email*, so it is shared across that message's `delivered`, `bounced` and
`complained` events, and deduping on it silently collapses three distinct
events into one. Resend's docs name the `svix-id` header as the dedupe key.
Shopify ships the same trap in a different shape — `X-Shopify-Event-Id` is
documented as "shared across all deliveries produced by the same merchant
action", while `X-Shopify-Webhook-Id` is the per-delivery key. Picking the
one with "event" in the name is the natural mistake.
- *It uses one key for two jobs.* Even a correct delivery key can't stop a
double-application that arrives by a *different path* — a reconciliation job
re-importing an order you already processed. The effect key has to be derived
from the business fact, not the HTTP delivery, or backfill becomes the thing
that corrupts your data.
- *It wraps the whole transaction in an idempotency catch.* The
`saas-billing-tax` pack documents this trap on the billing side: a
transaction-wide catch swallows an unrelated constraint violation and reports
"processed" while the database diverges. In an ingestion pipeline it is
strictly worse — burning the idempotency key means the retry is deduped away
and the effect never happens at all. And in raw Postgres it needs a
`SAVEPOINT`, without which the transaction is already aborted by the time you
catch anything.
- *It skips stale events uniformly.* An ordering watermark is correct for
snapshot events and catastrophic for accumulating ones: refuse a late invoice
as "stale" and that payment is never counted, ever. The two need different
treatment and the difference has to be declared, not inferred.
Scale envelope
Derived from the pinned stack (Next.js 16 route handlers, `pg` 8.22, Postgres
17.10, a `SKIP LOCKED` worker), not from a benchmark. **Nothing here is
load-tested**; the acceptance suite proves correctness, not throughput, and it
says so in ACCEPTANCE.md. Treat every figure as an order-of-magnitude estimate.
- **Ingest is deliberately cheap:** one HMAC or JWS verification plus one
`INSERT … ON CONFLICT DO NOTHING`. On a small managed instance that is
comfortably inside Shopify's 5-second total / 1-second connect budget — the
tightest published response limit of the five providers, and the one the
design targets. The realistic bound at this tier is the web service's request
concurrency, not Postgres.
- **Processing scales by adding worker processes**, with no queue broker, no
lease table and no Redis: `FOR UPDATE SKIP LOCKED` is the entire scheduler.
Workers need no coordination because snapshot handlers are watermark-guarded
and accumulate handlers are commutative — that is a property of the design,
not a tuning parameter.
- **The `enrich` step is the per-event cost that matters.** Google Play
notifications require a `purchases.subscriptionsv2.get` call before any key is
derived, so Play throughput is bounded by that API, not by your database. The
other four providers need no outbound call at all.
- **First ceiling: table growth, not throughput.** `webhook_deliveries` is
append-only and the pack ships **no retention policy on purpose** — the right
retention is "at least as long as your longest recovery lookback", which
depends on which providers you use (Apple's history goes back 180 days,
Stripe's 30, Pub/Sub's 7 by default). At roughly 1–2 KB of JSONB per row, a
million deliveries a month is on the order of a gigabyte or two a month.
Deciding your own `delete from webhook_deliveries where status = 'done' and
received_at < now() - interval '<N> days'` is a one-line cron, and it is a
decision the pack refuses to make for you rather than one it forgot.
- **Second ceiling: a single hot resource.** Effects on one `resource_key`
serialize on that row's lock. Thousands of events per second spread across
many resources is a different problem from thousands against one — the second
is the one that needs a redesign, and the redesign is partitioning the
resource, not replacing the pipeline.
- **Upgrade path:** more workers first (free, no code change); then a retention
cron; then Postgres vertical scale; and only after all three does replacing
the `SKIP LOCKED` loop with a broker become the right move. The worker loop is
the only thing you'd replace — the keys, the ledger and the ordering rules are
unaffected.
**What it comfortably covers as shipped:** a single product taking webhooks from
one to five of these providers at the volume a small-to-mid SaaS, storefront or
mobile app generates — thousands to low millions of events a month. **What it
isn't sized for out of the box:** multi-tenant ingestion where each tenant has
its own provider credentials (the config is per-app, not per-tenant), or a
single resource under sustained high-frequency contention.
**What it costs to skip the pack (modelled, not measured):** building this layer
the way most people actually work — one agent on a top-tier model, no cheap-tier
routing, no parallel subagents, no prompt-caching discipline — models out at
roughly **$100–400 in API tokens**, driven mostly by an exploration factor: five
signature schemes and five sets of retry semantics are a lot of documentation to
read badly. That range is this pack's own *estimated* build volume repriced at
Opus-class rates, and it will be replaced with a measured figure when the receipt
lands. The honest reading: buy this for the bug classes, not the token
arithmetic. The bugs here are the silent, off-by-one, discovered-in-a-month kind.
Target models
- claude-opus-5
- claude-sonnet-5
- claude-haiku-4-5
Supported stacks
- nextjs-postgres-worker
node 24.18.1 next 16.2.12 react 19.2.8 react-dom 19.2.8 pg 8.22.0 @types/pg 8.20.3 jose 6.2.7 @apple/app-store-server-library 3.1.0 svix 1.99.1 typescript 6.0.3 vitest 4.1.10 tailwindcss 4.3.3 @tailwindcss/postcss 4.3.3 @types/node 24.3.0 @types/react 19.2.2 @types/react-dom 19.2.2 postgres 17.10 stripe-api-version 2026-06-24.dahlia shopify-admin-api-version 2026-07 apple-notifications-version 2.0 play-rtdn-version 1.0
Estimated buyer token cost
820,000 tokens for one build pass (estimate, not a guarantee).
FAQ
What if the build fails?
`verify.sh` is the arbiter and it's deterministic — no network, no live provider credentials, locally signed payloads for all five schemes, and a pinned `postgres:17.10-alpine` container it starts and removes. Full refund within 14 days of purchase if `verify.sh` fails on a supported configuration (the pinned stack and a model from the pack's matrix) — reply to your Stripe receipt email with the verify output or the `FAILURE-REPORT.md` your agent is instructed to write. If your agent modified a frozen file (`test/`, `migrations/`, `verify.sh` or the pinned configs) to make a check pass, that falls outside the guarantee — the fix is to fix the implementation, not the test, per the pack's own verify prompt.
Why is there no receipt yet, and what does that change?
This pack is `status: draft`: no clean-room harness run has been recorded, so there is no published pass rate, no measured wall time and no measured token cost. Everything numeric in this listing is labelled as an estimate derived from the architecture and the pins. What does *not* change is the arbiter — the acceptance suite ships in the pack and you can run it on your own machine before you trust a word here. When the receipt lands it will publish M of N passing runs, per-run token volume and measured cost, and any of these estimates it contradicts should be read as wrong.
Isn't this what Hookdeck or Svix already do?
Partly, and where they overlap they're good at it. A hosted gateway solves **delivery** durability — receiving, retrying, replaying, a hosted dead-letter — and if that's your problem, a gateway is a reasonable buy. What it can't solve is what happens inside *your* database when the same business fact arrives twice, or out of order, or through a completely different path: the effect key, the per-resource watermark, and the reconciliation job for a provider that offers no replay at all. Those live in your code either way. The two also compose fine — a gateway in front of this pipeline is a coherent architecture, not a contradiction.
Do I need all five providers?
No. Each provider is one adapter file plus its handlers; the ones you don't configure simply have no secret and fail closed. Building all five is still worth it even if you use one, because the acceptance suite grades all five and the cross-provider comparison is where the design decisions become obvious — but nothing forces you to register endpoints you don't need, and `GO-LIVE-CHECKLIST.md` explicitly tells you not to.
What models does this need?
Tasks are pre-tagged `cheap`/`standard`/`capable` in `TASKS.md` and routed accordingly. Only two tasks are Opus-tier — T5, the forty-line effect ledger the whole guarantee rests on, and T9, the worker — plus the final verification pass. Everything else is Sonnet or Haiku. The PRD's estimate is **3 agent sessions** inside a $20/mo plan, whose binding constraint is prompt volume in Claude's 5-hour rolling window and weekly cap rather than raw tokens (Anthropic doesn't publish an exact token-per-window figure, so treat the session count as an expectation). Plan on **~820k fresh tokens** for one build pass — extrapolated from a sibling pack's measured figure, not measured here.
Can I customize it after the build?
That's the intended path. The demo projection (`{status, plan, credits}`) exists only to make double-application visible to the tests; replacing it with your own model is the first thing you should do. What you keep is the handler shape — a declared `snapshot`/`accumulate` mode, an effect key derived from the resource, and a pure `apply` — because those three are what the guarantees are built on. Adding a sixth provider is one adapter file, one entry in the registry, and its handlers.
Why does verification need Docker, when your other packs don't?
Because this pack's claims are about concurrency and an in-process database can't test them. Competing workers claiming rows with `FOR UPDATE SKIP LOCKED`, two transactions racing on one effect key, a `SAVEPOINT` rolling back under a genuine unique violation — all of them need genuinely concurrent connections. Proving exactly-once effects on a single-connection database would prove nothing, so `verify.sh` pins `postgres:17.10-alpine`. If you'd rather not run Docker, export a `DATABASE_URL` to any Postgres 15+ and the container step is skipped entirely.
Do I own the code, and what happens when a provider changes something?
Yes — once built it's your repo: plain TypeScript, plain SQL, four Postgres tables, no runtime call back to this pack or to OneShot. On provider change: this is a one-time purchase with free updates and **no timing promise** — it is not a subscription and not a maintenance service. `MIGRATION.md` ships in the pack and exists so your own agent can do the upgrade: it names every moving part, links the upstream release notes for each of the five providers, and carries a provider-scoped migration prompt that re-runs the full acceptance suite. That suite is the entire argument for self-service migration — it turns "I changed the Apple adapter" into a verified statement. Our own commitment, stated the same way in every pack: *packs listed for sale are re-verified against current platform rules on our schedule (quarterly, plus whenever an authority announces a mandate); every re-verified version is free to prior buyers; we do not promise a delivery date, and we delist rather than sell a pack we can no longer verify.*
Prescribed services
- Railway — Railway referral credits
Disclosure: some links on this page are affiliate links. We may earn a commission if you sign up through them, at no extra cost to you. We only link to services the pack actually verified against.