Don't trust webhooks

15 July 2024

Webhook requests often contain rich data beyond the fact that some event has occurred.

It’s tempting to use this data to update your own representation of the third party data.

@app.route("/injest-new-order-webhook")
def ingest_new_order_webhook():
	webhook_orders = request.json()["new_orders"]
	for webhook_order in webhook_orders:
		models.Order(**webhook_order).save()

But beware:

• Any downtime or bug on your end becomes a data integrity risk by missing your chance to process the webhook (and its retries).
• You have to authenticate the sender of the webhook and verify its contents. This isn’t hard in theory but is often forgotten in practice.
• Do you trust your webhook sender to
  • retry webhooks in the case of errors?
  • retry webhooks over a long enough time period for the cause of error to be resolved?
  • never send duplicate webhooks?

Instead, treat webhooks as simple notifications that trigger a resync via a normal API. Also run this resync as a regular job to automatically recover from issues.

@app.route("/injest-new-order-webhook")
def ingest_new_order_webhook():
	fetch_new_orders.enqueue()


@cron(every=ORDER_FETCH_INTERVAL)
def fetch_new_orders(since=ORDER_FETCH_INTERVAL * 2):
	api_orders = requests.get(
		ORDERS_API_ENDPOINT,
		params={"since": since}
	).json()["new_orders"]
	for api_order in api_orders:
		order = models.Order.from_dict(api_order)
		if not order.exists():
			order.save()