Idempotency in payment APIs

The client sends POST /charges, your API charges the card, and the response is lost to a load balancer timeout. The client does not know whether the charge happened. If it retries, it can charge twice. If it does not retry, it can lose the sale. The only reasonable way out is to make retries safe, and that is what an idempotency key solves.

What idempotent means here

defIdempotencyaka idempotence

An operation is idempotent if running it once or many times leaves the system in the same state. In HTTP, PUT and DELETE are idempotent by definition. POST is not, and you have to build it.

The idempotency key is a unique value that the client generates, usually a UUID, and sends in the Idempotency-Key header. The server uses it to recognize that two requests are the same attempt. The first time, it runs the operation and stores the response. Later requests get the stored response without running the operation again.

Stripe popularized the pattern, and there is now an IETF draft that standardizes the header. This article follows the status codes that draft proposes.

The flow with a retry

The second request never reaches the processor. Note also that the API passes the same key to the processor. That matters later.

What the server stores

For each key, the store keeps four things.

• The key, scoped per client. Two different merchants can send the same UUID, and they must not collide.
• A fingerprint of the body, a SHA-256 hash. It detects that someone reused the key with a different request.
• The state, in-flight while the request runs and completed once there is a response.
• The response, status and body, so every retry gets the same answer.

export type StoredResponse = { status: number; body: unknown };

export type IdempotencyRecord =
  | { state: 'in-flight'; fingerprint: string }
  | { state: 'completed'; fingerprint: string; response: StoredResponse };

export type AcquireResult =
  | { acquired: true }
  | { acquired: false; existing: IdempotencyRecord };

export interface IdempotencyStore {
  acquire(key: string, fingerprint: string, lockTtlMs: number): Promise<AcquireResult>;
  complete(key: string, response: StoredResponse, retentionMs: number): Promise<void>;
  release(key: string): Promise<void>;
}

The fingerprint needs a canonical body

JSON.stringify keeps the insertion order of the keys. If a client sends {"amount":1500,"currency":"ARS"} and the retry sends the same keys in a different order, the hash changes. For a Node server that hashes the parsed body, this is usually fine, because the same client always serializes the same way. If you have heterogeneous clients, sort the keys before hashing.

The Express wrapper

I built it as a handler wrapper instead of a middleware that intercepts res.json. This way it stores the response before it sends it, and the handler’s return type is explicit.

import { createHash } from 'node:crypto';
import type { Request, RequestHandler } from 'express';
import type { IdempotencyStore, StoredResponse } from './idempotency-store.js';

const LOCK_TTL_MS = 30_000;
const RETENTION_MS = 24 * 60 * 60 * 1000;

type JsonHandler = (req: Request) => Promise<StoredResponse>;

export function idempotent(store: IdempotencyStore, handler: JsonHandler): RequestHandler {
  return async (req, res) => {
    const key = req.get('Idempotency-Key');
    if (!key) {
      res.status(400).json({ error: 'idempotency_key_required' });
      return;
    }
    const scopedKey = `${res.locals.merchantId}:${key}`;
    const fingerprint = createHash('sha256').update(JSON.stringify(req.body)).digest('hex');

    const lock = await store.acquire(scopedKey, fingerprint, LOCK_TTL_MS);
    if (!lock.acquired) {
      const existing = lock.existing;
      if (existing.fingerprint !== fingerprint) {
        res.status(422).json({ error: 'idempotency_key_reused' });
        return;
      }
      if (existing.state === 'in-flight') {
        res.status(409).set('Retry-After', '1').json({ error: 'request_in_progress' });
        return;
      }
      res.status(existing.response.status).set('Idempotent-Replayed', 'true').json(existing.response.body);
      return;
    }

    let response: StoredResponse;
    try {
      response = await handler(req);
    } catch (error) {
      await store.release(scopedKey);
      throw error;
    }
    await store.complete(scopedKey, response, RETENTION_MS);
    res.status(response.status).json(response.body);
  };
}

The route looks like this: app.post('/charges', authenticate, validateChargeBody, idempotent(store, createCharge)). Validation goes before the wrapper. A malformed body returns 400 without using up the key, and the client can fix the body and retry with the same key.

The concurrent part

Two retries can arrive at the same time, for example when the client has a short timeout and the server is still processing the first request. If acquire runs a SELECT and then an INSERT, both requests see that the key does not exist and both charge. Acquisition has to be a single atomic operation.

INSERT INTO idempotency_keys (key, fingerprint, state, locked_until, expires_at)
VALUES ($1, $2, 'in-flight', now() + interval '30 seconds', now() + interval '24 hours')
ON CONFLICT (key) DO UPDATE
  SET fingerprint = EXCLUDED.fingerprint,
      state = 'in-flight',
      response = NULL,
      locked_until = EXCLUDED.locked_until,
      expires_at = EXCLUDED.expires_at
  WHERE (idempotency_keys.state = 'in-flight' AND idempotency_keys.locked_until < now())
     OR idempotency_keys.expires_at < now()
RETURNING key;

If it returns a row, the request acquired the key. If it returns nothing, the key exists and is still valid, and the store reads it to build existing. The WHERE in DO UPDATE covers two recovery cases. An expired lock from a process that died can be taken again, and an expired key can be reused.

In Redis, the equivalent is SET key value NX PX 30000. It works the same way for the lock, but the response then lives in a different store from the one that holds the charge.

A curl session

$ curl -s -i -X POST localhost:3000/charges -H 'Idempotency-Key: 7f9c1e2a' -H 'Content-Type: application/json' -d '{"amount":1500,"currency":"ARS","source":"tok_visa"}'HTTP/1.1 201 CreatedContent-Type: application/json; charset=utf-8{"id":"ch_1","amount":1500,"status":"succeeded"}# same key and same body, the API returns the stored response$ curl -s -i -X POST localhost:3000/charges -H 'Idempotency-Key: 7f9c1e2a' -H 'Content-Type: application/json' -d '{"amount":1500,"currency":"ARS","source":"tok_visa"}'HTTP/1.1 201 CreatedIdempotent-Replayed: trueContent-Type: application/json; charset=utf-8{"id":"ch_1","amount":1500,"status":"succeeded"}# same key with a different amount$ curl -s -i -X POST localhost:3000/charges -H 'Idempotency-Key: 7f9c1e2a' -H 'Content-Type: application/json' -d '{"amount":2500,"currency":"ARS","source":"tok_visa"}'HTTP/1.1 422 Unprocessable EntityContent-Type: application/json; charset=utf-8{"error":"idempotency_key_reused"}

The second request returns the same charge id. For the client it is indistinguishable from the first, except for the Idempotent-Replayed header.

Status codes

One detail about what gets stored. If the handler returns a card decline as 402, that 402 is also stored and replayed. The decline is a result, and retrying with the same key must not change it. To try another card, the client sends a new key.

Where to store the keys

For a payments service I pick Postgres. Writing the charge and the response in the same transaction removes the window between running and storing. Redis wins when volume is high and the endpoint has no database of its own.

Summary

• The client generates the key and reuses it on every retry of the same attempt.
• The key is scoped per client and carries a fingerprint of the body to detect reuse.
• acquire is a single atomic operation. A SELECT followed by an INSERT is not enough.
• Store the response before you send it, and keep it for at least 24 hours.
• Pass the key to every downstream service that has effects. Without it, an expired lock opens the door to a double charge.