Rate limiting for the public API

Our public API has no rate limiting. One misbehaving client can degrade the service for everyone, and we have no way to offer usage tiers. Per-key limits at the gateway, with a fixed monthly quota and a burst allowance, close both gaps.

• api-gateway (Node 22, Fastify) terminates all public traffic.
• API keys live in Postgres (api_keys table) and are validated by key-auth middleware.
• No request accounting exists anywhere in the stack.
• p99 latency budget at the gateway is 12 ms; any solution must stay inside it.

Only Redis gives replica-wide accuracy within the gateway's latency budget.

The in-process bucket fails because the gateway runs six replicas and clients would get six independent budgets. The vendor option solves problems we do not have yet.

Redis-backed sliding window, one key per API key per window.

// rate-limit.ts
export const checkLimit = async ({
  redis,
  apiKey,
  limit,
  windowSeconds,
}: CheckLimitArgs): Promise<LimitResult> => {
  const now = Date.now();
  const windowStart = now - windowSeconds * 1_000;
  const key = `rl:${apiKey}`;

  const [, , count] = await redis
    .multi()
    .zremrangebyscore(key, 0, windowStart)
    .zadd(key, now, `${now}:${crypto.randomUUID()}`)
    .zcard(key)
    .expire(key, windowSeconds)
    .exec();

  return { allowed: count <= limit, remaining: Math.max(0, limit - count) };
};

Limits are read from the existing api_keys row so plans can differ per customer:

ALTER TABLE api_keys
  ADD COLUMN rate_limit_per_minute integer NOT NULL DEFAULT 600,
  ADD COLUMN rate_limit_burst integer NOT NULL DEFAULT 100;

The gateway change is small; the middleware slots in right after key auth:

Rejected requests receive 429 Too Many Requests with Retry-After and the standard RateLimit-* headers. Clients can also inspect their budget directly:

{
  "limit": 600,
  "remaining": 483,
  "resetSeconds": 21,
  "burst": 100
}

1. Ship the middleware dark: measure, log, never reject.
2. Watch one week of production traffic; tune default limits so fewer than 0.1% of legitimate requests would be rejected.
3. Enable enforcement for internal keys only; verify alerts and dashboards.
4. Enable enforcement globally behind the rate-limiting feature flag.
5. Remove the flag after two quiet weeks.

The design deliberately favors availability when its shared dependency fails.

• Clock skew between replicas. The window uses Redis server time via TIME, not gateway clocks.
• Hot keys for very large customers. Shard the sorted set by minute bucket if any single key exceeds 50k requests per minute; not built now, documented as the known upgrade path.

• Unit tests for checkLimit covering boundary counts, expiry, and the fail-open path.
• An integration test that runs 1,000 concurrent requests against two gateway replicas and asserts a single shared budget.
• A load test proving the middleware adds less than 1.5 ms at p99.

1. Should unauthenticated endpoints (/health, /docs) share a per-IP limit, or stay unlimited?
2. Do we owe existing enterprise customers notice before enforcement, and how much?
3. Is 429 plus Retry-After enough, or do we also want a usage dashboard in this iteration?