Rate Limiting With Nothing but Redis

This post started as a note to my future self. Every few months I relearn the same lessons about rate limiting, so this time I'm writing them down.

When we finally measured, the numbers were worse than anyone guessed. Around 41% of the time spent on a typical change went into working around rate limiting rather than building anything.

Some background

The setup was ordinary: Redis at the edges, Node.js doing most of the heavy lifting, and a handful of scripts nobody wanted to own. It worked well until usage roughly doubled over a single quarter.

We had good intentions. We even had a document describing how rate limiting should work. The trouble was that the document and the code had quietly gone their separate ways.

What we changed

We started by writing down the one rule we actually cared about, then deleted every bit of code that didn't serve it. It's not glamorous, but it works.

Stripped of the project-specific bits, it looks like this:

UPDATE jobs
SET status = 'running', started_at = now()
WHERE id = (
  SELECT id FROM jobs
  WHERE status = 'queued'
  ORDER BY created_at
  FOR UPDATE SKIP LOCKED
  LIMIT 1
)
RETURNING id, payload;

This alone let us delete a surprising amount of code elsewhere in the project.

A few things that helped along the way:

Pair on the first few changes so the knowledge doesn't live in one head.
Make the default the safe choice, not just the documented one.
Write the migration guide before writing the migration.
Delete dead code aggressively; it's the cheapest refactor there is.

Trade-offs

We still haven't solved schema migrations. That's a post for another day, and honestly, I'm not sure we'll get it right the first time either.

There are cases where this falls apart. If your team is very small or your traffic is very spiky, the extra moving parts may not pay for themselves.

Takeaways

If you take one thing away from this, let it be this: rate limiting is a design decision, not a cleanup task. Make it early and make it on purpose.