Patterns, Technologies, and Concepts

• Source & Deep Dive

×

Key Takeaways

Requirements

You nailed the availability vs consistency trade-off right away and improved your latency requirement from 10ms to 5ms on your second attempt, showing good responsiveness to feedback.

• A rate limiter sits in the critical path of every single request, so its latency budget must stay under 5ms. At 10ms, you risk noticeably degrading user-facing response times at scale since every request pays that cost.

Core Entities

You demonstrated a strong grasp of the core entities in a rate limiter system, correctly identifying Rules, Clients, and Requests right away.

• The three core entities of a rate limiter are Rules (define the limits, e.g. 100 requests per minute per user), Clients (the identity being rate limited, e.g. user ID or API key), and Requests (the incoming traffic being counted and evaluated against the rules). Always start your rate limiter design by anchoring around these three concepts.

System Interface

You designed a clean and complete rate limiter function signature, with only a minor note about keeping the identifier inputs simple and intentional.

• When designing a rate limiter API, pick one primary requester identifier like clientId or ipAddress, not both, unless you have a specific reason to use them together. Using both without explanation adds confusion because they often represent the same thing, which is who is making the request. If you do need both, be ready to explain the difference, for example clientId identifies an authenticated user while ipAddress can catch unauthenticated abuse.

High Level Design

You demonstrated strong fundamentals across placement, algorithm selection, and client response design, with only minor gaps in naming specific parameters and tightening your explanations.

• Token bucket has two key knobs: bucket capacity and refill rate. Bucket capacity controls the maximum burst size allowed, while refill rate controls the sustained request rate over time. Knowing these two terms by name makes your algorithm justification much sharper in an interview.

• When rate limiting at the API gateway, the main tradeoff is that the gateway only sees request-level identity like client ID or IP. It lacks business context from the application layer, so very fine-grained or business-logic-specific limits are harder to enforce there compared to in-application rate limiting.

• For distributed rate limiting behind multiple backend instances, per-instance counters will not enforce a global limit because each node only tracks its own traffic. You need a shared state store like Redis to maintain a single global counter across all instances.

• When returning a rate limit rejection, always use HTTP 429 and include response headers like Retry-After and RateLimit headers on that same rejected response. This gives clients immediate recovery guidance and reduces wasteful retry storms.

Deep Dives

You showed strong progression throughout this section, quickly closing gaps when given feedback, though you initially struggled with two specific patterns: how to safely coordinate config rollouts across distributed gateways, and how to design a proper fallback allowlist during Redis outages.

• When rolling out config changes to distributed gateways, use a staged deploy-then-activate pattern. First push the new rule to all gateways without activating it, wait for each gateway to acknowledge receipt back to the config store, then send a single activation signal. This prevents split-brain enforcement where some gateways enforce the new rule and others still enforce the old one.

• A fallback allowlist for use during Redis outages needs three things to be useful: it should live in each gateway’s local in-memory cache, it should be periodically synced from Redis while Redis is healthy (every few seconds), and even clients on the allowlist should still be capped at a conservative rate tied to actual backend capacity. Without a cap, a compromised known-good client could still overwhelm your backend.

• For dynamic rate limiting rules that need to change at runtime without redeployments, use a push-based config store like ZooKeeper with gateway watchers. Gateways subscribe to change notifications and keep a local copy of rules so config lookups are never on the hot request path. If ZooKeeper goes down, gateways keep serving the last known good rules so the data plane stays up independently of the control plane.

• When discussing Redis atomicity for rate limiting, pick one approach and stick to it. Lua scripts are generally preferred over MULTI/EXEC because a Lua script runs entirely server-side in a single round trip, folding the read, refill calculation, decision, and write into one operation. This directly contributes to keeping per-request latency under your target.