Patterns, Technologies, and Concepts

• Source & Deep Dive

×

Key Takeaways

Requirements

You demonstrated strong instincts in the requirements gathering phase, correctly prioritizing availability over consistency and grounding your design in concrete numbers like latency targets and scale estimates.

• For a distributed cache, high availability beats strong consistency. A cache miss is acceptable, but a cache being down forces all traffic to hit your database, which can cause cascading failures. Always call out this trade-off explicitly.

• 10ms is a solid target latency for cache get and set operations. In-memory stores like Redis typically respond in under 1ms locally, so 10ms accounts for network overhead and gives you a realistic SLA to design around.

• Anchor your cache design with three key capacity numbers: total data size (e.g. 1TB tells you how many nodes you need), RPS (e.g. 100k tells you throughput requirements), and latency target (e.g. 10ms tells you you need in-memory storage, not disk).

Core Entities

You correctly identified the core entities of a distributed cache on your first attempt, showing a solid foundational understanding of how caches are structured.

• A distributed cache stores data as key-value pairs. The key is a unique identifier (like a user ID or URL) and the value is the cached data (like a serialized object or string). This simple structure is what makes cache lookups fast, typically O(1) time.

API

You initially missed the TTL field in your cache write API but quickly corrected it and finished with a clean, complete design by your third attempt.

• A cache write API (POST/PUT) should always include an optional ttl field in the request body, for example { "value": "v1", "ttl": 3600 }. TTL stands for time-to-live and tells the cache how many seconds to keep the entry before automatically deleting it. Without it, users have no way to control expiration.

• LRU (Least Recently Used) eviction is an internal cache policy, not a user-facing API endpoint. You do not need to expose it as a REST endpoint. Instead, mention it as system behavior that happens automatically when the cache reaches its memory limit.

High Level Design

You demonstrated solid instincts across all four responses and showed good progression in refining your answers, with the main gaps being around concurrency handling and making your whiteboard diagrams match your verbal explanations.

• When designing a cache with a shared hashmap, always address concurrent access explicitly. A simple fix is using a thread-safe map or wrapping mutations in a lock. Without this, multiple simultaneous writes can corrupt the data structure.

• For TTL support, store an expiration timestamp (not the raw TTL duration) alongside each value in the hashmap. On every get, compare the stored timestamp to the current time. If expired, delete the key immediately and return null. Storing a timestamp keeps reads simple because it is always just one comparison.

• A background cleanup job complements lazy expiration for TTL caches. Lazy expiration only cleans up keys when they are read, so keys that are never accessed again will sit in memory forever. A periodic job scans for and removes those dead entries to keep memory usage from growing unbounded.

• LRU cache eviction uses a hashmap plus a doubly linked list together. The hashmap gives O(1) key lookup, and the linked list tracks recency order. On every get or set, move the accessed node to the head. On eviction, remove the node at the tail. All four operations, get, set, delete, and evict, should run in O(1) time.

Deep Dives

You demonstrated strong progression across multiple attempts, starting with a solid replication foundation and refining it to include replica catch-up mechanics, consistent hashing for sharding, and live migration strategies.

• When a replica reconnects after lag or downtime, it uses its last acknowledged offset to request only the missing entries from the replication log on the leader. This is called offset-based replay and is how systems like Kafka and Redis handle replica recovery. Without this, a promoted replica could be missing recent writes and serve stale or incorrect data.

• If a replica has fallen so far behind that the leader’s replication log no longer contains the replica’s last offset, offset-based replay is not enough. The replica must first do a full snapshot sync from the leader to get a complete copy of the data, and then resume offset-based replay from that point forward.

• Consistent hashing with virtual nodes means that when you add a new node group, only the keys in the affected hash range need to move. This is much better than modulo-based hashing where adding a node reshuffles nearly all keys. Virtual nodes also help distribute load more evenly across node groups.

• During a live migration to a new node group, use a dual-write and dual-read strategy to keep requests available throughout the transition. Once the new node group confirms it has received all migrated keys, you update the hash ring to point exclusively to the new group and stop the dual-write logic. This is how you safely cut over without dropping traffic.