Patterns, Technologies, and Concepts

• Source & Deep Dive

×

Key Takeaways

Requirements

You initially set a query latency target that was too slow for a leaderboard-style system, but corrected it on your second attempt and landed on a strong set of non-functional requirements.

• Top-K leaderboard queries should target sub-50ms latency, not 500ms. These are simple sorted reads from an in-memory data structure like a Redis sorted set, so they should feel near-instant to users. 500ms is more appropriate for complex database queries or cross-region calls.

• Count-Min Sketch is a common approximate counting algorithm used at massive scale (like YouTube) to track view counts. It trades a small margin of error for huge savings in memory and processing. In interviews, being open to approximations shows maturity, but if the system explicitly requires exactness, call that out as a deliberate trade-off.

Core Entities

You initially modeled views as a vague entity but quickly corrected course to use a ViewEvent with a timestamp, which is the right approach for time-based aggregations.

• Model user actions as discrete events with timestamps, not just counts. A ViewEvent entity stores who viewed what and when, which lets you aggregate views into any time window like the last hour or last day. A plain ‘Views’ count throws away the timestamp data you need for flexible time-based queries.

• When designing a Top K system, time windows are a first-class concern at the data model level. A TimeWindow entity or field lets you scope rankings to different horizons like hourly, daily, or all-time without reprocessing raw data from scratch each time.

API

You started by missing the k parameter entirely but iterated well across attempts, arriving at a complete and well-structured top-K API design by the end.

• A top-K leaderboard endpoint needs a k query parameter so the caller can specify how many results they want, for example GET /views/top-k?window=hour&k=100. Without it, the server has no way to know if the caller wants top 10 or top 1000.

• For top-K leaderboard APIs, return results as an array of objects with both the identifier and the count, like { videoId, viewCount }. The count is a first-class piece of data on a leaderboard, not just metadata, so it belongs explicitly in the response shape rather than hidden inside a partial object type.

• When designing an API with an enum-like parameter such as a time window, always enumerate the valid values explicitly in your design, for example window = ‘hour’ | ‘day’ | ‘month’ | ‘all-time’. This makes the contract clear and prevents ambiguity for API consumers.

• When K is bounded and small enough to fit in a single response (like up to 1000 items), you can skip cursor-based pagination and return all results at once. Cursor pagination is most useful when result sets are unbounded or very large.

High Level Design

You gave a solid end-to-end design with good instincts, but left some ambiguity around how aggregated counts are stored and how larger time windows are derived from smaller ones.

• When designing a view counter system, store one row per video per time bucket with a pre-aggregated count, not one row per raw event. For example, a table with columns video_id, hour_bucket, and view_count is far more efficient to query than scanning millions of raw events.

• Larger time windows like day, month, or all-time should be computed by rolling up smaller buckets. For example, a daily count is the sum of 24 hourly rows for the same video. This avoids storing redundant data while keeping queries fast and predictable.

• When you describe a data model in an interview, always be explicit about the granularity of a row. Saying ‘I store hourly data’ is ambiguous. Saying ‘each row represents one video for one hour and holds a running count’ removes all doubt and shows you have thought through the storage contract.

Deep Dives

You demonstrated strong streaming fundamentals throughout this section and improved quickly when given feedback, particularly around Kafka partitioning, exactly-once recovery, and approximate counting with Count Min Sketch.

• Partition Kafka topics by the entity you are aggregating (like videoId) so that events for the same video always land on the same partition and the same Flink task manager. This is what makes parallel in-memory aggregation correct and avoids cross-worker state sharing.

• Flink achieves exactly-once recovery by storing the Kafka read offset inside the same checkpoint as the in-memory aggregation state. The offset is not committed to Kafka until after the checkpoint succeeds, so on recovery you resume from exactly the right position without double counting.

• To prevent a cache stampede when a Redis key expires, use request coalescing so only one request goes to the database while all other concurrent requests wait for that single result. Once the result is back it gets written to Redis and all waiting requests are served from cache. You can also keep the old stale value alive with a short TTL extension so waiters get a fast response instead of blocking.

• A Count Min Sketch paired with a min-heap of size K is a practical way to maintain an approximate top K leaderboard inside Flink. The sketch gives you fast frequency estimates with low memory, and the min-heap lets you compare each new candidate against the current minimum and evict it in O(log K) time. The tradeoff is that you lose exact counts and may get false positives near the cutoff rank.