Design Job Scheduler

Patterns, Technologies, and Concepts

Job Scheduler

Key Takeaways

Requirements

You demonstrated strong systems thinking in this section, correctly identifying the key requirements for a job scheduler including high availability, at-least-once execution, and realistic performance targets.

  • For a job scheduler, at-least-once execution is the right durability guarantee. This means a job may run more than once during failures, which is acceptable because missing a job entirely is worse than running it twice. Design your jobs to be idempotent so duplicate runs do not cause problems.

  • In distributed systems, high availability and strong consistency often conflict. For a job scheduler, you can relax consistency, for example when two nodes briefly disagree on job state, as long as jobs eventually run. This trade-off is common in systems where uptime matters more than perfect accuracy at every moment.

Core Entities

You did well identifying the core entities for a job scheduler, showing strong intuition about the difference between a task definition and a running job instance.

  • When designing a scheduler, a Job entity can often hold scheduling information directly, such as a cron expression or interval, rather than needing a separate Schedule entity. A separate Schedule entity is worth adding only if multiple jobs share the same schedule or if schedules need to be reused and managed independently.

API

You designed a clean and well-structured API for this section, with only minor gaps around pagination and auth token patterns for external APIs.

  • For external-facing APIs, never trust a client-supplied user_id as a query parameter. Instead, derive user identity from an auth token in the request header, such as a JWT Bearer token. This prevents users from impersonating others by simply changing the ID in the request.

  • When a GET endpoint can return a large number of results, always include explicit pagination parameters. A cursor-based approach uses a cursor and limit field, where the cursor points to the last seen item. This is preferred over page-based pagination for large or frequently updated datasets because it avoids skipping or duplicating records as data changes.

High Level Design

You demonstrated a solid grasp of the core scheduler architecture across both attempts, with only minor gaps around how to model and query data for specific access patterns.

  • When storing cron or scheduled jobs in DynamoDB, always include a next_run_time field as a queryable attribute. A raw schedule string like 0 9 * * * cannot be queried efficiently. Your watcher process needs to ask “which jobs are due right now?”, so you need a concrete timestamp it can filter or sort on.

  • In a job scheduler, the executions table should store all execution states including pending and scheduled, not just completed runs. This means you write an execution record with status scheduled when a job is first created, then update it to running and finally success or failed. This single table then powers the full monitoring view without needing to join multiple sources.

  • When designing a read or monitoring API, be explicit about which table is the primary source. For a job scheduler, job definitions and execution history are separate tables. The monitoring endpoint should read from the executions table first, and only optionally join job metadata if the client needs extra details like the job name or schedule string.

Deep Dives

You demonstrated strong progression across this section, quickly picking up on scheduling and retry patterns, with your main early gap being the lack of a concrete auto-scaling execution layer to back up your horizontal scaling claims.

  • When a job is created with a near-term scheduled time that falls inside the current scan window, the Schedule Service should push it directly to SQS with a delivery delay set to the exact scheduled timestamp. This bypasses the periodic watcher scan entirely and ensures the job is not missed or delayed by the next polling cycle.

  • Saying “add more workers” is not enough for high concurrency systems. The strong answer is to run workers as containers in an auto-scaling group like ECS, with scaling triggered by SQS queue depth. This turns burst handling from a manual operation into an automatic response, which is what makes 10,000 concurrent executions realistic.

  • SQS visibility timeout is your safety net for worker crashes. When a worker dies mid-job, the message becomes visible again after the timeout expires and another worker can pick it up. Set the visibility timeout relative to expected job duration, and extend it for long-running jobs to avoid duplicate execution while a healthy worker is still processing.

  • Use job type metadata tagged at scheduling time to control intra-container concurrency. CPU-bound jobs should default to one job per container to avoid resource contention. I/O-bound jobs can safely run with higher thread concurrency inside one container because threads waiting on network or disk do not consume CPU, improving utilization without noisy-neighbor risk.