Design a Distributed Job Scheduler

Requirements

• •Allow registering, updating, and deleting cron jobs via API
• •Trigger jobs at their scheduled times with at-most-once or at-least-once semantics
• •Prevent duplicate execution when multiple scheduler instances are running
• •Track execution history and status (success, failure, running) per job
• •Support pausing and resuming individual jobs without deleting them

Non-Functional Requirements

• •Job trigger latency < 5 seconds from scheduled time
• •Support up to 100,000 registered cron jobs
• •System remains available if one scheduler instance dies
• •Handle bursts of 10,000 jobs triggering within the same minute

Constraints

• •Jobs are defined by cron expressions and an HTTP callback URL
• •Multiple scheduler instances run for high availability
• •Job execution time is not bounded — jobs may run for minutes or hours

Evaluation Criteria