A cron job can exit successfully and still fail the workflow

A scheduled process can exit with code zero while the business task is still incomplete. A backup script may finish before its upload is durable. An import may return successfully after skipping every malformed row. A billing job may write invoices but fail before notifying customers. Process success and workflow success are different states.

That distinction is why log collection alone is a weak safety net for quiet background work. Logs help explain a failure after someone knows to look. A completion heartbeat answers the earlier question: did the expected work finish within the expected window?

Send the heartbeat after the critical work

Put the heartbeat at the end of the business-critical path. If a script pings before the backup reaches remote storage, the monitor confirms only that the script started. The strongest signal is a ping emitted after verification: the object exists, the export contains accepted rows, or the downstream API acknowledged the operation.

A heartbeat should prove the outcome you care about, not merely that a process woke up.

The integration can stay deliberately small. A successful job calls a unique HTTPS endpoint with curl or an HTTP client. The monitor records the time and compares it with the expected interval. This loose coupling works across shell scripts, containers, queues, serverless functions, and old applications that are difficult to instrument.

Use an interval and a grace period

A daily job rarely starts at the exact same second. Queue delays, database locks, larger inputs, deployments, and daylight-saving changes can shift completion. Treating every late minute as an incident creates alert fatigue. Model two values instead: the expected run interval and a grace period that reflects normal variance.

The grace period should come from observed duration, not optimism. If a job normally completes in 12 minutes and occasionally needs 20, a 30-minute grace period may be reasonable. If the underlying business deadline is stricter, the workflow itself needs more capacity rather than a quieter monitor.

Open one incident and close it on recovery

A missed heartbeat should open a single incident instead of sending the same notification every polling cycle. When a new heartbeat arrives, close that incident and send a recovery message. The pair gives operators a useful duration and prevents an inbox full of duplicate symptoms.

Keep the check idempotent. Multiple scheduler workers must not create multiple incidents for the same missed window, and concurrent heartbeats must not corrupt the last-seen state. A small SQLite deployment can handle this pattern when updates are transactional and the scheduler has a single clear owner.

Keep the alert path outside the failed component

A monitor running on the same machine as the job cannot report a full host failure. Self-hosting gives control over data and deployment, but it also makes failure-domain design explicit. For important work, place the monitor on another host and configure more than one notification path, such as email plus Telegram or a generic webhook.

The monitor itself also needs observation. An external uptime check for its public endpoint is inexpensive and closes the most obvious blind spot. For higher-value workflows, record the last successful notification delivery and test the alert channel periodically.

Design the recovery procedure before the alert

An alert is useful only when the recipient knows what to do next. Include the job name, expected schedule, last heartbeat, incident start time, and a link to a short runbook. The runbook should answer whether rerunning is safe, how to detect partial work, and who owns the downstream system.

Start with one job whose silent failure already has a real cost. Measure normal completion time for a week, choose a defensible grace period, and run a controlled failure. If the alert arrives but the recovery steps remain ambiguous, the monitoring is not finished.

A working reference implementation

I build SourceBento and sell the complete source code for a small self-hosted Cron Monitor implementing this pattern with TypeScript, Express, SQLite, Docker Compose, missed-run incidents, recovery alerts, and email, Telegram, or webhook notifications. The public demo is available at https://sourcebento.com/demo/cron/ and does not require registration. This disclosure is included because the operational guidance above should be useful whether or not the product fits your stack.