Cron jobs fail silently
A cron job that fails is invisible by default. There is no exit status on your screen, no log unless you wrote one, and mail delivery on modern servers is usually disabled. Everything below is a variation of “it worked when I ran it by hand”.
1. The environment is not your shell
Cron starts a minimal environment: PATH is usually just /usr/bin:/bin, no aliases, no ~/.bashrc. A script that calls /usr/local/bin/restic works interactively and fails at 02:10.
PATH=/usr/local/bin:/usr/bin:/bin
SHELL=/bin/bash
MAILTO=ops@example.com
2. The shell is not the one you think
/bin/sh on Debian is dash. Arrays, [[ ]] and source do not exist there. Put the shell in the crontab or use a shebang and call the script directly.
3. Nothing checks the exit status
set -euo pipefail
exec 9>/var/lock/backup.lock
flock -n 9 || { echo "already running"; exit 0; }
Without flock, a slow backup overlaps the next one and both fail quietly. Without set -e, the script continues after the first error and exits zero.
4. Output goes to nobody
Cron mails stdout and stderr only if a mail transfer agent exists. Most containers have none. Write your own log with a timestamp and one result line, and check that line somewhere else:
exec >>/var/log/backup.log 2>&1
echo "$(date -Is) start"
restic backup /srv --quiet
echo "$(date -Is) done rc=$?"
5. Time zones and DST
Cron uses the system time zone unless CRON_TZ is set. A job at 02:30 disappears once a year in a DST switch, and it is always the job that matters.
The five-minute audit
For every crontab on the host: does the command exist in cron's PATH, is the shell right, is there a lock, does it write one line with a result, and where is that line checked? Any “I don't know” is a job that will fail silently this year.