Bash trap EXIT: Why Cleanup Runs but the Exit Code Is 0

Our auto-publish script stages the build output in a temporary directory, uploads it to Cloudflare Pages, and cleans up on the last line. The last line is the part that never runs. When a deploy takes longer than the CI job allows, the runner kills the script and the directory stays behind, along with whatever wrangler was still writing into it.

So I added the one-line fix bash gives you for this, directly under the mktemp:

work=$(mktemp -d)
trap 'rm -rf "$work"' EXIT

Disk usage on the runner went back to normal, which felt like the end of it. Then a deploy was killed, the trap did its job, and there was nothing in the logs to say the run had gone badly.

The trap runs on a SIGTERM, and $? is 0

Reduce it to the part that matters:

#!/bin/bash
cleanup() { echo "trap saw $?"; }
trap cleanup EXIT
sleep 60

Now send it SIGTERM, which is what a CI runner or systemd sends when it has decided you are done:

$ ./sleepy.sh & pid=$!; sleep 1; kill -TERM $pid; wait $pid; echo "process exited $?"
trap saw 0
process exited 143

The handler fired without me ever trapping TERM, which is worth knowing on its own. The number it printed is the problem: the process left with 143, the status a shell reports for a process killed by signal 15, while the trap saw 0.

The advice you find everywhere, and which I follow, is to capture $? on the first line of the handler so you keep the real status. It is correct, and I verified it: a script ending in exit 7 shows 7 in the handler. It just does not hold when a signal did the killing. The 143 comes from the signal, not from anything the script decided.

For me that meant the guard that emails me about failed deploys stayed silent on exactly the runs I care about. A killed deploy filed itself as a success.

And it fires before your child process dies

The signal goes to the script. The command it is running is a separate process that gets nothing, and bash is in a hurry to leave.

$ pgrep -f 'sleep 60' | wc -l
0
$ ./sleepy.sh & pid=$!; sleep 1; kill -TERM $pid; wait $pid
trap saw 0
$ pgrep -f 'sleep 60' | wc -l
1

The trap ran. The work was still going afterwards. bash executed the handler and exited immediately, orphaning the child. So a handler whose job is rm -rf "$work" removes a directory that a live process is still writing into. Three runs of that test left three orphaned sleeps, and nothing escalates them.

That is the part I had backwards. I thought of the trap as a wall at the end of the script, with everything already finished on the far side.

Trap the signal and the timing moves

The obvious fix is to stop letting the signal do whatever it wants. Note it, then exit on purpose. That works, and it moves the problem instead of removing it.

#!/bin/bash
trap 'echo "handler at $(date +%S)"' TERM
echo "started at $(date +%S)"
sleep 6

Start it, send TERM a fraction of a second later, and watch the handler:

started at 42
handler at 48

bash does not run a trap handler while it is waiting on a foreground command. It notes the signal and runs the handler once that command returns. A graceful shutdown built this way waits for whatever is running at that moment, and the six seconds above could just as easily be a ten-minute upload.

There is a second edge here. If the handler never calls exit, the script carries on to the end of the file and exits 0. Mine did, which turned SIGTERM into a no-op.

Background the work, then wait on it

The exit code, the timing and the orphan all have the same root: the script does not own the process doing the work. Background the long command, keep the pid, and wait on it. wait is interruptible, and that is the entire trick.

#!/bin/bash
set -euo pipefail

work=$(mktemp -d)
child=""
sig=""

on_signal() { sig="$1"; }
trap 'on_signal INT' INT
trap 'on_signal TERM' TERM

cleanup() {
  local rc=$?
  trap - EXIT INT TERM
  if [[ -n $child ]] && kill -0 "$child" 2>/dev/null; then
    kill -TERM "$child" 2>/dev/null || true
    wait "$child" 2>/dev/null || true
  fi
  rm -rf "$work"
  (( rc == 0 )) || echo "exit $rc${sig:+ after $sig}" >&2
  exit "$rc"
}
trap cleanup EXIT

wrangler pages deploy "$work" & child=$!
wait "$child"

Against the same test, the handler now runs as soon as the signal arrives, the child is dead before the directory is removed, and the process exits 143, which is the number the trap sees. When a trapped signal arrives during wait, the builtin gives up immediately with a status above 128. set -e ends the script there, and the trap finally has a real number to compare against.

The same script was hiding a second success

With the exit codes finally visible, the other reason my script never reported a bad run was easy to spot. Every long step pipes through tee so the log file keeps a copy:

set -e
node scripts/generate-blog-post.js 2>&1 | tee -a "$LOG_FILE"

if [[ $? -ne 0 ]]; then
  exit 1
fi

Two problems stacked on two lines. Without pipefail, that pipeline reports tee's status, so set -e has nothing to react to. And by the time $? is read on the following line, it is still tee's status. That check has never once been true.

$ false | tee /dev/null >/dev/null; echo "PIPESTATUS=${PIPESTATUS[*]} status=$?"
PIPESTATUS=1 0 status=0

The 1 is right there in the array, one variable away from the code that would act on it. Add set -o pipefail at the top and the pipeline reports the generator's status, set -e ends the script, and the trap's exit code starts meaning something.

What I keep coming back to is that a trap is not try/finally. It is a hook the shell calls while it is already on the way out, at a moment when it may have lost track of its children. Treating it as cleanup that always tells the truth is how a killed deploy gets filed as a success.

The skeleton lives in Snippet Ark with the measured outputs pasted into the note beside it, because I will not remember which line matters in six months. It sits next to the strict mode notes and the piece on jq exit codes: bash does tell you what happened, and reading the wrong value is how you end up believing it.