How an empty firewall file bricked my whole network

I took down my entire home network — every VLAN, every device — by deploying a firewall config that did not exist. The router was fine. The hardware was fine. The config file on the router was exactly zero bytes long, and OpenWrt considered that perfectly valid. This is the postmortem.

What was I trying to do?

My setup: a Banana Pi R4 running OpenWrt as the router, VLANs for network segmentation, and all configs tracked in a git repo. Deploys go over SSH: edit locally, push to the router, check, reload. The push looked like this:

cat configs/firewall | ssh router 'cat > /etc/config/firewall && fw4 check && fw4 reload'

I ran it from the wrong directory. There was no configs/firewall relative to where I stood, so the local cat failed. But a pipeline’s exit status is the last command’s, and the SSH side ran regardless. The remote cat > /etc/config/firewall received zero bytes of input and dutifully truncated the router’s firewall config to an empty file. Then fw4 check looked at the empty file, found nothing syntactically wrong with it, and fw4 reload applied it.

Why did the whole network die?

Because fw4’s defaults for a missing/empty config are the safe-for-the-router, fatal-for-you kind: input, forward and output policies all drop. Every packet in every direction, on every VLAN, hit a drop policy. There were no DHCP leases, no ping, no SSH, from anywhere. The router was up and healthy the whole time, silently discarding everything, including my attempts to get back in and fix it.

Why was diagnosis so confusing?

Two red herrings ate most of the diagnosis time.

First, I power-cycled the router mid-diagnosis. When it came back just as dead, I started fearing boot corruption: had the power cut mid-write broken the flash? That fear was wrong, but I could not rule it out from outside a router that answered nothing.

Second, I plugged in directly and probed with untagged ARP, and got silence. That silence had nothing to do with the firewall: my LAN ports are tagged-only trunks, so untagged frames go nowhere by design. The probe could never have gotten an answer, and I read the silence as a dead router.

And a third trap specific to the hardware: on the Banana Pi R4, the RST button is a hard reset. Failsafe mode listens on the WPS button. If you hold the wrong one, you are not entering failsafe; you are just rebooting into the same locked-out state.

How did I recover?

I entered failsafe mode (via the WPS button), which brings the router up at 192.168.1.1 with no firewall in the way. From there I restored the real firewall config (19 KB of it, sitting safely in the git repo) and rebooted. The network came back.

The recovery had one bonus find: a latent adblock boot race. The blocklists were being fetched before the WAN was up, with no interface trigger to retry, so adblock could come up empty after a reboot. Fixed with adb_trigger='wan' so it waits for the WAN interface.

What did I learn?

• Configs in git saved the day. Recovery was “copy a file and reboot” instead of reconstructing a firewall from memory in failsafe mode.
• Harden deploy scripts. Abort on empty input, checksum after push, reload only after verification. The hardened version:

#!/usr/bin/env bash
set -euo pipefail

CONF="configs/firewall"

# Refuse to push a missing or empty file
test -s "$CONF" || { echo "abort: $CONF missing or empty" >&2; exit 1; }

sum_local="$(sha256sum "$CONF" | cut -d' ' -f1)"

ssh router 'cat > /etc/config/firewall' < "$CONF"

# Verify the bytes actually arrived before touching the live firewall
sum_remote="$(ssh router 'sha256sum /etc/config/firewall' | cut -d' ' -f1)"
[ "$sum_local" = "$sum_remote" ] || { echo "abort: checksum mismatch, not reloading" >&2; exit 1; }

ssh router 'fw4 check && fw4 reload'

• Know your failsafe button before the outage. On the BPI-R4 that means WPS, not RST. Look yours up while your network still works.
• A set -o pipefail and a test -s would have prevented the whole incident. Two lines of shell are the entire cost of not bricking your network.

I write these postmortems because running your own infrastructure means owning your own outages. That is the deal, and I still think it is a good one. More on why in why another tech blog.