Every script I write backs up what it touches

Two rules, on every hardening script I write. Back up whatever you’re about to change, with a timestamp. And be safe to run twice.

Neither is clever. Both came from getting it wrong.

The first one

I changed a working sshd config on a machine I couldn’t easily get back into, and I had no copy of what it had looked like ten seconds earlier, which meant that the difference between a five-minute fix and a genuinely unpleasant evening came down to one cp I hadn’t bothered to type. It was a small change. It was the wrong small change.

So now, before anything is written:

cp -a "$f" "$f.bak-$(date +%Y%m%d-%H%M%S)"

Timestamped, not .bak. A plain .bak gets overwritten by the second run, so the moment you most need the original is exactly the moment you have just destroyed it, which is not a hypothetical I constructed for this post but the specific way I learnt to put the date in.

cp -a rather than cp, so permissions and ownership survive. A backup of a config file that comes back with the wrong mode isn’t a backup, it’s a second problem waiting for you.

The second one

Run it twice, get the same result. No duplicated lines in sshd_config, no rule appended to the firewall for the third time, no service bounced for no reason.

This matters more than it sounds like it should, because you will run it twice. Because you weren’t sure it finished. Because someone else ran it an hour ago and didn’t mention it. Because you pushed it across thirty machines in a loop and eleven of them had already had it applied last month by a version of you who also forgot to write it down.

In practice that means checking for the setting before adding it, and editing in place when it’s there rather than appending. Which is more code, and slower to write, and the reason people skip it.

What it costs

Three lines. Ten minutes.

What it buys is that I’ll actually run the thing on a production box on a Thursday afternoon. That’s the whole return. A script I’m nervous about is a script that sits unrun while the machine stays unhardened, and an unhardened machine is the actual risk, not the script.

The confidence is the deliverable. The backup file is just how you buy it.

Where it doesn’t help

It won’t save you from a change that’s wrong in a way you don’t notice for a fortnight. The backup is sitting right there and you have no idea you need it.

It won’t save you from locking yourself out, either, which is the specific failure I care most about on a remote box. That needs a different habit: make the change, keep the session you’re already in open, and open a second one to test. If the second connection fails you still have the first, and you can put it back. I’ve used that more than once, and every time it was on a change I was sure about.

Sure is where it gets you.