Knowing “Why It Works” Shouldn’t Be Optional

“I hand-tuned the code till it was reliable, we won’t need to worry about it any more”
 — #CowboyDeveloper

When you’ve got a product thats’s been around long enough, you eventually run into an (unfortunately, not at all rare) member of the species known as #CowboyDevelopers. You know the type, the guy — yes it is usually a TechBro — who knows it all, and even worse, is the only guy who knows how that one critical component works.

The funny thing is, if you dig deep enough, you usually find something like the following

1. 1. He’s the only guy who knows what those cryptic logging messages mean.
   (“expired secondary return in previous ack not found”)
2. 2. The log messages don’t actually refer to a specific error, but are indicative of some meta-condition.
   (“Oh, that actually means that one of the ACK buffers on a remote node has filled up”)
3. 3. This meta-condition refers to something that most everyone else doesn’t even know exists
   (“Wait, why are there ACK buffers? And why are they only 4-deep?”)
4. 4. If you dig deep enough, you’ll find that even he doesn’t know why it works
   (“Yeah I started buffering the ACKs because it didn’t work without the buffers. And i keep increasing the buffer size till it started working. And it seems to fail with any buffer-size other than 4”)

/via https://bit.ly/2qxTV12

There are many many reasons why you can end up in this place, from things slipping through the cracks to egotistical founders who are also the lead developer. The results are always the same though — a ticking time bomb that will go off, and per Murphy, go off at a time when it can cause maximal damage.
So yeah, deal with the issue when you find it (and don’t forget to deal with the #CowboyDeveloper, while you are at it!)