Using Git to automate errors

Last week, I was writing about the wonders of Git and how it can be helpful to limit the risk of making easily avoidable errors. It is also incredibly funny that as I was adding some of the finishing touches, I actually made easily avoidable mistakes in this process. Let’s get started. This post will follow a simple format, again :

• Timeline
• Root cause
• Impact analysis
• Actions taken
• Lessons learned

Timeline

On Friday the 24th of July, as I was bringing the finishing touches to this article, I had only one thing left to be done, opening a merge request to my dev branch, then validate. Once that is done, I can push the changes at a later date to the main branch, which used to base this specific website. The merge was performed on an incomplete and broken script that I was reworking at the time, and would cause issues in the long term. The script is made to cleanup same-host backups.

Root cause

Due to having neglected to change the default branch in my GitLab, I’ve opened the merge request to main. This compounded with the 2:00 AM time of the day. I had had a pretty nice cocktail a few hours earlier, which compounded fatigue, making me rather reckless in the way I operated. The finishing touch was the earlier than usual wake up time.

It can be said that the root cause is thus a combination of factors :

• The default branch on GitLab was main
• I was too tired to make those changes and made mistakes
• I kept going and made it worse

Impact analysis

Updates wouldn’t get automatically published and backups not cleaned. It was annoying, that’s it. Only the workflow for the website is impacted, and is easily recoverable.

Actions taken

I’ve reverted the main branch to a stable point by reverting the commits, and republished the branch on the repository after fixing. While attempting to fix it, I made a wrong revert and had to correct that as well. I then made a change in the GitLab settings to change the default branch to dev, in order to mitigate these mistakes. I’ll elaborate on the workflow a bit more in a later blogpost.

Lessons learned

Don’t do IT when tired, or be well acquainted with Murphy’s Law. But in all honesty, the lesson learned here is on the importance of guardrails. My usual process is to create a branch, commit on that branch, push it, merge into dev, and when ready merge into main. I had not put in place a guardrail to the merge request workflow precited. My preferred guardrail here is to change the default branch to dev. There’s however another way of doing which would also absolutely work, which is to create a freeze branch for that matter. In my case, I’ve decided. In yours, it’s up to you and what you feel is more natural.

The other part is, even if I’m pretty used to have Git nowadays, I still have some more mistakes to do before I can truly consider that I master it. That’s also the whole purpose of my homelab, so I call it a successful failure!