Yeah, something this big is absolutely not one engineer’s fault. Even if that engineer maliciously pushed an update, it’s not their fault — it was a complete failure of the organization, and one person having the ability to wreck havoc like this is the failure.
And I actually have some amount of hope that, in this case, it is being recognized as such.
No they won’t, not if they’re in the slightest bit competent.
Blameless post-mortem culture is very common at big IT organizations. For a fuck-up this size, there are going to be dozens of problems identified, from bad QA processes, to bad code review processes, to bad documentation, to bad corner cases in tools.
There will probably be some guy (or gal) who pushed the button, but unless what that person did was utterly reckless (like pushing an update while high or drunk, or pushing a change then turning off her phone and going dark, or whatever) the person who pushed the button will probably be a legend to their peers. Even if they made a big mistake, if they followed standard procedures while doing it, almost everyone will recognize they’re not at fault, they just got to be the unlucky person who pushed the button this time.
Yeah, something this big is absolutely not one engineer’s fault. Even if that engineer maliciously pushed an update, it’s not their fault — it was a complete failure of the organization, and one person having the ability to wreck havoc like this is the failure.
And I actually have some amount of hope that, in this case, it is being recognized as such.
I agree but they will still blame it all on that one guy.
No they won’t, not if they’re in the slightest bit competent.
Blameless post-mortem culture is very common at big IT organizations. For a fuck-up this size, there are going to be dozens of problems identified, from bad QA processes, to bad code review processes, to bad documentation, to bad corner cases in tools.
There will probably be some guy (or gal) who pushed the button, but unless what that person did was utterly reckless (like pushing an update while high or drunk, or pushing a change then turning off her phone and going dark, or whatever) the person who pushed the button will probably be a legend to their peers. Even if they made a big mistake, if they followed standard procedures while doing it, almost everyone will recognize they’re not at fault, they just got to be the unlucky person who pushed the button this time.