As CrowdStrike and its enterprise customers recover from the recent outage catastrophe, and it already being public knowledge that a pushed update caused the problem, the company has hired two security firms to look further into the issue.
The external code review was announced in a root causes analysis (PDF), while it was already known in the course of a post-incident review that a system designed to validate content (a ‘Content Validator’) failed to kick in, allowing a faulty IPS Template Instance intended to detect attacks to validate, causing crashes due to out-of-bounds memory reads.
CrowdStrike has announced it intends to mitigate similar broken update disruption in the future by staggering template deployment across devices, and that its content validator now has runtime bounds, preventing the same kind of memory issues from happening. It also intends to perform more internal testing, but only time will tell if this will have much material impact.
CrowdStruck (with a corporate lawsuit)
Even if you aren’t completely sure what a content validator is or how exactly memory reads can go above their station, you can probably imagine that a phased update rollout system sounds like a good idea for a company with software installed on millions of Windows PCs.
CrowdStrike’s shareholders have been thinking along the same lines, and have already filed a class-action lawsuit against the company for failing to implement such a system. Delta, meanwhile, are suing on the basis of lost revenue over a six-day period – which CrowdStrike say, perhaps with good reason, is Delta’s fault, actually,
Then again, it also said, about the shareholders case, that it believes the case ‘lacks merit’, and it’s hard to argue that one given that the implementation, or lack thereof, of a rolling patch system, lies entirely at CrowdStrike’s feet.
Via The Register