The federal government is anticipating up to a fortnight of residual issues from a bad cybersecurity update that bricked an estimated 8.5 million Windows devices worldwide.
Home Affairs Minister Clare O’Neil said a further national coordination mechanism meeting occurred on Sunday, again with involvement from managed detection and response vendor CrowdStrike, whose software caused the issue.
An update to a single configuration file in CrowdStrike’s tooling put Windows machines in reboot loops, and eventually displaying a ‘blue screen of death’. Industries across the world were impacted.
O’Neil said that there had “been a huge amount of work over this weekend to get the [Australian] economy back up and running” after cascading impacts observed in sectors from government and finance to transportation and retail.
“However, it will take time until all affected sectors are completely back online,” O’Neil said in a series of threaded posts on X.
“In some cases, we may see teething issues for one or two weeks.”
CrowdStrike is continuing to maintain a list of remediation guidance and resources for impacted teams and environments.
In a separate blog post, Microsoft estimated that CrowdStrike’s configuration file update “affected 8.5 million Windows devices, or less than one percent of all Windows machines.”
“While the percentage was small, the broad economic and societal impacts reflect the use of CrowdStrike by enterprises that run many critical services,” Microsoft’s vice president of enterprise and OS security David Weston wrote.
One of the challenges in recovering machines is that it requires IT staff to physically attend to each impacted machine.
At one point, news reports noted guidance that up to 15 reboots per machine may be required.
O’Neil said on X that CrowdStrike is “now close to rolling out an automatic fix to the issue with their update, as is Microsoft.”
“This should increase the speed at which systems across the economy are back online,” she said.
Microsoft’s Weston wrote that CrowdStrike has helped it “develop a scalable solution that will help … Azure infrastructure accelerate a fix for CrowdStrike’s faulty update.”
He further outlined cooperative efforts occurring between “global cloud providers, software platforms, security vendors and other software vendors, and customers” to get systems operational.
“We learn, recover and move forward most effectively when we collaborate and work together,” Weston said.
“We appreciate the cooperation and collaboration of our entire sector, and we will continue to update with learnings and next steps.”
O’Neil said that she was “personally thankful to every worker, both in IT and across the economy, who has worked through this weekend” to reboot impacted machines.
