Tech
A guide to working in the Silicon Valley tech scene – Information Age
Airlines, banks, hospitals and other risk-averse organizations around the world chose cybersecurity company CrowdStrike to protect their computer systems from hackers and data breaches.
But all it took was one faulty CrowdStrike software update to cause global disruptions Friday that grounded flights, knocked banks and media outlets offline, and disrupted hospitals, retailers and other services.
“This is a function of the very homogenous technology that goes into the backbone of all of our IT infrastructure,” said Gregory Falco, an assistant professor of engineering at Cornell University. “What really causes this mess is that we rely on very few companies, and everybody uses the same folks, so everyone goes down at the same time.”
The trouble with the update issued by CrowdStrike and affecting computers running Microsoft’s Windows operating system was not a hacking incident or cyberattack, according to CrowdStrike, which apologized and said a fix was on the way.
But it wasn’t an easy fix. It required “boots on the ground” to remediate, said Gartner analyst Eric Grenier.
“The fix is working, it’s just a very manual process and there’s no magic key to unlock it,” Grenier said. “I think that is probably what companies are struggling with the most here.”
While not everyone is a client of CrowdStrike and its platform known as Falcon, it is one of the leading cybersecurity providers, particularly in transportation, healthcare, banking and other sectors that have a lot at stake in keeping their computer systems working.
“They’re usually risk-averse organizations that don’t want something that’s crazy innovative, but that can work and also cover their butts when something goes wrong. That’s what CrowdStrike is,” Falco said. “And they’re looking around at their colleagues in other sectors and saying, ‘Oh, you know, this company also uses that, so I’m gonna need them, too.’”
Worrying about the fragility of a globally connected technology ecosystem is nothing new. It’s what drove fears in the 1990s of a technical glitch that could cause chaos at the turn of the millennium.
“This is basically what we were all worried about with Y2K, except it’s actually happened this time,” wrote Australian cybersecurity consultant Troy Hunt on the social platform X.
Across the world Friday, affected computers were showing the “blue screen of death” — a sign that something went wrong with Microsoft’s Windows operating system.
But what’s different now is “that these companies are even more entrenched,” Falco said. “We like to think that we have a lot of players available. But at the end of the day, the biggest companies use all the same stuff.”
Founded in 2011 and publicly traded since 2019, CrowdStrike describes itself in its annual report to financial regulators as having “reinvented cybersecurity for the cloud era and transformed the way cybersecurity is delivered and experienced by customers.” It emphasizes its use of artificial intelligence in helping to keep pace with adversaries. It reported having 29,000 subscribing customers at the start of the year.
The Austin, Texas-based firm is one of the more visible cybersecurity companies in the world and spends heavily on marketing, including Super Bowl ads. At cybersecurity conferences, it’s known for large booths displaying massive action-figure statues representing different state-sponsored hacking groups that CrowdStrike technology promises to defend against.
CrowdStrike CEO George Kurtz is among the most highly compensated in the world, recording more than $230 million in total compensation in the last three years. Kurtz is also a driver for a CrowdStrike-sponsored car racing team.
After his initial statement about the problem was criticized for lack of contrition, Kurtz apologized in a later social media post Friday and on NBC’s “Today Show.”
“We understand the gravity of the situation and are deeply sorry for the inconvenience and disruption,” he said on X.
Richard Stiennon, a cybersecurity industry analyst, said this was a historic mistake by CrowdStrike.
“This is easily the worst faux pas, technical faux pas or glitch of any security software provider ever,” said Stiennon, who has tracked the cybersecurity industry for 24 years.
While the problem is an easy technical fix, he said, it’s impact could be long-lasting for some organizations because of the hands-on work needed to fix each affected computer. “It’s really, really difficult to touch millions of machines. And people are on vacation right now, so, you know, the CEO will be coming back from his trip to the Bahamas in a couple of weeks and he won’t be able to use his computers.”
Stiennon said he did not think the outage revealed a bigger problem with the cybersecurity industry or CrowdStrike as a company.
“The markets are going to forgive them, the customers are going to forgive them, and this will blow over,” he said.
Forrester analyst Allie Mellen credited CrowdStrike for clearly telling customers what they need to do to fix the problem. But to restore trust, she said there will need to be a deeper look at what occurred and what changes can be made to prevent it from happening again.
“A lot of this is likely to come down to the testing and software development process and the work that they’ve put into testing these kinds of updates before deployment,” Mellen said. “But until we see the complete retrospective, we won’t know for sure what the failure was.”
___
Associated Press writer Alan Suderman in Richmond, Virginia, contributed to this report.
Transport providers, businesses and governments on Saturday are rushing to get all their systems back online after long disruptions following a widespread technology outage.
The biggest continuing effect has been on air travel. Carriers canceled thousands of flights on Friday and now have many of their planes and crews in the wrong place, while airports facing continued problems with checking in and security.
At the heart of the massive disruption is CrowdStrike, a cybersecurity firm that provides software to scores of companies worldwide. The company says the problem occurred when it deployed a faulty update to computers running Microsoft Windows, noting that the issue behind the outage was not a security incident or cyberattack.
Here’s the Latest:
Microsoft: 8.5 million devices on its Windows system were affected
Microsoft says 8.5 million devices running its Windows operating system were affected by a faulty cybersecurity update Friday that led to worldwide disruptions.
A Saturday blog post from Microsoft was the first estimate of the scope of the disruptions caused by cybersecurity firm CrowdStrike’s software update.
“We currently estimate that CrowdStrike’s update affected 8.5 million Windows devices, or less than one percent of all Windows machines,” said the blog post from Microsoft cybersecurity executive David Weston.
“While the percentage was small, the broad economic and societal impacts reflect the use of CrowdStrike by enterprises that run many critical services.”
Weston said such a significant disturbance is rare but “demonstrates the interconnected nature of our broad ecosystem.” Windows is the dominant operating system for personal computers around the world.
Austrian doctors’ group calls for better data protection for patients
In Austria, a leading doctors organization said the global IT outage exposed the vulnerability of health systems reliant on digital systems.
“Yesterday’s incidents underscore how important it is for hospitals to have analogue backups” to safeguard patient care, Harald Mayer, vice president of the Austrian Chamber of Doctors, said in a statement on the organization’s website.
The organization called on governments to impose high standards in patient data protection and security and on health providers to train staff and put systems in place to manage crises.
“Happily, where there were problems, these were kept small and short-lived and many areas of care were unaffected” in Austria, Mayer said.
Germany warns of scams after major IT outage
BERLIN — The German government’s IT security agency says numerous companies are still struggling with the consequences of a far-reaching technology outage.
“Many business processes and procedures have been disturbed by the breakdown of computer systems,” the BSI agency said on its website.
But the agency also said Saturday that many impacted areas have returned to normal.
It warned that cybercriminals were trying to take advantage of the situation through phishing, fake websites and other scams and that “unofficial” software code was in circulation.
The agency said it was not yet clear how faulty code ended up in the CrowdStrike software update blamed for triggering the outage.
European airports appear to be close to normal
LONDON — Europe’s busiest airport, Heathrow, said it is busy but operating normally on Saturday. The airport said in a statement that “all systems are back up and running and passengers are getting on with their journeys smoothly.“
Some 167 flights scheduled to depart from U.K. airports on Friday were canceled, while 171 flights due to land were axed.
Meanwhile, flights at Berlin Airport were departing on or close to schedule, German news agency dpa reported, citing an airport spokesman.
Nineteen flights took off in the early hours of Saturday after authorities exempted them from the usual ban on night flights.
On Friday, 150 of the 552 scheduled inbound and outbound flights at the airport were canceled over the IT outage, disrupting the plans of thousands of passengers at the start of the summer vacation season in the German capital.
German hospital slowly restoring its systems after widespread cancellations
BERLIN — The Schleswig-Holstein University Hospital in northern Germany, which on Friday canceled all elective surgery because of the global IT outage, said Saturday that it was gradually restoring its systems.
In a statement on its website, it forecast that operations at its two branches in Kiel and Luebeck would return to normal by Monday and that “elective surgery can take place as planned and our ambulances can return to service.”
Britain’s transport system still trying to get back on track
LONDON — Britain’s travel and transport industries are struggling to get back on schedule after the global security outage with airline passengers facing cancellations and delays on the first day of summer holidays for many school pupils.
Gatwick Airport said “a majority” of scheduled flights were expected to take off. Manchester Airport said passengers were being checked in manually and there could be last-minute cancellations.
The Port of Dover said it was seeing an influx of displaced air passengers, with hourlong waits to enter the port to catch ferries to France.
Meanwhile, Britain’s National Cyber Security Center warned people and businesses to be on the lookout for phishing attempts as “opportunistic malicious actors” try to take advantage of the outage.
The National Cyber Security Center’s former head, Ciaran Martin, said the worst of the crisis was over, “because the nature of the crisis is that it went very wrong very quickly. It was spotted quite quickly and essentially it was turned off.”
He told Sky News that some businesses would be able to get back to normal very quickly, but for sectors such as aviation it would take longer.
“If you’re in aviation, you’ve got people, planes and staffs all stranded in the wrong place… So we are looking at days. I’d be surprised if we’re looking at weeks.”
Germany airline expects most of its flights to run normally
BERLIN — Eurowings, a budget subsidiary of Lufthansa, said it expected to return to “largely scheduled” flight operations on Saturday.
On Friday, the global IT outage had forced the airline to cancel about 20% of its flights, mostly on domestic routes. Passengers were asked to take trains instead.
“Online check-in, check-in at the airport, boarding processes, booking and rebooking flights are all possible again,” the airline said Saturday on X. “However, due to the considerable extent of the global IT disruption there may still be isolated disruptions” for passengers, it said.
Delta Air Lines and its regional affiliates have canceled hundreds of flights
DALLAS — Delta Air Lines and its regional affiliates canceled more than a quarter of their schedule on the East Coast by midafternoon Friday, aviation data provider Cirium said.
More than 1,100 flights for Delta and its affiliates have been canceled.
United and United Express had canceled more than 500 flights, or 12% of their schedule, and American Airlines’ network had canceled 450 flights, 7.5% of its schedule.
Southwest and Alaska do not use the CrowdStrike software that led to the global internet outages and had canceled fewer than a half-dozen flights each.
Portland, Oregon, mayor declares an emergency over the outage
PORTLAND, Ore. — Mayor Ted Wheeler declared an emergency Friday after more than half of the city’s computer systems were affected by the global internet outage.
Wheeler said during a news conference that while emergency services calls weren’t interrupted, dispatchers were having to manually track 911 calls with pen and paper for a few hours. He said 266 of the city’s 487 computer systems were affected.
Border crossings into the US are delayed
SAN DIEGO — People seeking to enter the U.S. from both the north and the south found that the border crossings were delayed by the internet outage.
The San Ysidro Port of Entry was gridlocked Friday morning with pedestrians waiting three hours to cross, according to the San Diego Union-Tribune.
Even cars with people approved for a U.S. Customers and Border Protection “Trusted Traveler” program for low-risk passengers waited up to 90 minutes. The program, known as SENTRI, moves passengers more quickly through customs and passport control if they make an appointment for an interview and submit to a background check to travel through customs and passport control more quickly when they arrive in the U.S.
Meanwhile, at the U.S.-Canada border, Windsor Police reported long delays at the crossings at the Ambassador Bridge and the Detroit-Windsor tunnel.
Latest
European Union accuses Microsoft of breaching antitrust rules by bundling Teams with office software
LONDON (AP) — Microsoft violated European Union antitrust rules with “possibly abusive” practices by tying its Teams messaging and videoconferencing app to its widely used business software, the bloc said.
The European Commission said Monday it informed Microsoft of its preliminary view that the U.S. tech giant has been “restricting competition” by bundling Teams with core office productivity applications such as Office 365 and Microsoft 365.
The commission, the 27-nation bloc’s top antitrust enforcer, said it suspects Microsoft might have granted Teams a “distribution advantage” by not giving customers a choice on whether to have Teams when they purchased the software. The advantage might have been widened by limits on the ability of rival messaging apps to work with Microsoft software, it said.
“We are concerned that Microsoft may be giving its own communication product Teams an undue advantage over competitors, by tying it to its popular productivity suites for businesses,” Margrethe Vestager, the commission’s executive vice-president for competition policy, said in a statement.
“And preserving competition for remote communication and collaboration tools is essential as it also fosters innovation on these markets.”
The commission took aim at Microsoft a day after accusing Apple of breaching the bloc’s new digital competition rulebook, in a flurry of regulatory action underlining Brussels’ leading role as a watchdog for Big Tech companies.
Microsoft made some changes last year in an effort to head off an penalty, including offering the software packages without Teams for European customers. But the commission said Tuesday the changes are not enough to address its concerns and that it needs to do more to “restore competition.”
“Having unbundled Teams and taken initial interoperability steps, we appreciate the additional clarity provided today and will work to find solutions to address the Commission‘s remaining concerns.” Microsoft President Brad Smith said in a prepared statement.
In April, the company also gave customers worldwide the option to get Microsoft 365 and Office 365 without Teams. The two software suites include programs like Word, Excel and Outlook.
Microsoft now has a chance to respond to the accusations, formally known as a statement of objections, before the commission makes its final decision. The company could face a fine worth up to 10% of its annual global revenue, or be forced to carry out “remedies” to satisfy the competition concerns.
The commission opened its investigation in July 2023 after rival Slack Technologies, which makes popular workplace messaging software, filed a complaint with Brussels. Alfaview, which makes videoconferencing software, also filed a separate complaint.
Slack, owned by business software maker Salesforce, had alleged that Microsoft abused its market dominance to eliminate competition — in violation of EU laws.
“The Statement of Objections issued today by the European Commission is a win for customer choice and an affirmation that Microsoft’s practices with Teams have harmed competition,” Salesforce President Sabastian Niles said. “We appreciate the Commission’s thorough investigation of Slack’s complaint and urge the Commission to move towards a swift, binding, and effective remedy that restores free and fair choice and promotes competition, interoperability, and innovation in the digital ecosystem.”
