Facebook says outage caused by ‘faulty configuration change’
Mistake or insider sabotage both plausible
Facebook Inc. blamed a “faulty configuration change” for a nearly six-hour outage on Monday that prevented the company’s 3.5 billion users from accessing its social media and messaging services such as Whatsapp, Instagram and Messenger.
The company in a late Monday blog post did not specify who executed the configuration change and whether it was planned.
Several Facebook employees who declined to be named had told Reuters earlier that they believed that the outage was caused by an internal mistake in how internet traffic is routed to its systems.
The failures of internal communication tools and other resources that depend on that same network in order to work compounded the error, the employees said. Security experts have said an inadvertent mistake or sabotage by an insider were both plausible.
“We want to make clear at this time we believe the root cause of this outage was a faulty configuration change,” Facebook said in the blog.
The Facebook outage is the largest ever tracked by web monitoring group Downdetector.
As the world flocked to competing apps such as Twitter and Tiktok, shares of Facebook fell 4.9 per cent, their biggest daily drop since last November, amid a broader selloff in technology stocks on Monday. Shares rose about half a per cent in after-hours trade following resumption of service and closed Tuesday at US$332.96, up 2.06 per cent.
“To every small and large business, family, and individual who depends on us, I’m sorry,” Facebook chief technology officer Mike Schroepfer tweeted, adding that it “may take some time to get to 100%.”
“Facebook basically locked its keys in its car,” tweeted Jonathan Zittrain, director of Harvard’s Berkman Klein Center for Internet & Society.
Twitter on Monday reported higher-than-normal usage, which led to some issues in people accessing posts and direct messages.
In one of the day’s most popular tweets, video streaming company Netflix shared a meme from its new hit show Squid Game captioned “When Instagram & Facebook are down,” that showed a person labelled “Twitter” holding up a character on the verge of falling labelled “everyone.”
Inside a Facebook group for ad buyers, one member wisecracked after service returned that “lots of people searched today ‘how to run Google ads for clients.’”
Facebook, which is the world’s largest seller of online ads after Google, was losing about US$545,000 in U.S. ad revenue per hour during the outage, according to estimates from ad measurement firm Standard Media Index.
Past downtime at internet companies has had little long-term affect on their revenue growth, however.
Facebook’s services, including consumer apps such as Instagram, workplace tools it sells to businesses and internal programs, went dark noon Eastern time (1600 GMT). Access started to return around 5:45 pm ET.
Soon after the outage started, Facebook acknowledged users were having trouble accessing its apps but did not provide any specifics about the nature of the problem or say how many users were affected.
The error message on Facebook’s webpage suggested an error in the Domain Name System (DNS), which allows web addresses to take users to their destinations. A similar outage at cloud company Akamai Technologies Inc. took down multiple websites in July.
On Sunday, Frances Haugen, who worked as a product manager on the civic misinformation team at Facebook, revealed that she was the whistleblower who provided documents underpinning a recent Wall Street Journal investigation and a U.S. Senate hearing last week on Instagram’s harm to teen girls.
Haugen was due to urge the same Senate subcommittee on Tuesday to regulate the company, which she plans to liken to tobacco companies that for decades denied that smoking damaged health, according to prepared testimony seen by Reuters.