Did ‘www.facebook.com’ go up for sale? A SYSTEM SO CENTRALISED THAT EVEN PHYSICAL ACCESS WAS RESTRICTED

First things first, facebook.com was never officially up for sale. Yet, once the BGP broke down taking Facebook services off the web, its Domain Name System (DNS) records became unreachable. If the BGP is the map, then think of the DNS as a phone or address book of the internet.

So as the BGP error was unfolding, configuration issues also caused the DNS of Facebook and the family of apps to disappear. The path, as well the destination, were broken. That meant websites that sell domain names, and automatically search for inactive domain names, started listing facebook.com as up for sale. Manual corrections were done to take down those listings, though a lot of Twitter users did get in on the fun with bids for the domain.

To correct the mess, Facebook engineers rushed to the California data centre. Since servers were unavailable online, they couldn’t remotely issue a patch to fix the error. Then, the engineers had trouble getting into the building because the authentication system, wherein badges are scanned to access the physical premises, was also down. Eventually they were able to get access to the servers. Facebook’s chief technology officer Mike Schroepfer said in an email to employees after services had been restored, that the issue was “affecting our networking backbone that connects all our data centres together.”

For systems as massive as this, rebooting after an update takes time – which explains the six hours it took for services to come online again. And even then, there are chances of access being inconsistent for a while as server databases are rebuilt. “This disruption ... had a cascading effect on the way our data centres communicate, bringing our services to a halt,” explains Santosh Janardahan, vice president of infrastructure at Facebook. “We also have no evidence that user data was compromised.”