Can’t stream? Blame the cloud
When a cloud-storage malfunction at Amazon temporarily knocked out services including Netflix, Slack, the Securities and Exchange Commission’s website, ‘‘smart’’ thermostats and my email server this week, it demonstrated how much we’ve all come to depend on one company’s infrastructure.
It also shows how the centralization of data has made the internet -- at least the part that most of us use -- more vulnerable than it was designed to be.
Although Amazon may be known mainly for two-day shipping, cloud computing is where it actually makes money. The company owns big data centers around the world and rents out computing power and storage on demand. This global cloud infrastructure service is dominated by just four players, and Amazon’s market share is greater than that of the next three combined.
During the dot-com days, if I wanted to sell pet products on the web, I had to buy my own computer, keep it online at all times and upgrade it if my business grew.
If sales didn’t take off, I’d be stuck with hardware I no longer needed. Today, aspiring e-tailers don’t need to make that capital investment - they can just rent computing capacity from a cloud provider. The infrastructure automatically scales to meet their demands.
Yelp, Netflix, Lyft, and Airbnb all use Amazon’s cloud infrastructure. Until last year, Dropbox provided its seamless storage service entirely on Amazon. Even when a company is big enough to buy its own data centres, it’s almost always cheaper to rent. Cloud computing has become a commodity, and it’s difficult to compete with a company that prioritises volume over profit margins. Apple still runs parts of icloud on Amazon’s servers, and only recently moved a large portion to Google’s cloud.
Unfortunately, when cloud providers fail, they drag their customers down with them. An Amazon glitch in 2011 took popular websites offline for several days. Even if cloud software somehow becomes error-free, nature won’t. Outages have always happened. The difference is that now they can affect thousands of internet services at once.
This isn’t how the internet was supposed to work. It was specifically designed not to rely on any single piece of infrastructure. During the Cold War, the U.S. military wanted a communications system that could survive a nuclear strike. Precellular telephone networks consisted of centralized switchboards, with physical lines running into every house.
Communication required a continuous connection. If a switchboard or phone line went down, the call was over.
To address the problem, a researcher at RAND Corp designed a distributed communications system, with nodes connected through multiple paths.
If one pathway broke, the information would take an alternate route. In 1969, the technology developed into ARPANET, which linked nodes through a network of long-distance telephone calls that never hung up. This eventually evolved into what we know as the internet.
Today’s internet is still distributed, but the data that live on the network are more centralized. Thanks to the economic advantages of cloud computing, a few providers control a large amount of capacity. As a result, a network that was designed to survive a nuclear attack is now susceptible to outages at a single company.
Fortunately, the technology itself still works as originally intended - the Department of Defense has its own distributed communications networks separate from the civilian internet, so military operations can continue after an attack. The rest of us, by contrast, will have to live with the fact that our favorite services may occasionally cease to function all at once.
Elaine Ou is a blockchain engineer at Global Financial Access, a financial technology company in San Francisco.