Amazon deliveries idled by AWS outage
Setback could cause lasting logjams. Sites of Netflix, Disney and others are affected.
An Amazon Web Services outage wreaked havoc on the e-commerce giant’s delivery operation, preventing drivers from getting routes or packages and shutting down communication between Amazon and the thousands of drivers it relies on, according to four people familiar with the situation.
Three delivery service partners said an Amazon.com Inc. app used to communicate with delivery drivers was down.
Vans that were supposed to be on the road delivering packages sat idle with no communication from the company, one person said. Amazon Flex drivers, independent delivery people who carry parcels in their own cars, couldn’t log in to Amazon’s app to get assignments, another person said.
Amazon said late Tuesday afternoon that much of its web service was working again. “We have mitigated the underlying issue that
caused some network devices in the US-EAST-1 Region to be impaired,” the company said on its AWS dashboard just before 6 p.m. Eastern time. “We are seeing improvement in availability across most AWS services.”
The company didn’t comment on the issues with its delivery operation.
The problems came amid Amazon’s crucial holiday shopping season, when the company can ill afford delays that could potentially create lasting logjams. One delivery business owner on the West Coast said the company halted deliveries Tuesday and planned to regroup
Wednesday.
Two delivery partners in earlier time zones said drivers who already had routes were instructed to put their phones in airplane mode and not log out of the Amazon routing app so they could continue making stops, but drivers who hadn’t already been assigned routes were sidelined.
The outage began about 10 a.m. Eastern time, according to DownDetector. At the height of the outage, the web monitoring site reported more than 20,000 complaints for Amazon and more than 11,000 for the company’s cloud computing arm, Amazon Web Services. By 1:45 p.m., the reported outages had declined by about half for AWS and twothirds for Amazon.
Multiple popular websites were also affected, including those operated by Coinbase, Robinhood, Disney and Netflix, according to DownDetector.
Disney said that although people were able to get into its theme parks, they were having difficulty checking in online and paying for purchases. Webcast presentations from Comcast and Altice USA at UBS’ Global TMT Conference experienced disruptions Tuesday, and the Charter Communications presentation was rescheduled for Wednesday.
Some Amazon services, including music and video streaming, the voice-activated Alexa platform and security arm Ring were affected too.
Just after 1 p.m. Eastern time, Amazon said it had identified the root cause of the issue and was working to fix it. Meanwhile, the company directed customers to alternative servers in its western region that weren’t experiencing problems. The increased errors were in the eastern North America region.
Amazon, in an update on its AWS dashboard, said it still hadn’t completely fixed the issue by 6 p.m. Eastern time, although “many services have already recovered.” Multiple Amazon cloudcomputing services had been affected, including Amazon DynamoDB and Amazon Elastic Compute.
Streaming service Netflix experienced a 26% drop in traffic after the AWS problems were reported, showing how quickly outages can ripple outward, said Doug Madory, an analyst at the network monitoring firm Kentik in San Francisco.
“It gets more and more complicated with software running these services, so when something goes sideways it can take a long time to figure out what went wrong and fix it,” he said. “Complexity has risks. You introduce unknown errors.”
AWS is the leading cloudcomputing provider, enabling companies to purchase computing power and software services on demand rather than maintaining their own data centers and teams in-house. Its customers include a wide range of industries and the federal government.