Bank’s latest outage raises new concerns
Wells Fargo problem calls tech systems into question
While it wasn’t the kind of blank-slate wipeout of the financial system envisioned in dystopian television series “Mr. Robot,” this week’s Wells Fargo outage — the San Francisco bank’s second failure this month — caused some to ask whether the banking systems we depend on are too fragile.
Starting Thursday and lasting through Friday morning, ATMs failed to show balances and transactions, web log-ons were rejected, and smartphone apps displayed errors. The bank ultimately blamed smoke in a data center.
That explanation perplexed customers and computer experts alike. Modern computing systems are distributed across multiple locations so that a single failure does not bring things down, said Spencer Kimball, a former Google software engineer whose company,
Cockroach Labs, develops database management systems for businesses.
“In today’s world, which is completely dependent on the availability, speed and correctness of data, an outage is simply no longer acceptable,” Kimball said.
Though Wells said the problem was largely resolved Friday, some customers said they continued to have problems with ATMs being down and debit cards getting declined. The bank had an unrelated outage a week earlier.
“I hope the Russians haven’t taken over my bank account,” tweeted foreign exchange trader Kim Krompass of Napa, one of many who sent the hashtag #WellsFargoDown trending.
Wells said the outage was a “contained issue,” not caused by a cybersecurity breach.
When smoke was detected at a Minnesota data center after routine maintenance, the bank automatically cut power to the site and attempted to reroute transactions to other centers. The transition did not go smoothly.
Cockroach Labs’ Kimball said this illustrated a “key oversight”: “Many companies may have a backup data center or some degree of redundancy, but there is often no workable plan in place for if — and, more likely, when — their primary data center goes down.”
Bank of America customers were shut out from their accounts for several hours in 2017 during a nationwide outage. The cause was not known at the time. In February 2018, a rollout of new technology contributed to a three-day outage at BB&T Bank that cost it $20 million and led it to spend $300 million on a new data center, American Banker reported.
Amit Hooda of Colorado went into a Wells Fargo branch in San Francisco on Thursday and again on Friday, after being unable to complete a transaction online.
“We weren’t able to do any international payments, and that takes two days,” Hooda said.
Wells Fargo chief Tim Sloan apologized for the inconvenience in a statement.
“While we restored operations throughout the day and continue to address customer concerns, our recovery from these issues was not as rapid as we or our customers would have expected,” Sloan said. “We will review the system issues in detail, and do all we can to ensure that this type of disruption doesn’t happen again.”
The bank said it would keep 5,500 branches open an hour past their usual closing times on Friday and Saturday.
In the 1990s, Wells Fargo was seen as an innovator in online banking, and it was a crucial behind-the-scenes player in getting startups like PayPal off the ground. More recently, its spending on technology in 2017 hit $8.2 billion, up 15 percent over 2016, when it saw a more modest increase. That represents 9.5 percent of its annual revenue of $86.4 billion.
In recent years, Wells Fargo management’s attention has been consumed by a swirling scandal over the creation of millions of fake customer accounts by employees pressured to meet aggressive sales goals. It paid $575 million to settle state investigations into the fraud claims in December.
The bank created a new role to oversee technology and information security teams, reporting directly to Sloan. Saul Van Beurden, who most recently was chief information officer of consumer and community banking at JPMorgan Chase, will fill the position as head of technology in April.