Bombing reveals telecom weakness
Four days after the bombing on Christmas Day in downtown Nashville crippled cell service, internet and even key tools for law enforcement across a multi-state region, White County, Tennessee residents still struggled to get through to the county’s emergency communications center.
Though the center’s landlines worked and officials pushed out a nonemergency number via social media, the rural Tennessee county’s 911 Emergency Director Suzi Haston said she remained shocked their wireless services were still out after the bombing damaged an AT&T building more than 90 miles away.
Even farther away, in Alabama, the bombing forced first responders to use two-way radio and text messaging systems after the state’s primary communications network for public safety workers, Firstnet, was disrupted.
The vulnerability of the telecommunications system in Tennessee and across the Southeast became clear Christmas Day.
Now, federal, state, and local officials are starting to demand answers from AT&T, asking how such a telecommunications nightmare happened and how to ensure it can’t happen again.
An attack on critical infrastructure
The AT&T building in Nashville that suffered damage houses connection points for regional internet and wireless communications. Although authorities have not yet said what motivated the suspected bomber, the blast from an RV laden with explosives brought communications across the region, from Georgia to Kentucky, to a halt.
It affected 911 call centers, hospitals, the Nashville airport, government offices and individual mobile users. Patients were left unable to contact pharmacies, issues with credit card devices hamstrung businesses big and small.
Similar facilities and data centers exist in cities across the nation. Some, like one in Chattanooga, Tennessee,
are located within blocks of key government buildings.
As news of the bombing spread, officials in Mississippi sought to fortify key communications systems, scrambling to ensure critical infrastructure sites in the state, such as ports along the Gulf Coast and oil and gas refineries, were secure. The New York Police Department even ramped up security at communication facilities across the city.
The Nashville bombing raises questions about potential vulnerabilities elsewhere in the United States, said Colin Clarke, a senior fellow at The Soufan Center. Beyond the big utilities, how does the nation protect these “pedestrian places” that fewer people know exist, he said.
“You’d walk right past that place on the street. You have to kind of know it was there if you wanted to target it. Hence, there’s no need for massive security,” said Clarke, who teaches at Carnegie Mellon University in addition to his role at the New York-based non-partisan strategy center. “But now we’re starting to rethink that … How do you harden these soft targets and make sure that this doesn’t happen again?”
The Cybersecurity and Infrastructure Security Agency at the Department of Homeland Security said officials are working with partners in the region to assess what happened.
Tennessee Gov. Bill Lee, who toured the crime scene a day after the blast, said his administration has had multiple conversations with AT&T officials since the bombing.
Lee said he expects AT&T to “strengthen that infrastructure” over the next month.
“Anytime you have a situation like this, you can look at the aftermath and see where your weaknesses were,” he said. “There’s a lot to learn.”
U.S. Sen. Rob Portman, R-ohio, who will serve as the top Republican on the Senate Homeland Security and Governmental Affairs Committee in the new Congress, said the bombing underscores the “significance of infrastructure security and how important that is to our economy and the safety of our communities.”
Meanwhile, U.S. Sen. Marsha Blackburn, R-tennessee, said she hopes to work with her colleagues to advance a bill she introduced in 2020 to strengthen the nation’s communications infrastructure. She said her bill aims to “enhance our domestic networks by making them more resilient, redundant, and interconnected.”
“The bombing in Nashville illustrates the continued danger posed by bad actors to disrupt telecommunications networks critical to the American economy,” Blackburn said. “Preventing potentially more serious future attacks on our homeland requires deeper partnerships with the private sector.”
Protecting public safety communications
One concern for lawmakers is the impact the blast had on emergency communications, including disruptions to AT&T’S public safety network which prioritizes traffic for first responders, FirstNet.
In 2012, following recommendations in the aftermath of 9/11, Congress passed legislation creating the First Responder Network Authority (Firstnet), authorizing it to enter into a public-private partnership to build a nationwide public safety broadband network.
In 2017, Firstnet awarded a 25-year, $6.5 billion contract to AT&T to build and maintain the nationwide network for public safety, according to a Congressional Research Service report.
As of April 2019, Firstnet reported to Congress that more than 7,000 publicsafety agencies were using the network, and Verizon has a similar program that competes for contracts nationwide.
But communities across the Southeast experienced Firstnet outages after the Nashville bombing — including in Tennessee and Alabama.
Experts had pointed to the potential danger of such an incident.
“Cellular networks tend to collapse exactly when they’re needed most — in the aftermath of a disaster,” Johnathan Tal, chief executive officer of Tal Global, an international security consulting firm, wrote in a 2018 analysis in industry publication Securityinfowatch.com. “These events come in conjunction with the increased vulnerability of this very same infrastructure due to inter-connectivity and growing complexity.”
Even an incidental interruption, he said, can easily “mushroom into a colossal disruption of life and commerce.”
“Of course, network designers and security experts are aware of these vulnerabilities and have developed mechanisms and procedures to contain and abate cyber and physical interferences with smooth operations, but the situation is far from secure,” Tal wrote at the time.
In 2017, members of Congress raised questions about Firstnet and the network’s reliability and redundancy.
At the time, Chris Sambar, an AT&T senior vice president, acknowledged the company needed “a public-safetygrade network that extends from the handsets to the central office,” but given current resources, “(it’s) not reasonable to think every tower will be at publicsafety-grade level,” according to Mission Critical, an industry publication.
A year later, in 2018, all 50 states and six U.S. territories had accepted the Firstnet/at&t plan to deploy the network in their state, but some opted in reluctantly, according the Congressional Research Service report.
By the Monday after the Nashville bombing, AT&T reported the majority of services had been restored through a combination of fixes, including generator repairs and a temporary network set up at Nissan Stadium near the central office facility.
Jim Greer, assistant vice president of corporate communications for AT&T, declined to comment on the company’s security procedures following the bombing. He cited CEO Jeff Mcelfresh’s emphasis in a Dec. 29 letter to customers on prioritizing security.
“Our buildings have been damaged, but our determination to serve you and our community is undeterred. You have my commitment that we’ll continue to work around the clock until service is restored. And we will continue to prioritize the security of all our facilities that serve customers across the nation,” Mcelfresh said.
On Firstnet, Greer said Thursday that emergency responders on the scene of the bombing had access to Firstnet and the service “stayed up for the hours afterward.”
Only after the loss of power from the explosion and damage to back-up generators due to water and fire were FirstNet customers impacted, he said.
“Within hours dedicated Firstnet portable cell sites were on air in Nashville. Despite the magnitude of the event, our team had nearly all services restored in about 48 hours,” Greet said in an email. “The capability and experience of our teams was a critical difference in responding and restoring service.”
What happened on Christmas Day illustrates the interdependence of the nation’s critical infrastructure, from communications to electricity to natural gas, said Tim Conway, an industrial control systems security specialist with the SANS Institute.
“Our critical infrastructure kind of leans on each other like dominoes,” said Conway, who previously oversaw operations technology for a Northern Indiana natural gas and electric company.
“How well that environment is managed and maintained will stop it from spreading across multi-states and taking down a global network,” Conway said. “So the impact that they had, being kind of where it was contained, is a sign of how well architected their network was and how well prepared they were.”
James Yacone, chief of mission at the SANS Institute and former assistant director of the FBI for critical incident response, isn’t critical of AT&T’S response.
While the incident did hit a “crossover” point, affecting electrical supply to a telecommunications node, Yacone praised AT&T’S quick response.
“I can tell you from 30 years of responding to critical incidents, they don’t generally fit into a nice silo, and so we’re seeing two critical infrastructures affected here, really, depending on how it cascaded out,” he said. “Are there other vulnerabilities like this? Absolutely, there are ... But I do think that AT&T responded to it and got control of it pretty quickly given the cascading effect of what they were dealing with.”
Disruptions worrisome, action sought
Still, the disruptions to service were worrisome for many.
White County in Tennessee doesn’t use AT&T for its landline phone service, which allowed the 911 communications center to still accept some calls while wireless service was down.
White County officials hope Tennessee will address the impact, demand answers and improve the state’s emergency communications systems.
“I do hope that AT&T can explain what the problem (was),” Haston said. She and other 911 directors in Tennessee’s Cumberland region have had a conference call every day since the attack, but haven’t gotten “satisfactory answers from AT&T.”
The Tennessee Emergency Communications Board called a special meeting in the wake of the attack, and Haston said she and other directors plan to “see what our voices can do.”
White County’s communications was back up Wednesday following the bombing and Haston said she believes the center isn’t missing wireless emergency calls.
Haston doesn’t think dispatchers missed a serious emergency while lines were down. But, she said, there is no way to really tell.
Contributing: Natalie Allison, Yihyun Jeong and Yue Stella Yu of The Tennessean in Nashville, Brad Harper of the Montgomery Advertiser and Justin Vicory of The Clarion Ledger in Jackson, Mississippi.