Quake may have zapped SA’s internet

Search for answers as access to global web interrupted for many

● South African internet service providers (ISPs) and their customers had a rude awakening 10 days ago when they discovered their “guaranteed” access to the global internet can be undone by seismic activity at sea.

Although there is no explanation yet for freak breaks in the undersea cables that provide international access to SA’s internet users, the most common causes are damage by ships’ anchors and earthquakes at sea.

However, the freak occurrence of two separate cables being cut simultaneously far out at sea, as happened on the morning of January 16, can only be explained by sea-bed activity.

One of the cables was cut in two places, and it is widely believed that a third major cable was also cut.

The cable damage mostly occurred in or near an area called the Congo Canyon, which starts inland and extends 220km out to sea. It is known for having the world’s strongest “turbidity currents”, underwater sediment avalanches stretching over hundreds of kilometres, which are known to destroy cables.

The most likely culprit is a 5.6 magnitude earthquake that struck the Atlantic Ocean near Ascension Island shortly before the cables were cut on the morning of January 16. The earthquake occurred just before 8am South African time, and local ISPs reported losing international access from just before 10am.

The epicentre of the quake was more than 1,000km off the coast of Africa, but disturbances caused by seismic activity at sea become more powerful as they approach the coast. Combined with turbidity currents, this could well have taken out cables in the area.

The West Africa Cable System (Wacs) was cut in two places, and the South Atlantic 3 (SAT3) cable in one location. Industry insiders say they believe that the Africa Coast to Europe (ACE) cable was also cut, but this has not been publicly confirmed.

SA is connected to the global internet via seven such cables, with a total capacity of 42.3 terabits per second (tbps). These cables, in turn, connect to additional cables connecting the west and east coasts of Africa, with a single cable running from Angola to Brazil providing another 40tbps.

However, it emerged in the past week that smaller ISPs in SA had bought capacity on only one or two cables. In a freak occurrence, two of the most commonly used cables, the Wacs and SAT3 cables, were cut simultaneously, plunging millions of internet users into data darkness.

Customers of the major mobile network operators — Vodacom and MTN — were largely unaffected, as these tend to have both partownership and access to most of the cables running up both the east and west coasts of Africa.

However, many ISPs use one of the two cables for their main access, and the other as a backup. It is unprecedented for both to go down, and ISPs such as Afrihost, Webafrica,

Axxess, and Mind The Speed were caught flat-footed. Internet users have been scathing in their criticism of their ISPs, but the reality is that access to each additional backup cable adds significantly to the cost of the service.

“We had complete redundancy on SAT3, should our Wacs capacity fail,” Afrihost CEO Gian Visser told Business Times.

“It was unprecedented that two completely geographically separate cables run by completely separate companies would fail within hours of each other. We also had emergency capacity on a third undersea cable, but this was not sufficient due to the total traffic routed to that cable by other ISPs in the same situation.

“When the severity and the time it would take to repair the cable breaks became apparent

Unprecedented that two completely geographically separate cables would fail within hours of each other Gian Visser

Afrihost CEO

we decided we needed to procure and set up more capacity on a completely different cable as soon as possible.”

The cost implications are massive, he said. “Having dedicated capacity, whether you use it or not, on any cable is extremely costly, so ensuring available capacity on multiple cables significantly increases the cost of providing connectivity to clients.

“On top of this, having a single route makes networking much easier to manage, in terms of the best client experience, so this means that a large portion of your cost is spent on unused capacity,” Visser said.

“In every situation in the past, having full redundancy on a completely separate cable was always sufficient.

“The incredibly unlikely scenario of two separate line breaks on two separate cables on the same day led to both redundant paths being compromised.”

Afrihost uses Wacs as its primary route and SAT3 as a secondary route. At present it is routing traffic over the Seacom and EASSy cables that run up the east coast of Africa.

Rain, an independent ISP that was the first to roll out 5G wireless broadband in SA, was marginally affected initially. However, said CEO Willem Roos, it buys capacity from multiple cable providers so that it has redundancy available to handle such situations.

“If the agreements are in place, failover [switching to a standby system] to other cables is a fast and simple process,” he said. “We have also designed our network to take maximum advantage of local peering and caching in order to minimise international traffic. We are pleased that our redundancy strategy worked well. However, as you would appreciate, it is costly to reserve this type of capacity.”

Vodacom spokesperson Byron Kennedy sympathised with the plight of smaller ISPs, pointing out that massive resources are needed for a guaranteed service.

“Success factors for a resilient network include elimination of single points of failure, equipment and infrastructure redundancy, and sufficiently dimensioned, geographically diverse routes able to withstand multiple simultaneous failure scenarios,” he said. “It takes weeks to implement fixed telecommunications links involving various multinational operators … All of this needs to be in place, together with efficient traffic engineering, to ensure all routes are available at all times to carry traffic.

“Restoration requiring human intervention and co-ordination among operators in different countries can be very time-consuming and inefficient. It requires significant resources and prior investment to ensure a carrier-grade service at all times.”

It was essential for carriers such as Vodacom to invest in such resources, he said, since more than 80% of the internet content in SA is typically sourced from Europe and North America.

MTN, the single biggest investor in the Wacs cable, also has a stake in the ACE and EASSy cables. It was briefly affected by the outage, said MTN SA’s executive for corporate affairs, Jacqui O’Sullivan.

“MTN SA currently uses the EASSy cable system, which has sufficient capacity to accommodate the international bandwidth demand,” she said.

“The shift in traffic onto the EASSy cable system resulted in an increase in international latency and a degradation in throughput for fixed-line subscribers, which was mitigated shortly after the Wacs outage. No mobile subscribers were affected.”