Why the future needs optical data centres
Researchers are leaping the hurdles holding back optical networks. Nicole Kobie reveals the technical challenges they’ve overcome – and what that could mean for the future of data centres
Researchers are now leaping the hurdles holding back optical networks. Nicole Kobie reveals the technical challenges they’ve overcome – and what that could mean for the future of data centres.
Driverless cars, AI, automated everything – these future technologies all have one thing in common: they’ll make and use huge amounts of data. And our current data centres may not be enough. “The industry is constantly innovating, but the challenge for some data centre providers is keeping up with the rate at which new technologies are emerging,” said Rob Spamer, director of data centres at Pulsant.
That’s particularly true with technologies such as neural networks, which are distributed across thousands of specialised processors that can churn through data 20 times that of a standard CPU. “Because they consumed so much more, they can communicate much more as well,” explained Georgios Zervas, associate professor at University College London’s department of electrical engineering. “You need networks that can sustain this growth.”
Data centres have kept up with the growth in data because Moore’s law has continued to hold: every couple of years, transmission speeds would double. That “law” is set to come to an end, which means we can no longer rely on faster chips to hold up network speeds. There are different ways to address the data centre quandary: build more of them, tweak setups for efficiencies or manage data better using AI. But it’s unlikely any of those ideas will be enough.
Researchers at UCL and Microsoft may have another solution: optical networks. That may not sound very futuristic, but while you may get your internet down a superfast optical line, most data centres still rely on electronic networking. That could be the significant change that brings data centres into the fast lane.
Unseen innovation
Before explorin g optical networking, it’s worth considering why data centres seem behind the curve of emerging tech. “The idea that data centre providers are slow to innovate probably comes to down the fact it can take time to implement new technologies to avoid disruption to services,” said Spamer. “Ultimately, data centre providers strive to be at the forefront of innovation but must also plan carefully to maintain service-level agreements.”
The last few years have seen serious innovation in data centres, notably the use of telemetry to fine-tune systems. “Controlling conditions such as cooling, chilling and humidity inside data centres can be a significant challenge,” Spamer said. “However, developments in telemetry are presenting data centres with the opportunity to automatically tune and control these elements, enabling optimal conditions for equipment consistently across all sites.”
Another solution is building more capacity, and while this may not solve all of our data hoarding woes, it means new technologies can be introduced for specific requirements. “With demand for rack space constantly growing, capacity can be another common challenge, which is why some providers are building surplus server halls,” Spamer said.
“As demand grows and technology evolves, those halls can be fitted out to serve customers’ higher density requirements with the latest innovations instead of subjecting customers to the time and cost that goes into optimising existing data halls that have older technology. Building data centres in phases allows providers to follow this strategy,” Spamer added.
Indeed, Spamer predicts that data centres will evolve into different types – some will remain massive where multiple partners hold data, others will become smaller, local operations. “These will essentially create a virtual bridge between centralised platforms and micro-edge locations such as base stations and masts.
“This is a trend that’s likely to continue, so the whole data centre model will become less centralised, making way for a grid-like architecture, with more providers establishing sites across various locations,” said Spamer.
In short, data centres are ripe for disruption – we just need the right tech to reboot how they operate.
Optical solution
Fully optical networks could be the solution, although there are real challenges to making it work – after all, if using faster networks was easy, they’d already be in place. At the moment, whenever a data centre needs more capacity, cloud providers and operators simply throw more electronic switches at their systems.
But with Moore’s law fading and technologies such as neural networks becoming ever more demanding, electronics can’t keep up. “If we want to create machines that have the same number of neurons as our brains, we need hundreds of thousands of processors to interconnect between them so they appear as a single machine,” UCL’s Zervas explained. “Electronics can’t do that, because they’re power hungry, take a lot of space, and impose a lot of penalties. Every time you use a network based on electronic switching, you increase the latency.”
Zervas and his colleagues believe optics can change all of this, but there are three main challenges. First, we’ll need a fast enough optical switch, as taking too long to process small data packets will negate any of the gains from going optical. Second, clocks need to be synchronised. And third, the network needs to be better managed, rather than just sending packets out and telling them to find their way as the internet works.
To begin, an optical switch needs to be superfast. Zervas says that a 125-byte data packet on a 100Gbits/sec comms link – which is what data centres tend to operate at – would take ten nanoseconds. “A switch needs to react in less than a nanosecond so the overhead is just 10%,” he said.
That’s already on the way. Researchers Chris Parsonson, Zak Shabka and Thomas Gerard at UCL have demonstrated one technique of switching as fast as half a nanosecond, an order of magnitude faster than had been done before, suggesting optical switches are on the way, though more work is needed. “The other key optical switching challenge is to design and prototype large-port-count (128 to 256 ports) switches that allow for a single server to communicate with as many others as possible yet at the speed I described above,” said Zervas.
The second challenge is clocks. “Current electronic switched networks are formed of switches with optical links and transceivers (transmitters and receivers) between them,” Zervas said. “Two transceivers on either side of the optical fibre link are in continuous communication and so their clocks can be easily synchronised so the data can be correctly recovered.”
Taking advantage of a fully optical network means having multiple transmitters talking to a single receiver. “What happens when you have two transmitters communicating to one receiver, but in the middle you have an optical switch… each has slightly different clocks in terms of frequency and in terms of phase,” Zervas explained.
There are a few ways to solve this problem. “One of the approaches we
“The whole data centre model will make way for a grid-like architecture, with more providers establishing sites”
used with Microsoft Research was to have one centralised clock that you broadcast to all servers… so they all have the same frequency,” he said. To address the phase differences, the receiver can tell transmitters which phase to operate at. “So when you arrive at my receiver, the data is going to be in phase.”
That does take time, but UCL researchers Zhixin Liu and Kari Clark managed it at fewer than 600 picoseconds, which is a 6% overhead. “It doesn’t slow you down,” he added.
The third challenge is controlling the network. Zervas compares sending packets over the internet to driving without a GPS; you follow the road signs. “You start, but you don’t know exactly which route until you see the sign,” he said. That’s fine for the internet, where a buffering video or missing pixel can be tolerated, but data centres require a guaranteed service – so such a network needs GPS. “It calculates your path but it also guarantees a space on the road, irrespective of traffic,” he said. So far, that’s been done with software, but that takes microseconds or