Big data needs the cloud
ADVANCES IN CLOUD STORAGE, ANALYSIS AND AI MAKE IT POSSIBLE
n ambitious star-mapping project highlights the growing importance of big data and the cloud. At an event in Berlin today, the European Space Agency (ESA) is unveiling the biggest set of data about the stars yet gathered. The positions and magnitudes of no less than 1.7 billion stars of our Milky Way galaxy have been gathered by the Gaia spacecraft, which took off in 2013 and began collecting data a year later.
The ship is also transmitting a vast range of additional data, with distances, motions and colours of more than 1.3 billion stars collected so far. And that is without counting temperature measures, solar system analysis and radiation sources from outside the galaxy.
“The extraordinary data collected by Gaia throughout its mission will be used to eventually build the most accurate three-dimensional map of the positions, motions, and chemical composition of stars in our galaxy,” according to a project document.
“By reconstructing the properties and past trajectories of all the stars probed by Gaia, astronomers will be able to delve deep into the history of our galaxy’s formation and evolution.”
The entire project would be impossible were it not for advances in cloud computing storage, big data analysis and artificial intelligence systems during this decade. The storage demands alone are mind-boggling.
The ESA roped in cloud data services company NetApp, which focuses on management of applications and data across cloud and on-premise environments.
NetApp was previously involved with the Rosetta space mission, which landed a spacecraft on a comet in 2016. Lauched as far back as 2004, 10 years later it became the first spacecraft to go into orbit around a comet, and its lander made the first successful landing on a comet.
“For the next two years, Rosetta was following the comet and streaming data,” says Morne Bekker, NetApp South African country manager.
“But with the comet speeding away from the sun at 120 000km/h, Rosetta would soon lose solar power. Scientists seized the opportunity to attempt what no one had ever tried before — to gather unique observations through a controlled impact with the comet. Despite blistering speeds and countless unknowns, the spacecraft landed just 33m from its target point.
“It’s quite phenomenal when