UAE companies must capitalise and cast light on dark data

The UAE is home to a massive accumulation of dark data in key industries, the untapped potential of which represents a business-critical opportunity, according to a new report.

Dark data – or digital information that is stored but not used – accounts for 48 per cent of the total data stored by organisations in the UAE, according to the 2018 UAE Databerg Report released last week by Veritas Technologies, a California global data management company.

“Large volumes of dark data that stays in the UAE points to the fact that managing dark data is becoming a business-critical issue for organisations,” said Damian Wilk, senior regional director for the Middle East at Veritas.

“By identifying the value of such data, organisations can move towards faster decision-making, greater operational efficiency and increased productivity.”

In most cases, businesses leave dark data dormant for practical reasons. The data might take too long to clean and any information gleaned from organising it would then be too old to be useful. In such instances, records may be incomplete or outdated, or be stored in obsolete file formats.

Dark data may include server log files that provide clues to website visitor behaviour and customer call detail records incorporating consumer sentiment data, according to TechTarget, a publisher of data security information. Mobile geolocation data can also reveal informative traffic patterns.

Bringing dark data into the light – that is, actually using it – can create new revenue streams, reduce waste and cut down on costs by providing insights that can make businesses more efficient.

Big data analytics studies large volumes of data to discover unseen links and correlations. Companies are adopting these applications to turn dark data into useful information to offer customers tailored services.

In 2016, big data analytics generated about $2 billion (Dh7.3bn) in revenue in the Middle East and Africa region, and is expected to reach $3.2bn by 2020, according to Statista.

“Data has been the most valuable commodity since the industrial revolution ... there is huge potential in the data that is yet to be managed,” said Hemesh Dogiparthi, director of technical services and support at StorIT, a Dubai specialist in enterprise data storage. He said that tapping into dark data could address pressing matters like climate change, global hunger and poverty.

Clean data – data that is useful to businesses – has more than doubled from 8 per cent in 2016 to 19 per cent this year in the UAE, according to the Databerg report. That has led to a significant reduction in socalled “redundant, obsolete or trivial data” from 43 to 33 per cent in the past two years.