Using Big Data Analytics to Produce High Quality Big Data Storage

With so much data available, using big data analytics can help control product quality and troubleshoot issues as quickly as possible

The preservation of human knowledge is of paramount importance to progress, now and in the future. And because the vast majority of new data is stored digitally, the need for reliable digital storage is greater than ever. The challenge today is ensuring that the drives mass-produced by the storage industry in order to keep up with the ever-growing need for data storage are manufactured according to the highest standards of quality. The solution to that challenge may lie in a relatively new but fast-growing field known as big data analytics.

The need for reliable data storage is particularly urgent in light of the fact that the amount of data stored every year is increasing rapidly. Indeed, much more data 60 www.dqindia.com is generated than is actually stored. For example, CERN generates close to a petabyte of data every second while particles fired around the Large Hadron Collider at velocities approaching the speed of light are smashed together. But CERN can only store approximately 25 PB of this data every year—equivalent to about 8,333 full 3 TB hard disk drives.

When a disk drive is manufactured it acts as an intelligent sensor that is aware of its own health and quality, and it stores its own sensor logs. These drives are tested for many days, and during that time, they might generate megabytes of test, diagnostic, and configuration data—as many as a1,000 variables logged for each drive. In addition, information is collected about every important