The Future of RAID
RAID is getting old. As drives get bigger, it struggles to cope. Rebuild times using parity can run into days, and reliability is falling. Really big operations, such as Google, Facebook, and Microsoft, have already moved away from RAID.
There are two approaches to improve things: to build on existing RAID levels, adding triple parity, for example, or to get clever with the math. Enter Rateless Erasure Coding. This uses mathematical functions to define a block of data as a potentially limitless sequence. The math is fiendish, but means your original data can be reconstructed from any subset of the translation equal to or slightly larger than the original—effectively, all the data becomes parity data. You can practically pick your level of redundancy. Want it secure after four failures, or more? It can be done, as can fast repairs after any loss. It’s early days, but it is promising. Microsoft uses a version of Erasure Coding in its massive Azure data centers.
More immediately pertinent for home use is the development of new file systems, which have RAID levels of data security and the flexible use of drives built in, rather than added on top.
ZFS (Zettabyte File System), originally from Sun, but now open source, includes many RAID functions, including using checksums, copy-on-write, dynamic striping, and more. It’s readily available on Linux systems. Btrfs (B-tree file system), from Oracle, is also available for Linux, and uses copy-on-write at its core. Beyond-RAID, from Drobo, offers benefits for a NAS box, although it is proprietary.
Of most interest is Microsoft’s ReFS (Resilient File System). It’s in Windows Server 2012. It will replace NTFS at some point, though it could be years before it’s the Windows standard. It brings many features of RAID right into the filesystem, with integrity checking, copy-on-write, metadata scrubbing, and more. Microsoft boasts that it offers hardware RAID levels of speed and data resiliency, and it has proved pretty capable on Server 2012. You can try it under Win 10 if you use Storage Space to create mirrored drives. So far, though, you can’t use it on your boot drive.
Will these kill dedicated hardware RAID? It’s doubtful—there will always be a need for ultra-reliable specialist storage, because some data will always be too important to lose, and some servers too important to go down.