Have patience
The customer rang up very early one morning.
Not terribly happy, wanted to do his accounts run but the server wasn’t showing him any of its drive shares. He sounded somewhat panicky, as he had taken the advice to have two distinct volumes kept in sync on different devices hooked up to the server – and both devices were offline together. This is not supposed to happen.
As is often the case, the basic culprit was Windows Update, which can take some storage offline without actually telling you what’s happened. In this case, the two volumes were iSCSI resources presented by a couple of small NAS boxes. I’ve had this issue at home myself, and the fix was remarkably simple: all you had to do was go into the Disk Manager and put the volumes online “manually”. That is, by clicking the little volume blob and using the context menu. Only thing was, on this occasion, the drives didn’t come back online.
It turns out the problem wasn’t with Windows at all. The problem was that both NASes were in perfect sync – same runtimes, same versions of their OS, even their history of power cuts was the same. This meant that when the customer started “checking” the connections, he even thought it might be a good idea to pull the power off the devices, let them cool down and start over.
Why is this a bad idea? Because he’d read my suggestion about buying brand-new, large-capacity drives for older but serviceable NAS devices. So whereas, when new, these machines might have had a 1TB drive or two in them, now each one had four 6TB storage-biased SATA drives. In normal operation this doesn’t matter a bit, but when a NAS has an unexpected power loss and restart, they can decide to throw a sulk while checking all their disks. And when I say sulk, I mean 18 hours! No neat warning pop-up, no handy button in the UI to tell it to skip that check. Grind, grind, for 18 hours.
This is why I spread my storage options at home. Using iSCSI is fine, but giving it only one NAS type to chew on is just transferring your hostage to fortune from one villain to another. Here, I have a Windows server with a little bit of local disk and a lot of NAS gigabytes accessed over the iSCSI Initiator. But I also have a FreeNAS implementation on a middle-aged Dell PowerEdge T110 server box, and even a Mac Pro using the last decent version of OS X Server. Each may have their foibles (anyone who understands default ACLs on FreeNAS, fancy some consultancy?), but at least those foibles are distinct and don’t overlap badly.
The only remedy for this rather bowelloosening shock behaviour is to acknowledge that you need more horsepower in your storage machinery. My aforementioned T110 is long in the tooth, I grant you, but on the other hand it has a reasonable Core i3 CPU and came with 48GB of RAM, which appears to give it ample cache to speed up operations on files that the little baby NAS devices can be chewing away at for more than a working day. I think my early spring project list is going to include a refresh of these technologies, assuming that running my storage all together on the Dell box is going to give me a life without the kind of agonising unexpected lockouts experienced by my client!