Damnit - my cache drives failed. I'm not sure what happened, but I'm thinking it happened when I was moving the server to install my new drives. The server is on the top of a shelf and I needed to rotate it a bit to get the drives in the bays.
I had no idea that there was even a problem except that windows kept throwing out an error when I tried to create a new folder on a share that used the cache for write speeds.
I re-started the server and bam, cache needed to be re-formatted.
I do weekly backups using the handy dandy CA Appdata Backup tool. This is the first time that I'm restoring on a production machine (I check the backup occassionly to make sure its still doing its thing) so I will let you know how it goes.
It has to copy over about 700G from spinning disks before I can be sure that it worked.
This is exactly why I typically use Synology NAS for most of my production work. It is very easy to fix if there is an enclosure issue, I throw the disks in a new NAS and keep working. And if there is a disk issue I can restore from backups (including configurations) simply.
With DIY servers there isn't the same level of consistency, at least when I build them. I'm just not as good at it. But for bulk storage, I can't get the same price/TB on anything else. This machine has a total useable space of 50TB with a mix of 3, 4, 6 and 8 TB drives.
I upgrade them as they fail so they are all different ages and unlikely to fail all at once, and on top of that I have them in dual parity so I can lose two drives without losing any data.
So this is what I'm going to be doing for the rest of the afternoon.
