Posts Tagged ‘Storage’
A few weeks ago, the company I work for had some issues with a server used to host a central file repository. It was nothing more than a windows server with a lump of drive space shared out, and a mapped drive on all of the user workstations. Backups were made using a backup appliance (managed by a vendor) that would take incremental snapshots of every file, and store the last X revisions until the device ran out of space. At that point it would start to overwrite the oldest files. As a secondary measure, full snapshots of the data were taken every X days, to guard against a double failure of the drive and the backup appliance.
Unfortunately, because of a known (unknown to us..) issue with a software package that was installed on the server, the master file table was being corrupted. Because the files were ‘changed’, the backup device would copy over the corrupted files, overwriting the backups. I’m sure you can see where things went from there..
Forutnately, once we realized what was happening, we were able to piece together %95 of the data that was lost. The remaining data consisted of files changed/created since the last full snapshot. I found myself in a position of having to sort through literally hundreds of thousands of files trying to find the most-recent, working copy of each particular file. Not impossible, but impossibly inefficient. I came across a tool called Beyond Compare. It allows you to compare two directory structures, and shows you any files or folders that are different between the two directories. You can tell it what criteria it should use to determine if files are the same, and it highlights the entire directory structure up to the top level if there are any differences, with different colors for different discrepancies. You can even open two files side-by-side to decide exactly which version you want to keep. At that point, you are a click away from correcting the problem. I would probably still be working on this if I had not found this tool.
So now we are back in business with a working copy of the drive. But what was to keep this from happening again? After all, we shouldn’t have used ignorance as an excuse once, and we definitely couldn’t use it twice. I started thinking about file systems, and I realized that there really was no good protection for this sort of failure, other than to have a file system that builds some sort of CRC or hash into the files / directory structure itself. I wondered if such a thing existed, and thats when I found out about ZFS.
ZFS stands for Zettabyte File System. If you’re not familiar, look it up because it has some interesting features. In my situation, I was mostly interested in it’s data integrity features. I won’t go into full detail here, but it fully covers our modest needs. The question was, what is the easiest way to build a windows share on a file system that windows doesn’t support?
That’s when I came across FreeNas. It’s an open-source NAS appliance in the form of a live-CD image. It runs acceptably on almost any hardware I threw at it. It’s extremely easy to use, it uses ZFS for storage, it’s relatively stable and mature, and best of all, it’s free.
Check it out!