File Server, part 2: Storage Needs

Posted

After spending about an hour screaming inventorying all the data I’ll be moving to my file server, I came up with (a) some staggering figures and (b) a solid group of ZFS filesystems I’ll create to keep things organized and easily shared.

Space Requirements

My first discovery is that I require a minimum of 1 TB of space for everything. Holy cripes; shoot me now. I know I need everything, too, because I started a cleanup project a little over a month ago. I know I’ll find more to delete as some may be overlap, but not much.

One useful habit I always picked up from my former boss Mike and co-worker Keith is never to let disk usage exceed 80 percent of capacity. This is how they managed the storage pools on our IBM i5 systems, and it always worked out quite well. (Note: unlike me, they knew what they were doing.)

Consistent with that guideline, I’d like to start out my file server’s life at somewhere between 30 to 40 percent usage, so I’ll need somewhere between 2.5 and 3 TB total usable space. Since I plan to create a storage pool using RAID-Z, this means I’ll need roughly 4 TB of physical disk space since one of those disks will be used for redundancy. Excellent, that’s 33% usage, so I shouldn’t have to worry about expanding storage for a few years. Right now that’s about $400 worth of disk - not too bad, considering it’ll last me for so long. And not to mention that having centralized storage will make me wet myself regularly - can’t put a price on that.

File System Requirements

As far as ZFS filesystems go, I’ve pretty much divided everything up based on what each will hold and what machines need access to it, so I can selectively turn on or off specific access methods for those filesystems.

  • Documents: work, finances, yo-momma jokes, old e-mail archives, etc. Needed everywhere and likely treated as a secondary home directory anywhere I log on.

  • Development: tons of projects on various platforms. Needed on Windows, Mac, and Linux.

  • Web Sites: serving up live copies to a separate web server, as well as holding archived or staging copies of web sites. Needed on Windows, Mac, and Linux.

  • Software: installers, cd images containing installers, etc. Needed on Windows and Mac; potentially Linux too.

  • Media: TV/movie files, DVDs, photos, music, production files, graphics. Needed on Windows and Mac.

  • Backups: full directories, Ghost images, Mac Time Machine volumes. Needed on Windows, Mac, and Linux. May create an iSCSI-targeted zvol for Time Machine backups, but not sure yet.

  • Virtual Machines: ESXi or Xen Server virtual disks. Shared via iSCSI.

Access Methods for All Machines

Due to the heterogeneous nature of my home network (two Windows machines, two Mac OS X machines, one Linux machine), I can’t pick only one native access method. What a downer. And I want to go with access methods that are native to each OS for speed and reliability reasons. So, here’s the smrgsbord of access methods for each platform:

  • Windows: SMB.
  • Mac OS X: SMB and SFTP. Maybe NFS.
  • Linux: NFS.
  • VM Hypervisor: iSCSI (by way of ZFS zvols).

Luckily, OpenSolaris can handle all of these methods natively. The most I’ll have to do is install a package or two.

What’s Next?

Since I’m going to confirm that everything will work for every machine prior to buying hardware, I now I have the joy of setting up four virtual 1TB disk drives under my OpenSolaris instance in VMWare. Once I get them pooled and raided correctly, I’ll be setting up all these filesystems, sharing them to the other machines, copying test data to them, and reading/writing to them from all the necessary places. If everything works out, then I can go to Fry’s (or Newegg) and buy real hardware…finally!

The next part in this series will cover the initial configuration of my virtualized OpenSolaris instance – networking, packages, service configuration, etc.

Comments

Nobody has posted any comments yet. Be the first!