NetBSD and RAIDframe

(written and maintained by Greg Oster <oster@cs.usask.ca> )

RAIDframe is a framework for rapid prototyping of RAID structures. RAIDframe was developed by the folks at the Parallel Data Laboratory at Carnegie Mellon University. RAIDframe, as distributed by CMU, provides a RAID simulator for a number of different architectures, and a user-level device driver and a kernel device driver for Digital Unix.

NetBSD now supports RAIDframe as a kernel-level device driver. The only major changes from the original RAIDframe code have been the addition of a NetBSD kernel interface to the driver, and the fixing of a few bugs. The RAIDframe functionality is primarily the same as was originally distributed in the RAIDframe 1.1 distribution from CMU, with the addition of component labels, hot adding of components, and a few other things (see below for details).

Some of the features of RAIDframe in NetBSD:

All RAID levels (with the exceptions of 4 and 6) have received a lot of testing.

Work on documentation is still in-progress. Comments on current documentation are welcome.

Recent Changes

Most recent changes to the RAIDframe driver in NetBSD, and other tidbits about the RAIDframe development I've been doing include:

Project History

In case anyone cares about how such a project might evolve, here is a brief outline of some of the key points in the development effort.

FAQ

Ok, so some of these havn't been asked of me yet: I'm just doing a bit of preventative maintenance... :-)

Q Where do I get it?

RAIDframe is an integrated part of NetBSD, and is available in all NetBSD releases since NetBSD 1.4. You'll note that it's only been compiled on i386, pmax, sparc, sun3, hp300, and alpha boxes at this point [May 24, 2001: This information is rather old - it's been compiled on a lot more boxes than this.], so if you're having problems with getting it going, PLEASE send me an email ( oster@cs.usask.ca), and I'd be more than happy to help out. (This is especially true if you are attempting to get the RAIDframe kernel driver going on a platform I havn't tested it on yet.)

Q Is it stable? Will I lose data? Can I trust it?

A Mostly. Not likely. Perhaps. :-) There is absolutely no way that I can guarantee that you will not lose data from using this software. But then, that holds true for just about anything. That being said, my stress-testing has involved load averages of 64+ over only 3 SCSI disks, and not a single byte has been lost in "normal operation". I've also re-built the NetBSD world a couple of times off a filesystem living on a RAID device, so in that sense at least it's self-hosting. How well the driver fares under "component failure" is less well tested, and is the subject of current testing. Update (Feb. 14, 2000):My home machine (had an uptime of 120+ days until it suffered a 2 drive failure) has a RAID 5 setup on it that I use every day for Real Stuff. I didn't even notice that the first drive had failed :( Update (March 22, 2000) My /, swap, and /u2 partitions on my main home machine are now all on RAID 1.

Q Is NetBSD-1.3.x supported?

A No, and while a back-port should not take more than a few hours to do, I have no intention nor inclination to do it.

Q What can I do to help?

A Play with the driver, and try to break it. If folks would like to help identify more bugs for me, that would be great. Stories of success, especially successful recoveries after a component dies, would be most welcome.

Q Why are kernels with RAIDframe so huge?

A RAIDframe is not small. It typically requires at least 500K of extra space [May 24, 2000?: Gee.. this is old too... on i386, I believe the additional space required is only about 320K these days. Feb. 22, 2004: RAIDframe weighs in at 150K for i386]. This makes it tight to get it working on something like a Sun 3/50 (kernels must be less than 1MB in size), but if you're going to be running this on a server, chances are an extra 500K isn't going to be much of a problem. Doing RAID well and with as many options as RAIDframe provides takes a lot of space. I am working on pruning out unnecesary bits from the driver.

Q Can I use disks of different sizes?

A Your physical disks can be of different sizes, but each component in the RAID set must be the same size. If you have say one 9GB SCSI disk, two 4.5GB SCSI disks, and two 10GB disks, you might do something like the following:

          RAID5#1   RAID5#2
   9GB:  /dev/sd0e /dev/sd0f
 4.5GB:  /dev/sd1e
 4.5GB:  /dev/sd2e
  10GB:  /dev/wd0e /dev/wd0f (1GB unused on /dev/wd0g)
  10GB:  /dev/wd1e /dev/wd1f (1GB unused on /dev/wd1h)
where each component in each RAID set is 4.5GB. This would give you 18GB on one RAID5 set, and 9GB on the other.
           RAID5#1   RAID5#2 
   9GB:  /dev/sd0e /dev/sd0f
 4.5GB:  /dev/sd1e
 4.5GB:            /dev/sd2e
  10GB:  /dev/wd0e /dev/wd0f (1GB unused on /dev/wd0g)
  10GB:  /dev/wd1e /dev/wd1f (1GB unused on /dev/wd1g)
and end up with 2 raid sets, each with 13.5GB of total storage. (you could also pull /dev/wd0g and /dev/wd1g into a RAID 0 or 1).

If you wanted all of the partitions into a single RAID set, then you might try to pull the above two sets together with a RAID 0. For RAID5#1, you might use:

START disks
/dev/sd0e
/dev/sd1e
/dev/wd0e
/dev/wd1e
and (for RAID5#2) you might use:
START disks
/dev/wd1f
/dev/wd0f
/dev/sd2e
/dev/sd0f
Note that the drives are ordered here in an attempt to avoid having a single IO go to two different partitions on the same disk. The reality is you don't want to use more than one partition from any one disk if you are concerned about performance.

Q I just added another component to my RAID set, but when I do a newfs I havn't gained any space!

A You are probably still using the old disklabel, not one that reflects the size of your new RAID set. Just to make sure you get what you want, I recommend a:

dd if=/dev/zero of=/dev/rraid0d count=2
(that would be:
dd if=/dev/zero of=/dev/rraid0c count=2
for non-i386 boxes.) to make sure the old label is gone. Then simply do a:
disklabel raid0 > /tmp/label
{edit /tmp/label and make it say what you want}
disklabel -R -r raid0 /tmp/label
This ensures that you have a fresh label that correctly reflects the RAID configuration.

TODO

Here, in a "need to fix very soon" order, is a list of stuff on my TODO list: Here, in no particular order, is a (very small, and very incomplete) list of the stuff on my TODO list for this project: (Both of the above lists are now very much incomplete.)

Other RAID-related links


Page last modified: October 31, 2007. Send comments to oster@cs.usask.ca.