Booting Ubuntu 10.10 Server from software RAID

I recently switched my home server from Windows Server 2003 to Ubuntu 10.10. The machine is an older Shuttle box I’ve had tucked away in various places over the past few years.

The plan was to disable the BIOS “fakeRAID”, install Ubuntu on one of the drives, copy my data over to it and then have the other disk join the new Linux software RAID array.  I figured out the procedure on a VirtualBox before attempting it on my actual machine.

Install the server with NODMRAID and partition the drive with no LVM, that is partition with “Guided – use entire disk”. We can’t have devicemapper getting in the way.

Install on first hard disk (/dev/sda).  If there’s any data you need off the other disk, now’s the time to mount it and copy it over.

To start, we need the mdadm package.

sudo apt-get install mdadm

You need two modules to be loaded at boot-time, raid1 and md.  In the kernel included with Ubuntu 10.10, md was compiled in, so I didn’t need to worry about loading it as a module.

I verified this by checking the kernel’s config:

fgrep CONFIG_MD /boot/config-$(uname -r)

Add the module raid1 to the /etc/raid1

sudo nano /etc/modules

Let’s load the raid1 module

sudo modprobe raid1

To ensure it is loaded:

lsmod | grep raid1

So, it seems we’re ready for prepare our other disk. A neat little trick is to use sfdisk to dump the partition table of one disk to configure another.

sudo sfdisk -d /dev/sda > sda.out
sudo sfdisk /dev/sdb < sda.out

but sfdisk responds with “I don’t like these partitions – nothing changed.”  Well, excuse me, sfdisk, but I don’t like your attitude. (Yes, I’m arguing with a disk utility now) So, I added the –force flag.  It grudgingly copied the partition table for me.  I verified it by comparing the output of sfdisk -l /dev/sda and sfdisk -l /dev/sdb.

Change the partition type of the Linux partition(s) to “Linux raid autodetect”

sudo sfdisk --change-id /dev/sdb 1 fd

We’re ready to create the array. We specify a RAID 1 array with 2 devices. The first is missing and the second is /dev/sdb1:

sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/sdb1

Now, we need an mdadm.conf. It seems like the way to do this is with

sudo cp /etc/mdadm/mdadm.conf /etc/mdadm/mdadm.oldconf
sudo cp /etc/mdadm/mdadm.conf mdadm.conf
mdadm --detail --scan >> mdadm.conf

It will put in a parameter metadata=00.90 that will cause warnings later. Apparently, this is a bug and it is safe to remove it.  After editing it, I copied back to where it belongs.

sudo cp mdadm.conf /etc/mdadm/mdadm.conf

Now, we format the raid volume

mkfs -t ext4 /dev/md0

Find out the UUID of the array

sudo blkid

Make a copy of /etc/fstab in your home directory and change root to mount to the UUID of the array.  Don’t change the real /etc/fstab quite yet.

Add a custom setup in GRUB2, I’m going to edit a copy of this file in my home directory too.

cp /etc/grub.d/40_custom 09_swraid1_setup
nano 09_swraid1_setup

Details here, but here’s what I added to my file:

menuentry 'Ubuntu, with Linux 2.6.35-22-server' --class ubuntu --class gnu-linux --class gnu --class os {
        recordfail
        insmod raid
        insmod mdraid
        insmod ext2
        set root='(md0)'
        linux   /boot/vmlinuz-2.6.35-22-server root=/dev/md0 ro   quiet
        initrd  /boot/initrd.img-2.6.35-22-server
}

This differs from the linked article in that ’2.6.35-22-server’ is my kernel release version (the one that ships with Ubuntu server) and I’m booting from md0.

Back to the physical console.

sudo telinit 1

Copy the above temp files (fstab and 09_swraid1_setup) to their actual locations

update-grub

Adjust ramdisk

update-initramfs -u

Ensure GRUB2 is on /dev/sdb and sda

grub-install /dev/sdb
grub-install /dev/sda

Mount the array on /tmparray

mount /dev/md0 /tmparray
rsync -aqxP / /tmparray

Restart and we have now booted off the array!

Change the partition type of sda now

sudo sfdisk --change-id /dev/sda 1 fd

Add it to the array:

mdadm --add /dev/md0 /dev/sda1

Welcome to the party, /dev/sda! Let’s watch it sync:

watch cat /proc/mdstat

Our special GRUB entry is no longer needed

rm -f /etc/grub.d/09_swraid1_setup
update-grub
update-initramfs -u

Done.

I worked out the kinks in the virtual machine and documented it in a draft of this post.  I used it as my guide when performing the install on the actual machine. It worked perfectly.

Update, Feb 25, 2011: Clarification in response to comments.

This entry was posted in Linux and tagged , , . Bookmark the permalink. Follow any comments here with the RSS feed for this post. Both comments and trackbacks are currently closed.

4 Comments

  1. Jason
    Posted February 18, 2011 at 8:07 am | Permalink

    Sure it’s obvious to most readers, but just for clarity, I’d add in this section:

    sudo sfdisk -d /dev/sda > sda.out
    sudo sfdisk /dev/sdb < sda.out

    that the sda.out should be edited to reference sdb before the second sfdisk command.

    Great walkthrough! Worked perfectly on my Dell R210 once I disabled the raid card in bios.

    • Posted February 25, 2011 at 10:36 pm | Permalink

      Hi Jason. That didn’t occur to me. I got this:

      sfdisk -d /dev/sda | sfdisk /dev/sdb

      from this article and expanded it to two commands because when I first ran it and got the, “I don’t like these partitions” complaint, I wanted to see what was being passed. I was a little paranoid about it, so I left it that way in the post. Changing the file in between never occurred to me. It worked anyway. I wonder if it is the cause of sfdisk not liking the partitions?

  2. Dewie
    Posted February 25, 2011 at 4:53 pm | Permalink

    First off, thank-you very much! I was able to follow this write-up (along with Jason’s comment above) to setup my RAID1 drives.

    Just a couple things you may wish to add in to make your article a bit more comprehensive:

    sudo sfdisk -d /dev/sda > sda.out
    For whatever reason I had to become a superuser before I was able to execute this/createe the sda.out file with this command. I could create it manually but couldn’t get this to work until I executed:
    sudo su

    sfdisk with the –force flag is:
    sudo sfdisk /dev/sdb < sda.out --force

    09_swraid1_setup
    This is a new file which will also live in the /etc/grub.d/ directory

    Command to mount the array on /tmparray:
    mount /dev/md0 /tmparray

    I am a complete Linux n00b and these were the largest stumbling blocks for me.

    • Posted February 25, 2011 at 9:58 pm | Permalink

      I would only consider myself an intermediate Linux user. I used it heavily for a time in the mid 1990′s, but fell comfortably back into the Windows world for a long time. I’ve been peeking at it by installing various distos from time-to-time and I think Ubuntu has reached the point where you can just use it without necessarily needing to be an expert.

      The post was largely written as notes to myself, which is why it might not be as clear to others. Reading it over again, I see how I might clarify it further.

      Thanks for the feedback and I’m glad it was helpful.