Encrypted RAID5 with cryptsetup (dm-crypt + LUKS)

Ok folks, after successfully converting a RAID1 to a RAID5 on my home server I decided to play around with the machine a little bit more.
With cryptsetup, a package available from the ubuntu repositories, I easily encrypted the whole RAID5 array in my ubuntu-server 8.04 LTS. Cryptsetup bundles the dm-crypt package used for encryption and LUKS for key management.
This is what I did:

ok first things first: I installed the cryptsetup package

me@server:~$ sudo apt-get install cryptsetup

Ater installation I loaded the corresponding modules with

me@server:~$ sudo modprobe dm-crypt
me@server:~$ sudo modprobe dm_mod
me@server:~$ sudo modprobe aes
me@server:~$ sudo modprobe cbc

Some mention to only load dm-crypt but somehow that didn't worked for me. You can check with

me@server:~$ lsmod

if they were loaded successfully.

me@server:~$ cat /proc/crypto

showed me the available crypto stuff. After unmounting the RAID5 with

me@server:~$ sudo umount /dev/md0

I could haul out the big guns

me@server:~$ sudo cryptsetup luksFormat -c aes-cbc-essiv:sha256 -s 256 -y /dev/md0

WARNING!
========
This will overwrite data on /dev/md0 irrevocably.

Are you sure? (Type uppercase yes):

of course I'm sure I have backups ;-) (yeah I'm a little picky about that).

• -s gives the key length in bit, so here it is 256bit which is also the maximum size
• -y makes sure to ask for the passphrase twice 
• -c gives the encryption parameters:
• aes is the used encryption algortihm
• cbc the block modus used
• essiv is the mode for the initialization-vector
• sha256 the hash-function which is therefor used
• and /dev/md0 is of course the device to be formatted.

Enter LUKS passphrase:
Verify passphrase:
Command successful.

What actually happened is that the header with the crypt information is written to the RAID5. Then I opened the RAID5 with

me@server:~$ sudo cryptsetup luksOpen /dev/md0 cryptraid
Enter LUKS passphrase:
key slot 0 unlocked.
Command successful.

Now cryptraid can be used like any other device (like a parttion under /dev/sdxY) on your system. The first thing I did was to make sure that no data can be restored

me@server:~$ sudo dd if=/dev/zero of=/dev/mapper/cryptraid bs=10M
dd: writing `/dev/mapper/cryptraid': No space left on device
286160+0 records in
286159+0 records out
3000597344256 bytes (3,0 TB) copied, 86115 s, 34,8 MB/s

because encrypted zeros look exactly like encrypted data. As you can see this took me a while (fyi: a day has 86400s...so you get the point). Next thing is to put a filesystem so I did a

me@server:~$ sudo mkfs.ext3 -c /dev/mapper/cryptraid
[sudo] password for me:
mke2fs 1.40.8 (13-Mar-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
183148544 inodes, 732567711 blocks
36628385 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
22357 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000, 214990848, 512000000, 550731776, 644972544

Checking for bad blocks (read-only test): done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 36 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

Thanks to the -c this took again the little while of 16h. But heh, who doesn't like to check for bad blocks? The final steps are just cosmetics. Never check again the filesystem, please, neeevvveeerrr.

me@server:~$ sudo tune2fs -c 0 -i 0 /dev/mapper/cryptraid
[sudo] password for me:
tune2fs 1.40.8 (13-Mar-2008)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds

give journal_data as mount option

me@server:~$ sudo tune2fs -o journal_data /dev/mapper/cryptraid

and set dir_index as option

me@server:~$ sudo tune2fs -O dir_index /dev/mapper/cryptraid

and make a short!!! check again

me@server:~$ sudo e2fsck /dev/mapper/cryptraid
e2fsck 1.40.8 (13-Mar-2008)
/dev/mapper/cryptraid: clean, 11/183148544 files, 5823464/732567711 blocks

To effectively use the encrypted RAID5 it needs to be mounted (lo and behold!)

me@server:~$ sudo mount /dev/mapper/cryptraid /mnt/RAID/

and done.

Because the server is a headless machine and I only use it via a remote console I didn't put a crypttab or mount points in fstab. I will simply write a script which handles luksOpen --> mount and umount --> luksClose respectively. So I can decide when to use the encrypted RAID and when not. Keep in mind that after luksOpen everyone who has access to the machine can see and work the encrypted data. Always make a luksClose after unmounting. But the management of an encrypted data server is up to everyones own degree of built-in paranoia ;-).

cheers

A few sources I used and abused were this, this and that apart from the almighty mother of doom G.