STO
Storage
Goals of this lab:
 Learn about how various common RAID levels work
 Gain experience configuring software RAID
 Learn about logical volume managers
 Gain experience configuring a typical volume manager
Prerequisites: LXB
REVISION: 2.1 [2015-08-28]
©2007-2015 DAVID BYERS
Table of Contents
Part 1: RAID ......................................................................................................................................................... 3
Part 2: LVM2 ....................................................................................................................................................... 4
Part 3: Putting it into practice............................................................................................................................. 5
Part 4: Q&A ......................................................................................................................................................... 5
IDA/ADIT
STO: STORAGE
IDA/ADIT
STO: STORAGE
PREPARATION
Complete these exercises before starting with the main lab. On-line documentation should be sufficient
to answer all the questions.
Exercise 1: Very basic RAID theory
1-1
What does RAID stand for?
1-2
What does JBOD stand for?
1-3
Explain roughly how RAID-0 works.
1-4
Explain roughly how RAID-1 works.
1-5
Explain roughly how RAID-5 works.
1-6
Explain roughly how RAID-6 works.
1-7
What is RAID 1+0 (sometimes known as RAID 10)?
1-8
What is RAID 0+1 (sometimes known as RAID 01)?
1-9
What is a hot spare?
Report:
No report required.
Exercise 2: Volume management with LVM2
2-1
Explain the following concepts: physical volume, volume group, and logical volume and how
they fit together.
2-2
The names of all commands that manipulate physical volumes start with the same two letters.
What are they?
2-3
The names of all commands that manipulate volume groups start with the same two letters.
What are they?
2-4
The names of all commands that manipulate logical volumes start with the same two letters.
What are they?
Report:
No report required.
Exercise 3: File systems
3-1
What is the main difference between an ext2 and an ext3 file system?
3-2
Which command is used to create a new ext3 file system?
3-3
If you want to increase the size of an ext3 file system, which command should you use?
3-4
What does the f s c k command do?
Report:
No report required.
IDA/ADIT
STO: STORAGE
1
MAIN LAB
One of the most important reasons for having a server is to store files on it, and managing storage is one
of the most critical tasks in many systems. If a storage system goes down, it often means that the
system becomes useless. This lab covers some of the basics of modern storage management.
m
Throughout this lab, we refer to /dev/ubdX, where X is a number. Depending on the version of UML you
are using, these may be named /dev/ubdN, where N is a letter. This does not change anything significant
– you still have the same number of devices, so mentally just map each number in the text to a letter
(except the letter a, which is your system disk).
Past problems: This lab is usually fairly straightforward. The most common problem is due to rebooting
at the wrong moment, and Linux automatically detecting a RAID set. Should this happen you should
disable automatic detection of RAID sets until you finish the lab (then re-enable it).
Part 1: RAID
Linux (like most operating systems) supports software RAID. In Linux, you control software RAID using
the mdadm command (in the past, a toolset called raidtools was the standard, but it has been mostly
replaced by mdadm).
Using mdadm, you create new block devices (named e.g. /dev/md0, /dev/md1), which represent various
configurations of other block devices. Since software RAID doesn’t really care what the underlying block
devices are, you can create RAID sets consisting of other RAID sets, of disks, or even data on other hosts
(by using network block devices).
Your server has been configured with four extra block devices, /dev/ubd2, dev/ubd3, /dev/ubd4, and
/dev/ubd5, each 5MB in size, which will be used for experiments with RAID.
Exercise 4: Configure RAID 0
4-1
Combine /dev/ubd2 and /dev/ubd3 into a RAID 0 set named /dev/md0
4-2
Create a file system on /dev/md0 and mount it on /mnt. How much space is there in the file
system?
Report:
No report required.
Before proceeding with the next exercise, you will have to unmount /dev/md0 and stop it with mdadm
in order to make the devices available again.
Exercise 5: Configure RAID 1
5-1
Combine /dev/ubd2 and /dev/ubd3 into a RAID 1 set named /dev/md0.
5-2
Create a file system on /dev/md0 and mount it on /mnt. How much space is there in the file
system? Why?
5-3
Add /dev/ubd4 as a spare in /dev/md0, then fail /dev/ubd3 (using mdadm). What happens?
IDA/ADIT
STO: STORAGE
3
Report:
No report required.
Before proceeding with the next exercise, you will have to unmount /dev/md0 and stop it with mdadm
in order to make the devices available again.
Exercise 6: Configure RAID 1+0
6-1
Combine /dev/ubd2, /dev/ubd3, /dev/ubd4, and /dev/ubd5 into a RAID 1+0 device named
/dev/md0. You may need to create intermediate raid sets to do this.
6-2
Create an ext2 file system on /dev/md0 and mount it on mnt. How much space is there in the
file system?
Report: No report required.
Before proceeding with the next part, you will have to unmount /dev/md0 and stop it with mdadm in
order to make the devices available again.
Part 2: LVM2
LVM is a volume manager for Linux that allows you to work logical, rather than physical, volumes.
Logical volumes allow a degree of flexibility that physical volumes simply cannot offer. Each operating
system has its own way of handling logical volumes, but most work more or less like LVM. In the future,
however, we can expect to see logical volume management, RAID, and the file system itself merged, as
Sun has done with ZFS in Solaris.
Exercise 7: Create logical volumes
7-1
Create physical volumes on /dev/ubd2, /dev/ubd3, and /dev/ubd4.
7-2
Create a single volume group named vg1 containing all physical volumes. Since you will be
working with very small volumes, make sure you set the physical extent size appropriately.
7-3
Create two logical volumes (lvol0 and lvol1), each 2MB in size. This will create devices named
/dev/vg1/lvol0 and /dev/vg1/lvol1, which can be used just like any other block device.
7-4
Create ext2 file systems, i.e. file systems without journals (without the –j option) on both
logical volumes, and mount them as /lv0 and /lv1. If you create filesystems with journals, then
there will be hardly any room for files, and the remaining labs won’t work.
Report:
No report required.
One of the advantages of using logical, rather than physical, volumes, is that they can be resized.
Additionally, if the volume group does not have sufficient free space, additional physical volumes can be
added.
Exercise 8: Manipulate volumes
8-1
Attempt to create a 2.5MB file on /lv1. This should fail.
8-2
Resize lvol0 to 3MB in size. Don’t forget to resize the ext3 file system that it contains as well
(using resize2fs). Mount it and attempt to create a 2.5MB file again. This time it should work.
8-3
Add a new physical volume to vg1. You can use /dev/ubd5 for this.
8-4
Migrate any data off the physical volume /dev/ubd2 to other volumes in vg1 and remove
/dev/ubd2 from vg1. You can now use /dev ubd2 for something else.
Report:
No report required.
Most logical volume managers support snapshots, which are copies of a logical volume as it appeared
when the snapshot was created. Snapshots are created by marking the volume as copy-on-write, which
IDA/ADIT
STO: STORAGE
4
makes it possible to create snapshots instantaneously. Snapshots have many applications, but the most
common is probably to make backups of a file system without having to take it off-line. First a snapshot
is created, and then a backup is made of the snapshot.
m
When a snapshot is made, only blocks that have been written to disk are actually included in the
snapshot. Since the operating system typically delays writing data to disk (for performance reasons), a
careless snapshot can miss data. The procedure in the exercise below is not recommended for
production systems.
Exercise 9: Snapshots
9-1
Copy some files to the /lv1 directory (it doesn’t matter which files).
9-2
Use the s y nc command to write any buffered data to disk, and then create a snapshot of
vg1/lvol1.
9-3
Run f s c k to check the consistency of the new snapshot (because you copied a live file
system, it is likely that f s c k will have to fix something), then mount it as /sn
9-4
Edit the files in /lv1 and then compare them to the corresponding files in /sn.
9-5
Unmount /sn and remove the snapshot from the volume group.
Report:
No report required.
Part 3: Putting it into practice
In this part of the lab you will use RAID and LVM to create directories that are needed in later labs (in
particular in the NFS lab). Before proceeding with this exercise, you need to remove all logical volumes,
volume groups, and physical volumes, created in the previous part, and any RAID devices created in the
first part of this lab.
Exercise 10: Create home directory volumes
10-1
Use RAID 1 to create a device on which you place an ext2 file system optimized for many
small files, that you mount as /home1 on the server. Make sure that /home1 is correctly
mounted at boot.
10-2
Use LVM to create a device on which you place an ext2 file system optimized for a smaller
number of large files, that you mount as /home2 on the server. Make sure that /home2 is
correctly mounted at boot.
Report:
No report required.
Part 4: Q&A
This lab doesn’t require any reports, so let’s ask some theoretical questions instead.
Exercise 11: Some theory
11-1
Why is RAID 1+0 generally considered more reliable than RAID 0+1?
11-2
As individual disk sizes grow, RAID-5 is no longer considered useful, and RAID-6 is also rapidly
becoming obsolete. Why?
11-3
What is the point of the file system journal in e.g. ext3?
11-4
Explain the purpose of using a volume manager, like LVM.
11-5
Optional: compute the probability of data loss over three years in a RAID-6 array consisting of
24 6TB SAS disks, with two hot spares (i.e. 22 disks in the RAID set). The RAID controller has
IDA/ADIT
STO: STORAGE
5
150MB/s throughput of which 80% can be used to rebuild. Assume MTBF of 145000 hours,
latent sector error of 10E-13.
Report:
IDA/ADIT
Answers to the questions above.
STO: STORAGE
6
FEEDBACK FORM
STO
Complete this feedback form individually at the end of the lab and hand it to the lab assistant when you
finish. Your feedback is essential for improving the labs. Each student should hand in a feedback form.
Do not cooperate on completing the form.
You do not need to put your name on the feedback form. Your feedback will be evaluated the same way
regardless of whether your name is on it or not. Your name is valuable to us in case you have made and
comments in the last section that need clarifications or otherwise warrant a follow-up.
For each section, please rate the following (range 1 to 5 in all cases).
 Difficulty: Rate the degree of difficulty (1=too easy, 5=too difficult)
 Learning: Rate your learning experience (1=learned nothing, 5=learned a lot).
 Interest: Rate your interest level after completing the part (1=no interest, 5=high interest).
 Time: How long did the part take to complete (in minutes)?
Difficulty
Learning
Interest
Time
(minutes)
Preparation
Part 1:RAID
Part 2:LVM2
Part 3:Putting it into practice
Part 4: Q&A
Overall
Please answer the following questions:
 What did you like about this lab?
 What did you dislike about this lab?
 Make a suggestion to improve this lab.
IDA/ADIT
STO: STORAGE
FEEDBACK FORM
STO
Complete this feedback form individually at the end of the lab and hand it to the lab assistant when you
finish. Your feedback is essential for improving the labs. Each student should hand in a feedback form.
Do not cooperate on completing the form.
You do not need to put your name on the feedback form. Your feedback will be evaluated the same way
regardless of whether your name is on it or not. Your name is valuable to us in case you have made and
comments in the last section that need clarifications or otherwise warrant a follow-up.
For each section, please rate the following (range 1 to 5 in all cases).
 Difficulty: Rate the degree of difficulty (1=too easy, 5=too difficult)
 Learning: Rate your learning experience (1=learned nothing, 5=learned a lot).
 Interest: Rate your interest level after completing the part (1=no interest, 5=high interest).
 Time: How long did the part take to complete (in minutes)?
Difficulty
Learning
Interest
Time
(minutes)
Preparation
Part 1:RAID
Part 2:LVM2
Part 3:Putting it into practice
Part 4: Q&A
Overall
Please answer the following questions:
 What did you like about this lab?
 What did you dislike about this lab?
 Make a suggestion to improve this lab.
IDA/ADIT
STO: STORAGE