LVM: Logical Volume Management, Version 2.NOTE: Version 2 is recommended if you are running a 2.6 kernel.
- 1 What is it?
- 2 Requirements
- 3 Setting it up
- 4 Using it
- 5 FAQ
- 5.1 How to use a LVM devices on a different machine:
- 5.2 A disk has died, how do I get the data off?
- 5.3 How do I build a mirrored LVM system with Debian?
- 5.4 My system can’t find the volume groups!
- 5.5 When setting up a volume group, half of the space is gone, whats up with that?
- 5.6 What is the maximum size of logical volumes?
- 5.7 What filesystem is best with LVM?
- 5.8 pvmove gives “device-mapper: error adding target to table” errors
- 5.9 My logical volumes are inactive and unusable!
- 5.10 My logical volumes are not active or usable on restart!
- 5.11 The system is not recognizing my boot LVM partitions and kernel panics.
- 5.12 How do I see if I have LVM support available?
- 5.13 How do I go from LVM1 to LVM2?
- 5.14 How do I go from a 2.4 kernel to a 2.6 kernel?
- 5.15 How do you get device mapper in the 2.4 kernels?
- 5.16 Glossary
What is it?¶
Normally /dev/hda is a drive, and /dev/hda1 is a partition of that drive. In LVM, those are called Physical Volumes (PV).
You put this PVs into a Volume Group (VG). From that Volume Group, you carve out Logical Volumes (LV). LVs act like partitions.
so it looks like this:
hda1 hdc1 (PVs on partitions or whole disks) \ / \ / diskvg (VG) / | \ / | \ usrlv rootlv varlv (LVs) | | | ext2 reiserfs xfs (filesystems)
2.4 kernel with device mapper patch, or 2.6 kernel with device mapper enabled. The stock debian 2.6 kernels have device mapper support.
Install the lvm2 user tools:
# apt-get install lvm2
Setting it up¶
Create Physical Volume¶
For a partition:
bc. # pvcreate /dev/hdb1
Or for a whole disk:
# pvcreate /dev/hdb
If adding an entire disk to a physical volume, the disk must not have a partition table.
Create Volume Group¶
# vgcreate testvg /dev/hda1 /dev/hdb1
Test to make sure it worked:
# vgdisplay testvg
Create Logical Volumes¶
To create a 1500MB linear LV named ‘testlv’:
root> lvcreate -L1500 -ntestlv testvg
Test to make sure it worked:
root> lvscan -v
Create the filesystem:
root> mke2fs /dev/testvg/testlv
Mount the new LV:
root> mount /dev/testvg/testlv /mnt
to reduce spool by 1 gig:
umount /dev/sympavg01/spool resize_reiserfs -s -1G /dev/sympavg01/spool lvreduce -L -1G /dev/sympavg01/spool mount -treiserfs /dev/sympavg01/spool /home/sympa/spool
what this is doing:
- unmount the device!
- make filesystem by 1 gig smaller
- make volume 1 gig smaller
- mount again
lvextend -L +1G /dev/sympavg01/archives resize_reiserfs -f /dev/sympavg01/archives
what this is doing:
- make the logical volume 1 gig bigger
- grow filesystem to fill the volume size
With an ext2/3 filesystem, it might be like this:
lvextend -L +300M /dev/mirror_vg/var_lv resize2fs /dev/mirror_vg/var_lv
This will grow /dev/mirror_vg/var_lv by 300 megabytes, and then make the ext2 or ext3 filesytem on that device grow to fill the device size.
removing a volume¶
lets say you have /dev/hda and /dev/hdb that are a part of a volume and you want to take
out /dev/hda if you have enough space on /dev/hdb you can issue a command which will move
all data off of /dev/hda (pvmove /dev/hda) that will move everything from /dev/hda onto
the rest of the disks in the volumegroup (in this example /dev/hdb)… it isn’t a fast
thing as it is a block copy, but you can see what is going on with a -v… then once it
is removed you can remove the disk from the volume group with a vgreduce then you can put a new disk in and redistribute the moved data to the new disk
copying over an existing partition¶
ok, this is not exactly an lvm issue, but it comes up a lot.
Lets suppose we have a new LV which we want to become our “usr” partition.
- mount the new partition on /mnt
- kill off any process which has files locked on /usr
- use find and cpio to transfer files from /usr to /mnt
- remove files in /usr/
- remount /mnt as /usr
it might look like this:
# mount /dev/vg1/usrlv /mnt # lsof /usr # kill <some pids> # cd /usr # find -xdev | cpio -vdamp /mnt # umount /mnt # cd /usr # rm -r * # cd / # mount /dev/vg1/usrlv /usr
This is a list of questions that I’ve seen come across the LVM mailing list
that deal with everything from how do I restore this LVM partition that
failed because one of the underlying disks died, to how to recover from
failures etc. It is mostly a random assortment of things that I’ve pulled
together, and is still a work in progress.
How to use a LVM devices on a different machine:¶If you have LVM setup on disks on machine A, and it goes down and you want to take the drives out and put them in machine B, what do you need to do to prep machine B beforehand to deal with this? You don’t need to do anything apart from vgscan/vgchange -ay when
you need to use the volumes, then mount them. vgscan/vgchange reads
the metadata off the disk and creates the necessary volume groups and
device files on the system. In otherwords, /dev/lvg1/lvol1 will be
created on machine B based on the metadata. Do not re-do vgcreate or
lvcreate because this will mangle the meta data for the other node.
A disk has died, how do I get the data off?¶Try dd_rescue www.garloff.de/kurt/linux/ddrescue (either to another disk of the same size, or to a file which can be burned to a CD or mounted in a loopback)
How do I build a mirrored LVM system with Debian?¶
- Start Debian install
- Create 20MB /boot filesystem/partition at the start of the disk.
- Then create a “temporary” root filesystem/partition the size that you want swap to eventually be (minimum 200MB or so to get minimal debian install onto it)
- Then create one partition for all remaining space
- Do not specify to mount /boot partition during installation so that /boot files are put on “temporary” root filesystem
- Do minimal Debian install (no to tasksel and dselect) into 250MB partition, specify no swap partition
- When all done installing truly minimal install, put second disk
- Use sfdisk to duplicate partition table to second disk
- Create raidtab mapping partition(s) 1, 2 and 3 to md0, md1 and md2 respectively
- mkraid /dev/md0
- Mount /dev/md0 on /mnt and copy /boot to it, remove /boot/* and mount /dev/md0 on /boot
- mkraid /dev/md2
- pvcreate /dev/md2
- vgcreate /dev/md2
- lvcreate root, mkfs … /dev/…/root
- lvcreate usr, mkfs … /dev/…/root
- lvcreate …
- mount /dev/…/root /mnt, /dev/…/usr /usr, …
- copy -rax / /mnt, cp -rax /usr /mnt, …
- Create initrd to reflect above — I use Adrian Bunk’s initrd-tools
- Edit /etc/lilo.conf to reflect /dev/md0 is boot partition and lvm lv as root partition
- edit fstab to reflect above changes
- mkraid /dev/md1
- add /dev/md1 to fstab as swap partition and swapon -a
My system can’t find the volume groups!¶
vgscan tells me no volume groups are found
lvscan — ERROR: “/etc/lvmtab” doesn’t exist; please run vgscan
vgscan — ERROR “creating lvmtab directory” creating lvmtab
vgscan — no volume groups found
mount -o remount,rw /
then check for /etc/lvmtab and /etc/lvmtab.d
ls: lvmtab.d: Input/output error
boot the machine with a boot disk and fsck root! Or do touch /forcefsk
and reboot single-user. Fsck says its clean? Try fsck -f
inactive pv hda10 is associated to unknown volumegroup
sympavg01 (run vgscan)
run vgscan, this will create lvmtab.d
then run lvscan
volumegroup is not active try lvscan -D, do a vgchange -a y
(instead of lvscan -D because lvscan -D will only scan, not activate)
When setting up a volume group, half of the space is gone, whats up
When you do the vgcreate command it says how large the physical
extent sizes are, and there is always one extent that is reserved at
the beginning of the partition for the LVM metadata. So if your
physical extent size is 32MB, then you will lose 32MB for the
metadata. Also, LVM volumes have to be multiples of the extent size
(32MB, 64MB, 96MB, etc.), so if you make a volume that is 93megabytes,
that is 3MB too little for a 3 extent volume, which leaves you with a
2 extent volume totalling 2 * 32 MB = 64MB.
What is the maximum size of logical volumes?¶
It depends on which version of LVM you are using. If you are
using LVM1, then there is a 255G limit on LV sizes (this is based on
PE size). If you use the
LVM2 metadata format, then you are limited only by disk space and the
maximum size that your kernel supports. If you create PVs in LVM1
format and then migrate to LVM2, then you will still be limited to
255G, unless you remake your VGs and LVs with the LVM2 tools. If you
made a VG with LVM1, or if you created it with the -M1 flag to
vgcreate in LVM2, then you have an upper limit which is imposed by the
What filesystem is best with LVM?¶
It all depends on what you need. All the different filesystems
work under LVM, it would be best to do some research into the current
technologies that each offer, and make an assessment based on what
your needs are. There are a number of benchmarks out there, and the
filesystems are changing rapidly, so these benchmarks will get out of
date and need to be updated.
Ext2: The most stable filesystem out there for Linux is ext2, think twice
about not using it, it actually is fast and has what most people need.
Seriously consider the reasons why you want to use something else,
because most journaling filesystems are still very much in a
development stage and are risky. Stick with ext2 if all you are
wanting is the ability to never fsck again, because this is not
something you will gain with a journaling filesystem. Disorderly
shutdowns, even with full journaling on your filesystems, can lead to
dataloss of all data that is stored in the buffercache, however
filesystem consistancy will probably be maintained.
Ext3: This is one of the most stable journaling filesystems out as of
this writing. It has been in the kernel for some time, and is mostly
just a hack ontop of ext2. It is simple for converting your ext2
systems to ext3 and back, but it requires some kernel patches if you
want to do online resizing. If you are using Redhat, probably should
stick to ext2/3, Redhat does that well and doesn’t do others so well,
so you are fighting if you do anything else (in fact the ext3 online
extend code is included in the latest RedHat Rawhide kernel).
Reiserfs: They say reiserfs has much better performance in dealing
with lots of small files, but it doesn’t have any bad block management
capabilities. If your drive gets bad blocks, you can very quickly be
rebuiling your entire filesystem from backup. Reiser currently performs rather
poorly on really large files (ie. databases). Oracle doesn’t support
the tuning of a system if it is running reiserfs, and ranks the
different filesystems as follows: ext2, ibm-jfs, ext3, reiser. Reiser
works great for maildir spools and usenet feeds. Because resier3 does
not do true data journaling (only metadata), it is able to bring
filesystem integrity back to normal after a bad shutdown, but some
data loss is possible. Reiser4 supposedly will fix this (as well as
add plug-ins). It seems a lot of people avoid Reiser on Redhat
(something having to do with redhat’s kernel interoperability causing
problems, they support ext3 well, so I would recommend sticking with
ext3 if you are using Redhat), but a lot of SuSE people use it, as it
is well integrated. I personally am a fan of Reiser for maildir spools
and list archives that are parceled out in very large individual
files, it has been pretty stable for us so far, although the website
graphics freak me out and the advertising you get when you create a
filesystem is always a bit of a shock (and “secret” sources make me
On that note, if you are using reiserfs, it balks badly at being a
readonly filesystem. Even if you mount it readonly, it still wants write
access to the journal. We have a situation in our shop where linux can not
access the disks in read/write mode. For those filesystems that were
bootable, we cannot use reiserfs.
JFS: You can dynamically grow this filesystem without intervention,
which is kinda neat, but scary. I have yet to see people consider JFS
stable in linux
XFS: ? In terms of stability, I’ve had significant problems with it in
the past, but it could’ve been a particular kernel patch/version. XFS
only made it into the official kernel in 2.4.23, so be aware…
pvmove gives “device-mapper: error adding target to table” errors¶
Do you have the device-mapper mirror target installed? Not
having it available is the most common cause of this problem. Some 2.6
kernels don’t include this.
- I cannot mount my snapshot volume, gives “wrong fs type, bad option,
bad superblock on /dev/etc or too many mounted filesystems” *
Try mounting with the parameters “-onouuid,ro”
My logical volumes are inactive and unusable!¶
Run “vgscan” and then “vgchange -ay” to activate all the VGs and the LVs.
My logical volumes are not active or usable on restart!¶
Check that you have your startup scripts available that start your
logical volumes. You should have one that does a “vgscan” and then a
“vgchange -ay” somewhere at bootup, without this your VGs and LVs wont
be available on bootup.
The system is not recognizing my boot LVM partitions and kernel panics.¶
You need to run “lvmcreate_initrd”, which will create an
lvm-initrd file in /boot. Then edit grub.conf to use the new initrd.
This will make your lvm tools available at boot and your VGs will come
online. To do this with a rescue disk do the following (change
/mnt/sysimage to whatever your actual root is mounted as):
mount -t proc proc /mnt/sysimage/proc chroot /mnt/sysimage /bin/bash source /etc/profile lvmcreate_initrd exit umount /mnt/sysimage/proc sync shutdown -r now
How do I see if I have LVM support available?¶
LVM1: cat /proc/devices and see if you have “lvm” listed, see if
you have the vgcreate command.
LVM2: check to see if you have the “lvm” base command by doing
“lvm version” – this should give you complete output and confirm that
device-mapper is available in the live kernel and that
/dev/mapper/control is setup correctly (see
device-mapper/scripts/devmap_mknodsh). You can look in /proc/misc to
see if there is device-mapper support in the running kernel.
How do I go from LVM1 to LVM2?¶
If you have a LVM1 setup on your Debian machine you just need to
switch to LVM2 by installing the lvm2 packages. lvm-common is the
Debian package that switches between LVM versions on the same system
You will get this package as a dependency when you install lvm10 or
lvm2. The Debian LVM binaries get installed libraries in
/lib/lvm- and lvm-common decides at run-time which one is
most appropriate to use.
How do I go from a 2.4 kernel to a 2.6 kernel?¶
You need to make sure you have CONFIG_DM_IOCTL_V4 support in your
kernel for device-mapper. You need to select CONFIG_BLK_DEV_DM so that
you can see the option for IOCTL_V4
How do you get device mapper in the 2.4 kernels?¶
Follow the directections for device mapper at evms.sourceforge.net/install/kernel.html
LVM1 = Userspace tools + kernel ioctls included in marcelo’s 2.4 tree
- LVM1 kernel ioctls are not included in or available for 2.6
- LVM1 userspace tools do not work with 2.6 kernels
dm = Kernel driver (GPL) for new volume managers to use.
- Included in Linus’s 2.6 kernels.
- Available as a patch for 2.4 kernels from the Sistina website.
- Knows nothing about volume manager’s on-disk metadata layouts.
- Userspace volume managers (e.g. EVMS and LVM2) communicate via a new
- This ioctl interface is currently “version 4” and we regard it as
- An old development version of this device-mapper ioctl interface known
as “version 1” has problems with it, is deprecated and should be
removed from kernel trees ASAP.
Always use “version 4” when building new kernels today.
libdevmapper = Userspace shared library (LGPL) which wraps a volume manager application interface around the device-mapper ioctls
- Can determine transparently whether the kernel device-mapper is using
“version 4” dm ioctl interface or the deprecated “version 1” interface
and adapt itself accordingly. configure --enable-compat
- Can only communicate with device-mapper: it cannot use LVM1 ioctls.
- Designed primarily for use by LVM2 tools. EVMS does not use it
- Some parts of the libdevmapper API are not yet stable and are likely
to get changed.
dmsetup = Userspace utility (GPL) which provides full command-line access to the libdevmapper API.
- Designed for use by shell scripts and for testing and debugging.
- Command line interface may be considered stable. New features may get
added, but we’ll try not to break existing commands.
LVM2 = New Logical Volume Manager command line tools (GPL) designed to be backward-compatible with LVM1 and offering new features and more flexibility, configurability and stability.
- Supports existing LVM1 on-disk metadata.
This means you do not have to make changes to your existing on-disk
LVM1 volumes to switch between using LVM1 and LVM2.
- Uses command lines similar to LVM1.
- By default uses a new on-disk metadata format supporting more
features than the original LVM1 version.
- Communicates with the device-mapper kernel driver via libdevmapper’s
- LVM1 uses block major number 58: dm selects one or more major numbers
dynamically as required instead.
- LVM1 uses character major number 109: dm selects a misc minor number
- There’s a (non-devfs) script for creating /dev/mapper/control at
startup (or after dm module load).
- You can use LVM1 tools with unpatched 2.4 kernels.
- You can use LVM2 tools with patched 2.4 and unpatched 2.6 kernels.
- Device-mapper support for snapshots and pvmove is so far released
only for 2.4. Patches for 2.6 are being tested.
- Multipath and mirror support are under development for 2.6.
(Then get back-ported to 2.4.)
Web download page: www.sistina.com/products_lvm_download.htm
The device-mapper tarball contains:
device-mapper kernel patches – needed only for 2.4;
userspace libdevmapper and dmsetup – needed with all dm kernels.
The LVM2 tarball contains the LVM2 command line tools.
Device-mapper mailing list: