Howtos Linux-Vserver With LVM And Quotas
From 5dollarwhitebox.org Media Wiki
Contents |
Summary
NOTE: This is currently in the works... don't trust it whilst this line exists
This guy sort of takes over where the previous one left off. You can view the basis of this howto by reading Linux-VServer on Debian Sarge. The first one gives you the basics on how to get Linux-Vserver Installed and gives you all the basic information. So if you haven't done that yet, why are you here?
I know that there are "Per Context Quota" patches to the linux-vserver software, as well as other "quota" related setups. However, and I may be wrong, at the time that I was doing this they didn't seem to work for me. If there is any easier way PLEASE shoot me an email and tell me that I am an idiot! If you're really nice you could even tell me why I'm such an idiot as well.... that would be pretty sweet.
Once you have a working Linux-Vserver system you may find yourself wanting to configure User/Group Quotas. I may be way off base here, but when I got to this point I found it *impossible* to find information on how to do this. The main quark was that quotas borks when trying to run on the virtual device '/dev/hdv1'. Therefore, I spent a good couple of days digging and this is what I came up with. There may be an easier way, but this way seems to work great for me.
- Author: BJ Dierkes
- Last Updated: January 24th, 2006
- Contact: wdierkes [at] 5dollarwhitebox [dot] org
Related Articles
- Linux-Vserver:
- Linux-VServer on Debian Sarge
- Linux-Vserver Script vserver-mount-fs (to accommodate LVM/Vroot/Quotas)
Preparation
Before getting started you will need the following. I am really not going to get to deep into this stuff because I'm hoping that you already have an understanding of the system that we're working on. All of my work is done on Debian boxes, but you should be able to use the information here on any distro.
Shopping List:
- Debian GNU Linux 3.1 Sarge
- Linux-Vserver util-vserver userland utilities (version 0.30.208-4)
- A 2.6.x Kernel compiled with Quota/Vroot/LVM/Linux-Vserver support
- Again you will probably want to visit: Linux-VServer on Debian Sarge
- LVM2 packages [On the Hostbox]
- Quota Tools packages [On the Guest Vserver]
- Free Unpartitioned Hard Disk Space
- Mknod (Available in the coreutils package in Debian)
Throughout the howto I will be referring to the Hostbox (debianhostbox) and the Guest Vserver (vserver1).
Hostbox Software Installation
If you do need to install the LVM2 or the Quota packages, you can simply do so through Apt on a Debian box:
Install LVM2 on the Hostbox:
debianhostbox:~# apt-get update && apt-get upgrade debianhostbox:~# apt-get install lvm2 util-vserver coreutils
That is really all you need to do for the basic software requirements preparation on the Hostbox
Configuring LVM
Before we can do anything with LVM we need to configure our hard disk for LVM. The first step is to create an LVM partition. In this example our hard disk is '/dev/hda'. We already have 'hda1', 'hda2', and 'hda3' so we are going to create 'hda4' as the LVM partition (Note my input is encased in '{ }' brackets:
Create Linux LVM Partition
debianhostbox:~# fdisk /dev/hda
The number of cylinders for this disk is set to 9729.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): { n } <- Create New Partition
Command action
e extended
p primary partition (1-4)
{ p } <- Create Primary Partition (You may need to select the Partition Number)
Selected partition 4
First cylinder (1353-9729, default 1353): { ENTER FOR DEFAULT } <- I'm using the next starting cylinder
Using default value 1353
Last cylinder or +size or +sizeM or +sizeK (1353-9729, default 9729): { ENTER FOR DEFAULT } <- Fill to the end
Using default value 9729
Command (m for help): { t } <- Change the Partition Type
Partition number (1-4): { 4 } <- Changing the Partition Type of Partition 4
Hex code (type L to list codes): { 8e } <- Changint to the Linux LVM Partition Type
Changed system type of partition 4 to 8e (Linux LVM)
Command (m for help): { w } <- Write changes to disk (This will trash data BE CAREFUL)
Create LVM Physical Volume
Once we have an LVM Partition then we need to configure a Physical Volume with LVM utilities:
debianhostbox:~# pvcreate /dev/hda4 Physical volume "/dev/hda4" successfully created
Create LVM Volume Group
A Volume Group can span across multiple Physical Volumes. Here, we only have 1 Physical Volume so things are easy:
debianhostbox:~# vgcreate vg0 /dev/hda4 Volume group "vg0" successfully created
Create LVM Logical Volume
The Logical Volume is the actual partition that you will be making for each Guest Vserver. Therefore I name the Logical Volume according to the Vserver Name. This is most helpful for scripting purposes as you will see in the future.
debianhostbox:~# lvcreate -L1024 -n vserver1 vg0 Logical volume "vserver1" created
This created a 1024MB LVM Partition located at '/dev/vg0/vserver1' and more importantly as '/dev/mapper/vg0-vserver1'.
Make A Filesystem
Once you have this new partition, you need to create a filesystem on it:
debianhostbox:~# mke2fs -j /dev/vg0/vserver1
Mount The Filesystem
You can do this however you like, however the standard util-vserver packages now have the Vserver root directies in /var/lib/vservers and that is whhere we are going to mount this bad boy:
debianhostbox:~# mkdir /var/lib/vservers/vserver1 debianhostbox:~# mount -t ext3 /dev/vg0/vserver1 /var/lib/vservers/vserver1
You now have a mounted LVM Partition just for your new vserver that you are going to create.
Create The Vserver
I am not going to really go into this too much, other that drop the command that I'm running. Again you will probably want to visit: Linux-VServer on Debian Sarge:
debianhostbox:~# vserver vserver1 build -n vserver1 \ > --hostname vserver1.domain.com \ > --interface eth0:192.168.0.105/32 \ > -m debootstrap -- -d sarge
Please note that is one command. The '\' backslashes simply indicate a new line for cleanliness.
Now that this is done, ensure that the vserver is stopped and unmount the partition:
debianhostbox:~# vserver vserver1 stop debianhostbox:~# umount /var/lib/vservers/vserver1
The Vroot Operations
Ok, this is where the magic happens. These steps are required because quota needs a "real" device and the '/dev/hdv1' thang just doesn't cut it. Therefore we use the linux 'vroot' to attach a 'real' block device to the virtual device in the Vserver. At this point the vserver should be stopped and the partition unmounted.
Mknod The Vroot Device
Using the 'mknod' utility we create a vroot block device for our virtual server. If you're not familiar with 'mknod' you should be (try 'man mknod'). Reference the following syntax:
mknod /dev/vroot/<device_name> b <major_number> <minor_number>
The 'b' tells us this is a 'block device'. The 'major_number' determines the device, and the 'minor_number' is a subset of the major number. To understand this better, try the following:
debianhostbox:~# ls -lah /dev/hda brw-rw---- 1 root disk 3, 0 Mar 14 2002 /dev/hda brw-rw---- 1 root disk 3, 1 Mar 14 2002 /dev/hda1 brw-rw---- 1 root disk 3, 10 Mar 14 2002 /dev/hda10 brw-rw---- 1 root disk 3, 11 Mar 14 2002 /dev/hda11 brw-rw---- 1 root disk 3, 12 Mar 14 2002 /dev/hda12 brw-rw---- 1 root disk 3, 13 Mar 14 2002 /dev/hda13 brw-rw---- 1 root disk 3, 14 Mar 14 2002 /dev/hda14 brw-rw---- 1 root disk 3, 15 Mar 14 2002 /dev/hda15 brw-rw---- 1 root disk 3, 16 Mar 14 2002 /dev/hda16 brw-rw---- 1 root disk 3, 17 Mar 14 2002 /dev/hda17 brw-rw---- 1 root disk 3, 18 Mar 14 2002 /dev/hda18 brw-rw---- 1 root disk 3, 19 Mar 14 2002 /dev/hda19 brw-rw---- 1 root disk 3, 2 Mar 14 2002 /dev/hda2 brw-rw---- 1 root disk 3, 20 Mar 14 2002 /dev/hda20 brw-rw---- 1 root disk 3, 3 Mar 14 2002 /dev/hda3 brw-rw---- 1 root disk 3, 4 Mar 14 2002 /dev/hda4 brw-rw---- 1 root disk 3, 5 Mar 14 2002 /dev/hda5 brw-rw---- 1 root disk 3, 6 Mar 14 2002 /dev/hda6 brw-rw---- 1 root disk 3, 7 Mar 14 2002 /dev/hda7 brw-rw---- 1 root disk 3, 8 Mar 14 2002 /dev/hda8 brw-rw---- 1 root disk 3, 9 Mar 14 2002 /dev/hda9
Notice the MAJOR ("3" the same for all) and MINOR (unique) numbers?
So now we create our vroot device for vserver1:
debianhostbox:~# mknod /dev/vroot/vserver1 b 4 0
Note again that the MINOR number must be unique!!! I can't tell you how fun that was running multiple vservers on the same vroot block device! haha! Additionally, if you will be using my 'vserver-mount-fs script later then this vroot name must be identical to the vserver name.
Mounting The Logical Volume
This is what you will need to do every time that you mount this partition:
debianhostbox:~# e2fsck -p /dev/vg0/vserver1 debianhostbox:~# mount -o usrquota,grpquota /dev/vg0/vserver1 /var/lib/vservers/vserver1 debianhostbox:~# rm -f /var/lib/vservers/vserver1/dev/hdv1 debianhostbox:~# vrsetup /dev/vroot/vserver1 /dev/vg0/vserver1 debianhostbox:~# cp -fa /dev/vroot/vserver1 /var/lib/vservers/vserver1/dev/hdv1
Here 'vrsetup' attaches the vroot device to the LVM partition.
Note: It was brought to my attention that mounting the partition with usrquota,grpquota is necessary in order to use quotas withing the guest operating system. Thank you Jérôme Charaoui.
Umounting The Logical Volume
This will need to be done everytime that the vserver's LVM partition is unmounted.
debianhostbox:~# mount -o remount,ro /dev/vg0/vserver1 debianhostbox:~# umount /dev/vg0/vserver1 debianhostbox:~# vrsetup -d /dev/vroot/vserver1
Here 'vrsetup -d' detaches the vroot device from the LVM partition.
Save Yourself: The 'vserver-mount-fs' Script
You can see how daunting this may be to have to run all of these command every time you want to mount an LVM based vserver. Well, hell why not write a script:
Configuring The Vserver
Now that we have the LVM partition setup, and the vserver created on it let get her started and configure quotas. I'm going through all of the steps here, so be certain that the vserver is stopped and the vserver is unmounted:
Enable Quota Control Capabilities
In order for the vserver to have Quota Control permissions, you need to give it the 'quota_ctl' capability:
debianhostbox:~# echo "quota_ctl" >> /etc/vservers/vserver1/ccapabilities
No, that isn't a typo. That is 'ccapabilities'.
Starting And Entering Vserver1
debianhostbox:~# e2fsck -p /dev/vg0/vserver1 debianhostbox:~# mount /dev/vg0/vserver1 /var/lib/vservers/vserver1 debianhostbox:~# rm -f /var/lib/vservers/vserver1/dev/hdv1 debianhostbox:~# vrsetup /dev/vroot/vserver1 /dev/vg0/vserver1 debianhostbox:~# cp -fa /dev/vroot/vserver1 /var/lib/vservers/vserver1/dev/hdv1 debianhostbox:~# vserver vserver1 start debianhostbox:~# vserver vserver1 enter vserver1:~#
Install The Quota Utilities
Now we are in our vserver. We need to first install the quota utilities:
vserver1:~# apt-get update && apt-get install quotatool quota
Configure Quotas
Now that we have the goods lets configure it. The following are pretty standard quota commands:
vserver1:~# quotacheck -maug vserver1:~# quotaon -a vserver1:~# edquota <username> vserver1:~# repquota -auvg
If you have problems here... well, your kinda going to have to replay the last *all* steps and figure something out. Please let me know of any errors or anything that will make these steps easier".
Conclusion
I am very, very aware that this is a *rough* howto. I performed these steps months ago and am certain that I need to do extensive testing to get this to a point of perfection. I hope that this at least helps get somebody on the right track. If you have input, suggestions, complaints, ideas, etc please shoot me an email. I plan on working on this stuff over the next couple of weeks where I will have some more time to do "beginning to end" testing.
Resources
- Linux-Vserver: http://www.linux-vserver.org
- HowTo to get you started: Linux-VServer on Debian Sarge
- Debian GNU Linux: http://www.debian.org
- I think this helped me at one point: http://vserver.strahlungsfrei.de/tiki-download_wiki_attachment.php?attId=9
- The 'mknod' Man Page
- Util-vserver "Flower Page": http://www.nongnu.org/util-vserver/doc/conf/configuration.html
