Warning Livedoc is no longer being updated and will be deprecated shortly. Please refer to https://documentation.tjhsst.edu.

VM Creation

From Livedoc - The Documentation Repository
Revision as of 09:50, 8 January 2016 by Andrew Hamilton (talk | contribs) (VM Installation)
Jump to: navigation, search

Storage Setup

We use iSCSI LUNs to back the partitions for our VM cluster. Each VM has at least two LUNs for a root and a swap partition. Some VMs may have additional partitions for specific purposes.

Once you know how many and what size partitions you will need, see the SAN/iSCSI Administration guide for detailed instructions on configuring a LUN for each required partition. When you are done, you should have device nodes in /dev/mapper for each new volume


We currently use ext4 as the main filesystem for all of our VM partitions except for swap partitions (obviously). Create filesystems on each of the new VMs partitions now; then mount them on the host VM server.

mkfs.ext4 /dev/mapper/apocalypse-<hostname>-root
mkswap /dev/mapper/apocalypse-<hostname>-swap

mkdir /mnt/<hostname>
mount /dev/mapper/apocalypse-<hostname>-root /mnt/<hostname>

Double-check that the device is writable; occasionally it gets locked RO for some unknown reason.

touch /mnt/<hostname>/t
rm /mnt/<hostname>/t

If this dosen't work you may need to recreate the device nodes

multipath -f apocalypse-<hostname>-root
multipath -f apocalypse-<hostname>-swap
iscsiadm -m session --rescan

VM Installation

We maintain a prebuilt base VM along with an excludes file that can be used to very quickly install a new VM without having to go through the normal Gentoo install process. Use the following commands to copy the stage64 image to the new VM.

scp stage64:newvm-excludes ~/
rsync -avSz --numeric-ids --exclude-from=~/newvm-excludes stage64:/ /mnt/<hostname>

If you need the kernel sources (only if you need to compile 3rd party modules such as AFS)

cd /mnt/<hostname>/usr/src/
git clone git://haimageserver/linux-stable.git linux-stable.git

VM Postinstall

There are a few postinstall steps that need to be completed on the copied image before it is ready to run. First, chroot into the new VM and set the root password.

chroot /mnt/<hostname> /bin/bash
source /etc/profile
export PS1="{<hostname>}$PS1"

If you have any additional partitions beyond root and swap, add them to /etc/fstab.

Configure the system's new identity in various files and make sure that no part of stage64's identity was copied over.

vim /etc/conf.d/hostname
vim /etc/conf.d/net

rm /etc/ssh/ssh_host_* #These should already be gone
rm /etc/krb5.keytab
ktutil -k /etc/krb5.keytab get -p ahamilto/admin host/<FQDN>
vim /etc/issue #Remove the warning and change the hostname
vim /etc/nagios/nrpe.cfg #Edit the bind IP and update check values if appropriate
vim /etc/security/access.groups #remove stage64 and add in appropriate hostname group

crontab -l #Configure appropriate backup time

More about backups can be found at on the Guardian Backup System page, this setup can be done before or after the VM is created.

Finally, exit the chroot and unmount all of the VM's partitions.

#umount additional partitions if needed, then.
umount /mnt/<hostname>

Libvirt VM Definition

Copy the stage64 XML configuration file to create the new VM's configuration; then edit it as detailed below.

cd /etc/libvirt/qemu
cp stage64.xml <hostname>.xml
vim <hostname>.xml

Set name, partitions, kernel, memory, and networking as appropriate Mac address is derrived from the IPv4 Address of the system

Load the new VM's configuration into libvirt and then start the new VM.

virsh define /etc/libvirt/qemu/<hostname>.xml
virsh start <hostname>