Warning Livedoc is no longer being updated and will be deprecated shortly. Please refer to https://documentation.tjhsst.edu.

CentOS VM Server

From Livedoc - The Documentation Repository
Jump to: navigation, search

Netboot

Start off with a normal CentOS minimal installation using netboot.xyz and install the EPEL repository by doing yum install epel-release

Networking

When you first install CentOS, you will have gotten networking using DHCP. Unfortunately, we cannot set up bridging and bonding using DHCP, so we will have to configure networking manually.

Installing Packages

While you still have networking, you will want to install some useful packages for debugging networking and enabling bridging. The list of packages to install is shown below.

bridge-utils
net-tools

Loading Modules

In a typical CentOS Minimal install, the kernel bonding option is loaded as a kernel module, so you want to tell the system to load it on boot by doing the following:

echo "modprobe bonding" >> /etc/rc.modules
chmod +x /etc/rc.modules

Configuring Networking

While NetworkManager is a useful tool, it is probably not a good idea to use it on a server, so we will be removing it. Below are the commands to manually configure networking to the Syslab configuration. Please make sure to change the IP address in the script below. You will probably need to use an ILO or connect a monitor to the server for this step.

  1 systemctl disable NetworkManager
  2 systemctl stop NetworkManager
  3 yum remove NetworkManager
  4 cat > /etc/sysconfig/network-scripts/ifcfg-bond0 << EOF
  5 DEVICE=bond0
  6 BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=0 primary_reselect=0 fail_over_mac=0 mode=802.3ad lacp_rate=0 ad_select=0"
  7 TYPE=Bond
  8 BONDING_MASTER=yes
  9 IPV6INIT=no
 10 NAME=bond0
 11 ONBOOT=yes
 12 EOF
 13 cat > /etc/sysconfig/network-scripts/ifcfg-br0 << EOF
 14 DEVICE=br0
 15 STP=no
 16 TYPE=Bridge
 17 BOOTPROTO=none
 18 IPADDR=<ENTER IP ADDRESS HERE>
 19 PREFIX=23
 20 GATEWAY=198.38.17.254
 21 DNS1=198.38.16.40
 22 DNS2=198.38.16.41
 23 DOMAIN="csl.tjhsst.edu tjhsst.edu sun.tjhsst.edu"
 24 DEFROUTE=yes
 25 IPV4_FAILURE_FATAL=no
 26 IPV6INIT=yes
 27 IPV6_AUTOCONF=yes
 28 IPV6_DEFROUTE=yes
 29 IPV6_PEERDNS=yes
 30 IPV6_PEERROUTES=yes
 31 IPV6_FAILURE_FATAL=no
 32 IPV6_ADDR_GEN_MODE=stable-privacy
 33 NAME=br0
 34 ONBOOT=yes
 35 EOF
 36 cat > /etc/sysconfig/network-scripts/ifcfg-br1802 << EOF
 37 DEVICE=br1802
 38 STP=no
 39 TYPE=Bridge
 40 IPV6INIT=no
 41 NAME=br1802
 42 ONBOOT=yes
 43 EOF
 44 cat > /etc/sysconfig/network-scripts/ifcfg-enp2s0f0 << EOF
 45 NAME="enp2s0f0"
 46 DEVICE="enp2s0f0"
 47 ONBOOT=no
 48 NETBOOT=yes
 49 IPV6INIT=yes
 50 BOOTPROTO=dhcp
 51 TYPE=Ethernet
 52 DEFROUTE=yes
 53 PEERDNS=yes
 54 PEERROUTES=yes
 55 IPV4_FAILURE_FATAL=no
 56 IPV6_AUTOCONF=yes
 57 IPV6_DEFROUTE=yes
 58 IPV6_PEERDNS=yes
 59 IPV6_PEERROUTES=yes
 60 IPV6_FAILURE_FATAL=no
 61 EOF
 62 cat > /etc/sysconfig/network-scripts/ifcfg-enp2s0f0bond << EOF
 63 TYPE=Ethernet
 64 NAME=enp2s0f0bond
 65 DEVICE=enp2s0f0
 66 ONBOOT=yes
 67 MASTER=bond0
 68 SLAVE=yes
 69 EOF
 70 cat > /etc/sysconfig/network-scripts/ifcfg-enp2s0f1 << EOF
 71 TYPE=Ethernet
 72 BOOTPROTO=dhcp
 73 DEFROUTE=yes
 74 PEERDNS=yes
 75 PEERROUTES=yes
 76 IPV4_FAILURE_FATAL=no
 77 IPV6INIT=yes
 78 IPV6_AUTOCONF=yes
 79 IPV6_DEFROUTE=yes
 80 IPV6_PEERDNS=yes
 81 IPV6_PEERROUTES=yes
 82 IPV6_FAILURE_FATAL=no
 83 IPV6_ADDR_GEN_MODE=stable-privacy
 84 NAME=enp2s0f1
 85 ONBOOT=no
 86 AUTOCONNECT_PRIORITY=-999
 87 EOF
 88 cat > /etc/sysconfig/network-scripts/ifcfg-enp2s0f1bond << EOF
 89 TYPE=Ethernet
 90 NAME=enp2s0f1bond
 91 DEVICE=enp2s0f1
 92 ONBOOT=yes
 93 MASTER=bond0
 94 SLAVE=yes
 95 EOF
 96 cat > /etc/sysconfig/network-scripts/ifcfg-VLAN1600 << EOF
 97 VLAN=yes
 98 TYPE=Vlan
 99 DEVICE=vlan1600
100 PHYSDEV=bond0
101 VLAN_ID=1600
102 REORDER_HDR=yes
103 GVRP=no
104 MVRP=no
105 NAME=VLAN1600
106 ONBOOT=yes
107 BRIDGE=br0
108 EOF
109 cat > /etc/sysconfig/network-scripts/ifcfg-vlan1802 << EOF
110 VLAN=yes
111 TYPE=Vlan
112 DEVICE=vlan1802
113 PHYSDEV=bond0
114 VLAN_ID=1802
115 REORDER_HDR=yes
116 GVRP=no
117 MVRP=no
118 NAME=vlan1802
119 ONBOOT=yes
120 BRIDGE=br1802
121 EOF
122 cat > /etc/sysconfig/network-scripts/ifcfg-vlan-vlan16 << EOF
123 VLAN=yes
124 TYPE=Vlan
125 DEVICE=vlan16
126 PHYSDEV=bond0
127 VLAN_ID=16
128 REORDER_HDR=yes
129 GVRP=no
130 MVRP=no
131 BOOTPROTO=none
132 IPADDR=<ENTER STORAGE IP HERE: FORMAT IS 172.16.[last 2 parts of 198.38 IP]>
133 PREFIX=16
134 DNS1=198.38.16.40
135 DNS2=198.38.16.41
136 DOMAIN="csl.tjhsst.edu tjhsst.edu sun.tjhsst.edu"
137 DEFROUTE=yes
138 IPV4_FAILURE_FATAL=no
139 IPV6INIT=no
140 NAME=vlan-vlan16
141 ONBOOT=yes
142 EOF
143 systemctl restart network

Common Problems with Networking

First things first, try rebooting and see if it fixes things. Also, I have noticed that sometimes the network takes a couple minutes to start working. You may also want to check the output of brctl show and ip route.

Authentication

You will have to set up Kerberos auth and NSS LDAP in order to log into the server with your CSL credentials.

Installing Packages

yum install nss-pam-ldapd krb5-workstation pam_krb5

Configuration

You will have to edit /etc/nsswitch.conf to query LDAP for users, but first configure NSLCD to search openldap1 for Sysadmin users.

cat > /etc/nslcd.conf << EOF
uri ldap://openldap1.csl.tjhsst.edu/

base dc=csl,dc=tjhsst,dc=edu
bind_timelimit 2

uid nslcd
gid ldap
base   group  ou=group,dc=csl,dc=tjhsst,dc=edu
base   passwd ou=sysadmins,dc=csl,dc=tjhsst,dc=edu
ssl no
tls_cacertdir /etc/openldap/cacerts
EOF

systemctl enable nslcd
systemctl start nslcd nscd

Copy krb5.conf from another server and generate a keytab for your current server as shown in Kerberos. Make sure to add root principals to /root/.k5login Make sure to limit the groups allowed to log in as shown below: If you mess this up, you may not be able to log in and you will have to start all over.

echo "session optional                        pam_mkhomedir.so skel=/etc/skel umask=0022" >> /etc/pam.d/system-auth
echo "session optional                        pam_mkhomedir.so skel=/etc/skel umask=0022" >> /etc/pam.d/password-auth

cat > /etc/security/access.groups << EOF
root
allaccess
<server name>
EOF

# Add the following line at the beginning of the auth section in both files
# auth required pam_listfile.so onerr=fail item=group sense=allow file=/etc/security/access.groups

Storage

In the Syslab, we use iscsi to export storage to VM servers. The following section will show you how to set up the VM server to get the disks from Apocalypse (storage pool) and Sonic. You may have to add targets for the new VM server as shown in SAN/iSCSI Administration.

Installing Packages and Configuration

yum install iscsi-initiator-utils device-mapper-multipath

cat > /etc/iscsi/initiatorname.iscsi << EOF
InitiatorName=iqn.1992-03.edu.tjhsst:initiator:<SERVER FQDN>.0
InitiatorAlias=<SERVER FQDN>
EOF

systemctl enable iscsid iscsi multipathd
systemctl start iscsid iscsi multipathd

You will have to add ISCSI targets by following the instructions at SAN/iSCSI Administration. Copy /etc/multipath.conf from another VM server and run the following commands:

multipath -F
multipath

Virtualization

Now that you have all the pertinent disks and authentication configured, you have to set up the VM server to actually serve VM's.

Installing Packages

yum install qemu qemu-kvm qemu-img libvirt libvirt-client libvirt-daemon

Configuration

There isn't much configuration for this part, but you can find out how to create XML files for a VM at VM Creation. All you need to do is enable and start the libvirt daemon.

systemctl enable libvirtd
systemctl start libvirtd