Warning Livedoc is no longer being updated and will be deprecated shortly. Please refer to https://documentation.tjhsst.edu.

Difference between revisions of "Todo - Summer 2014"

From Livedoc - The Documentation Repository
Jump to: navigation, search
(initial summer todo list)
(No difference)

Revision as of 00:37, 31 July 2014

The following list of items should ideally be accomplished over the summer. If you are interested in working on a task below, please add your name next to the system name, see iodine-ldap for an example. Items in red need downtime notification provided to users and the Sysadmins mailing list at least 24 hours in advance of maintenance. All other items should have a downtime notice provided to the Sysadmins mailing list at least 6 (preferably 24) hours in advance. Maintenance times should be posted here and loaded into Nagios as soon as they are available for general awareness.

General Notes

Downtime notifications for items highlighted in Red should be posted via Iodine at least 24 hours in advance (but ideally as soon as possible). They should include both a start time and estimated end time. Be generous when estimating end-times :).

In general, a good pattern to follow when updating systems is: Reboot, Update, Reboot again, Verify. This way you are sure that the system is in working order before beginning work. The Verify step is also very important to make sure that you leave systems in working order :). When running updates, check for unexpected downgrades in the package list before starting an emerge; these can indicate packages that need a later version unmasked or keyworded. Always run updates from within a screen on the host system in case of an unexpected disconnection.

You should also make sure you have a current backup (If the system is running Guardian, check /root/scripts/backup.log to make sure the last backup is current and successful).

For paired/redundant systems (eg: casey/smith or ns1/ns2), there should be at least 24 hours between the maintenance windows for the two servers to allow time for any subtle problems to surface.

Please do not claim tasks unless you intend to start working on them shortly. Claiming a bunch of jobs right off the bat leaves other people looking for things to do.

Uupdates

VM Servers

Antipodes

  • update system software
  • install and configure salt for configuration management

Galapagos

  • update system software
  • install and configure salt for configuration management

Littleblue

  • update system software
  • install and configure salt for configuration management

Vega

  • update system software
  • install and configure salt for configuration management

Waitaha

  • update system software
  • install and configure salt for configuration management

VMs

bugs

  • update system software
  • install and configure salt for configuration management

casey

  • update system software
  • install and configure salt for configuration management

cups2

  • update system software
  • install and configure salt for configuration management

fcpsapps

(coordinate system downtime with Mr. Brandon Kosatka)

  • update system software
  • install and configure salt for configuration management

gitlab

  • update system software
  • install and configure salt for configuration management
  • Fix gitlab software to start properly on boot

haimageserver

  • update system software
  • install and configure salt for configuration management

iodine

  • update system software
  • install and configure salt for configuration management
  • Transfer from fryingpan to sonic

iodine-ldap (Andrew Hamilton)

  • update system software
  • install and configure salt for configuration management
  • Transfer from fryingpan to sonic

ion

(Updates should be coordinated with the ion development team)

  • update system software
  • install and configure salt for configuration management
  • Transfer from apocalypse to sonic

license

  • update system software
  • install and configure salt for configuration management

lists

  • update system software
  • install and configure salt for configuration management

mysql1

  • update system software
  • install and configure salt for configuration management

ns1

  • update system software
  • install and configure salt for configuration management

ns2

  • update system software
  • install and configure salt for configuration management
  • Transfer from fryingpan to apocalypse

openafs1

  • update system software
  • install and configure salt for configuration management
  • Transfer from apocalypse to sonic

openafs2/openafs6

  • update system software
  • install and configure salt for configuration management
  • Rename openafs6 to openafs2

openafs4

  • update system software
  • install and configure salt for configuration management
  • Transfer from apocalypse to sonic

opeanfs5

  • update system software
  • install and configure salt for configuration management
  • Transfer from apocalypse to sonic

openldap1

  • update system software
  • install and configure salt for configuration management

openldap2

  • update system software
  • install and configure salt for configuration management
  • Transfer from fryingpan to apocalypse

=openvpn

  • update system software
  • install and configure salt for configuration management

smith

  • update system software
  • install and configure salt for configuration management

=stage64

  • update system software
  • install and configure salt for configuration management (make sure this configuration is either imageable or excluded in the newvm-excludes file)

=steeltoe

  • update system software
  • install and configure salt for configuration management

www

  • update system software
  • install and configure salt for configuration management

Other Servers

Crate

  • update system software
  • install and configure salt for configuration management

Barrel

  • update system software
  • install and configure salt for configuration management

Guardian

  • update system software
  • install and configure salt for configuration management

openafs3

  • update system software
  • install and configure salt for configuration management

mirror

  • update system software
  • install and configure salt for configuration management

Infrastructure Changes

Salt

Nagios/NRPE

Add saltstate to configure Nagios/NRPE. This will likely need to use an accumulator to permit the overriding or addition of checks (such as diskspace and dovecot checks on casey/smith). New saltstate should ideally also fix Bug 878, Bug 1063, and Bug 1066.

LDAP Configuration

NSS LDAP configurations should be pushed to all servers by Salt. This includes making sure the necessary software is installed and pushing appropriate config files. Friendly warning, casey and smith are NOT setup with LDAP accounts.

Kerberos Configuration

Kerberos configurations should be pushed to all servers by Salt. This should include everything except the keytab and .k5login files.

Multipath Configuration

multipath.conf should be unified (Bug 983) and deployed via Salt.

.bashrc configurations

Push out root .bashrc files to all servers/VMs. Also a good opportunity to resolve Bug 1062.

NTP configurations

Push out ntp.conf configurations to all servers. Pushed configuration should make sure to restrict all operations to localhost for security.

LDAP configuration changes

All systems should be configured to point to an LDAP master IP (currently ldap-sun.tjhsst.edu, a new name would be good) and the two OpenLDAP IPs. ldap1 and ldap2 should be removed from configuration (as they are redundant with openldap1 and openldap2. This should ideally be done with Salt.

Graduates

  • Archive 2014 AFS home directories to openafs2/6
  • Deactivate 2014 email accounts (excluding graduated sysadmins) after August 1st
  • Archive deactivated 2014 maildirs to Apocalypse
  • generate 2014 maildir archive for transfer to tape
  • generate 2014 AFS archive backup for transfer to tape

OpenStack (Andrew Hamilton and Samuel Damashek)

Continue to work on the openstack cluster. Networking remains the major blocking point.