Dylan is on Vacation
The unix systems are very stable, but if a meteor should hit the server room, you may have to take action.
Check Status
Do this first!
http://seattlecentral.edu/nagios/
Call Dylan's Cell
Do this second. Seriously. I don't mind. I'll also try and check email, so you can email me less-urgent problems. I can do %99 of my job from an internet cafe.
Look for Other Causes
I'm not there breaking things, so make sure it's not a problem somewhere else in the chain. The status check above can be helpful with that. Firewall changes are a common cause of problems that look like unix problems but aren't. That includes firewalls at district as well as firewalls here at SCCC.
Disk Space
I've recently cleared out space in the most full directories, but this is still the most likely thing to go wrong while I'm away (Still not likely at all.) The most active directories are log directories and user home directories.
Web
If a web server stops working, the easiest thing to do is switch to the backup web server. See the Failover_Howto.
Because my old docs are in hard-to-remember-how-to-edit docbook XML, I'm writing this here:
Our web pages live on two servers, Tintin & Arthur. At this time, though I would like to reverse this, requests go first to Arthur. If these requests are on a special list (/etc/httpd/onarthur.conf), Arthur processes them, otherwise Arthur transparently proxies the requests to Tintin. Keeping the onarthur.conf file up-to-date is vital. It is actually a list of RewriteRules who's only job is to skip the rule that proxies off to Tintin. They work by saying "skip X rules". X, in this case is the number of other onarthur rules+1 for the proxy-to-tintin rule. In order to make maintaining this list somewhat sane, I wrote a quick perl script called 'skipper.pl' which lives in /etc/httpd on arthur. Just run "./skipper.pl < onarthur.conf > onarthur.conf.new ", give onarthur.conf.new a look-over and if it looks good, copy it over onarthur.conf and restart the web server.
DNS/DHCP/NTP
These are handled by redundant servers. If one of them dies, the other will do the job. If one of them starts giving bad data, you can turn it off and the other will do the job.
Student Accounts
The student account information from registration comes from the HP3K at district. The password expires every 60 days (I think..). When this password expires, badness generally follows. Please see The_HP3K_Password to see how to deal with this.
Read This
Here's my old "Emergency How-To". It's out of date, but a lot of it is still valid.
These are all things in my "Emergency How-To" that I know are wrong. There are almost certainly things that should be on this list that aren't.
Switching redundant web servers from active to standby and back again. See Failover_Howto.
- Anything Firewall related. My firewalls are gone!
Anything describing hardware or physical locations. More up-to-date but still incomplete is My_Hardware_List.
This page is password protected. The username and password are the same as you would use to log onto Arthur with SSH.
https://www.seattlecentral.edu/dmartin/secure/howto/index.html