ADMIN LOG
What is this?
This is a journal of what I've been doing with admining the SeekingFire network. It's loosely based on the same idea as the NetBSD journal.
Note that this is in reverse chronological order.
The log
May 10 2005
Been awhile since I updated this. Anyway, I figured out why OpenSSH in FreeBSD 5.X (-current, mostly) doesn't work except going to backforty. The other hosts have their hostname set to the .com version rather than the .prv version, and OpenSSH isn't smart enough to cope.
23 Dec 2004
Thoth stopped relaying mail for some reason. I noticed that when my daily status mails from the various boxes didn't come in as expected. It looks like clamd died, which caused the milter to issue 451 "retry later" rejects rather than get stuck with mail that it couldn't filter. Restarted the various clam componenets and it seems happy again. I'll have to keep an eye on this.
15 Dec 2004
Decided to pkg_deinstall argus on Thoth. Withotu rasort working, and thus withou a way to actually manipulate the log data, it wasn't going to be very useful.
14 Dec 2004
Note to self: I need to clean out the old ROSPA.CA Kerberos realm stuff on Surya (and in the krb5.conf files everywhere, as well as the DNS zone). Connected to this is moving Caliban's hostname to the seekingfire.com domain.
13 Dec 2004
Argus isn't working correctly it seems. The rasort utility, used to rotate the logs, just dumps core. I've commented it out of Thoth's /etc/crontab, but if it can't be fixed I'll have to ditch argus and find something else.
11 Dec 2004
Moved the IPv6 tunnel to Hurricane Electric over to Caliban from Coyote. Ran into a few stupid problems: I need to remember that net.inet6.ip6.accept_rtadv must be 0, not 1, on the rtadv server or we end up in loop. It's now working, including rc.firewall6 in "simple" mode, so I get the dancing kame :-) Pinging outside ipv6 addresses seems to be blocked, however. I'll have to look into that someday.
10 Dec 2004
Installed Argus, but on Thoth rather on Caliban as planned. The ramon client tool just segfaults on sparc64 while it worked fine on the i386 box. The port is fairly ugly: no RCng script, no log file rotation script, nothing. Some sample output:
[root@thoth /var/log/argus]# ramon -M Matrix -N 25 -r argus.out -n host 24.72.10.209 10 Dec 04 13:54:50 24.72.10.209 24.72.10.95 396 396 58344 58344 10 Dec 04 13:55:08 24.72.12.11 24.72.10.209 77 6 10934 852 10 Dec 04 13:56:19 24.72.10.209 204.83.206.106 10 10 1500 1260 10 Dec 04 14:09:16 24.72.145.164 24.72.10.209 6 6 372 324 10 Dec 04 14:04:10 24.72.10.95 24.72.10.209 8 2 556 108 10 Dec 04 13:56:14 24.72.10.209 24.72.10.255 7 0 882 0 10 Dec 04 14:05:20 24.185.208.227 24.72.10.209 3 3 186 162 10 Dec 04 13:55:12 24.72.10.95 24.72.10.209 2 2 120 120 10 Dec 04 14:00:36 68.146.197.242 24.72.10.209 1 0 62 0 10 Dec 04 14:01:35 68.146.159.124 24.72.10.209 1 0 62 0
I'll need to keep a eye on disk usage in /var/log/argus/ and /usr/local/argus/archive/.
09 Dec 2004
Starting setting up Apache on Thoth so that I can finally retire Coyote. Tried Apache2 ... HTML::Mason is happy with it. Decided to go with Apache2 then.
17 Nov 2004
Backforty build ipe rather quickly, as compared to Caliban last month:
real 9m21.807s user 4m13.216s sys 0m21.049s
15 Nov 2004
Playing with Kerberos and OpenSSH on 5.3-RELEASE via GSSAPI. Turns out that it's fairly easy. First, one must turn on some options in /etc/ssh/sshd_config:
GSSAPIAuthentication yes GSSAPICleanupCredentials yes
Next, one must turn on some options in /etc/ssh/ssh_config (yes, teh client-side also defaults to disabled):
GSSAPIAuthentication yes GSSAPIDelegateCredentials yes
I found that OpenSSH 3.8 and news uses gssapi-with-mic rather than the old gssapi method, which makes it incompatible with older versions. I also found that OpenSSH takes the hostname to use when checking it's keytab entry not from the client nor from DNS (via the IP in use) but rather from hostname. Most annoying, but according to section section 2.14 of the Kerberos FAQ, most likely correct.
14 Nov 2004
Finally got that 7-bay DEC JBOD tower working. Bus resets are occuring because of marginal cabling somewhere (likely within the ancient tower), and the stupid Symbios SCSI controller that I'm using in Thoth doesn't provide knobs to drop the bus speed. So I wrote an RCng script for Thoth that uses camcontrol to force the drives to a lower speed where bus resets do not occur. rcorder is useful for testing the order that scripts in /etc/rc.d/ will be executed.
root@thoth# cat /etc/rc.d/camcontrol_tillman
#!/bin/sh
#
# PROVIDE: camcontrol_start
# REQUIRE: root
# BEFORE: mountcritlocal bgfsck
. /etc/rc.subr
name="camcontrol_tillman"
rcvar=`set_rcvar`
start_cmd="camcontrol_start"
stop_cmd=":"
camcontrol_start()
{
echo "camcontrol_tillman has started ..."
echo -n "Drive: da0 "
camcontrol negotiate da0 -R10 -q -a
echo -n "da1 "
camcontrol negotiate da1 -R10 -q -a
echo -n "da2 "
camcontrol negotiate da2 -R10 -q -a
echo -n "da3 "
camcontrol negotiate da3 -R10 -q -a
echo -n "da4 "
camcontrol negotiate da4 -R10 -q -a
echo -n "da5 "
camcontrol negotiate da5 -R10 -q -a
echo "da6 "
camcontrol negotiate da6 -R10 -q -a
echo "camcontrol_tillman has finished."
}
load_rc_config $name
run_rc_command "$1"
Naturally, I have to set camcontrol_tillman_enable="YES" in /etc/rc.conf for this to be enabled. Since setting this up, I only encounter bus resets during boot (while the drives are being detected) at the expense of only having a 20MB/s bus instead of a 40MB/s bus.
5 Nov 2004
Technically this happened a few days ago, but who's counting? Anyway, I discovered during a Samba upgrade that it now uses it's own password file in /usr/local/private/smbpasswd rather than the system /etc/passwd file. This caused me lots of confusion at first. It's handy because accounts can be seperately managed. It's not handy because typing smbpasswd by itself, as root, results in a NULL root password when I don't even want root to have SMB access. Gah. Why can't Windows do NFS like everyone else?
4 Nov 2004
Installed lang/fpc-devel, which required a one-line patch to the Makefiel to remove the redundent sysctl items on one line. Annoying, it installs it's config file to ~/.fpc.cfg.sample of root rather than using /usr/local/etc.
25 Oct 2004
Revised the Coyote migration info below. Yeah, that means the old version is lost. Version control would take care of this, but it's just not that important to me ;-).
15 Oct 2004
Installed mail/up-imapproxy in order to cache IMAP logins for Squirrelmail. Webmail seems a lot faster now. According to netstat, it's lisening on all interfaces. It's being blocked by the firewall rules on the external interface, but I'd really prefer that it boudn itself only to the loopback IP.
14 Oct 2004
Installed faceplant, article at http://www.perl.com/pub/a/2004/04/01/masongal.html?page=1. I had to submit a new perl module port via PR (#72706) to satisfy it's dependencies. Messed around with the code a fair bit to get the display to work well with my current web site design (fairly narrow). I also ``optimized'' the larger picture directories with find . -name "*.jpg" -exec convert -sample 400x500 {} {} \; in order to get all the ``originals'' to roughly the right size and thus avoid processing overhead.
For some reason a Bash ``for'' loop didn't work for this, it kept getting tangled in filenames with embedded spaces.
7 Oct 2004
For the past 24 hours or so Caliban has been building /graphics/ipe. This amazing amount of time is due to two things: the many dependencies (I didn't have X or qt installed on that box) and the occassional port build that needs input (like Ghostscript).
So I tried ipe. It's a really nice looking vector-graphics program with LaTeX support. It's less crufty looking than xfig, which is nice.
It supports ``snap-to'' operations, which really help line things up, but then doesn't track those operations and so moving objects means a lot of manual work.
4 Oct 2004
Not much happened over the weekend -- ports tree freeze is still in effect pending the 5.3 release so only a minor samba update (for Athena) has come through lately. I pulled down the beta7 ISOs onto mirrors.accesscomm.ca this morning, 5.3-release is due Oct 17.
Caliban stopped allowed Kerberos logins from Blues and Dell on Friday. Very odd, since it works fine from Coyote and Athena. I suspect it's related to DNS changes but as all hosts have the same krb5.conf and use the same internal DNS server I haven't found a problem to fix yet. I did send a note to the MIT Kerberos mailing list (dated Fri, 1 Oct 2004 21:45:43 -0600 and with subjcet ``Realm mapping gone wrong on some hosts'') that lead to some off-list ideas.
Update: Caliban is fixed now. Jeffrey Altman gave me a hand off-list. It turns out that my forward and reverse DNS names didn't match (caliban.rospa.ca versus caliban.seekingfire.com). I fixed up the reverse pointer and all was well.
28 Sept 2004
Started my plan to switch Coyote and Blues. Coyote acts as my firewall (among other things) and runs FreeBSD -stable). Blues is a barely-used internal workstation running RedHat -- useful mostly for use as a screen and X host. The hardware will be swapped so that Coyote can migrate to FreeBSD -current (with no downtime) and Blues can migrate to FreeBSD -current (with some downtime, but that doesn't matter).
The draft map of services and install notes currently looks like this:
| Service | Notes |
| latd (done!) | Easy -- should be a straight install from ports. I'll need my /etc/ethers and /usr/local/etc/latd.conf files though. |
| apache (done! | Moderate -- the data itself is pulled from an NFS mount, and thus doesn't need to be moved. Is migrating to Apache2 a good idea at this time? (No--I finally have my Mason setup happy, don't touch it). I've gotten rid of my remote logging scheme so that means that the logs and awstats utility need to be migrated for continuity. This means that /etc/daily.local needs to be migrated, and probably converted to proper periodic form. |
| squirrelmail (done!) | Easy -- straight install from ports. Save my config files. Tied to the Apache stuff. DNS also needs to change. |
| tunnels (done!) | Easy -- mostly a straight install of OpenVPN from ports. I'd prefer to re-write my rc scripts in RCng style, though, and read from a data directory. These will move to Caliban, though, not Thoth. |
| firewall(partially done ... the coyote stuff is done) | Difficult -- my rules are quite complex, and tied into things like source-routing, traffic shaping, IPv6 and NAT/RNAT. Moving to pf amd altq makes sense. This will move to Caliban for the most part, not Thoth. |
| traffic shaping | Unknown -- I'm currently using IPFW with dummynet, but converting to pf with altq is attractive. Not critical at to the migration. Caliban, not Thoth. |
| ipsec | Possibly difficult -- -current doesn't like IPSEC in the kernel, preferring FAST_IPSEC. FAST_IPSEC didn't like IPv6 earlier this summer (though it appears to be fixed now). This implies migrating +racoon+ and associated config files like /usr/local/etc/racoon/* and /etc/ipsec.conf. I don't have a pressing need for this at the moment. |
| ipv6 (done!) | Moderate. I have a /64 from HE, with the tunnel terminating on this box. This needs to be moved to the new box. Running rtadvd on the new box is also needed (along with it's config file). This should move to Caliban, not Thoth. |
| quagga (done!) | Easy -- Should be a simple port install (save my config files!). Setting up log rotation of some kind would be useful. |
| smtp relay (done!) | Unknown. Ditch this and simply RNAT to Athena? Reimplement an SMTP relay but this time with a proper user-name database? Stick with sendmail and milters? |
| dns (done!) | Easy -- data is NFS mounted. Migrate to new chrooted bind9 in -current. |
| ssh (done!) | Easy -- just turn it on on both port 22 and port 8080. |
| serial console (done!) | Easy -- turn this on so that I can run headless. /boot.config and /etc/ttys look important. |
| inetd (done!) | Easy -- only Kerberos telnet and kshell are enabled. Install MIT krb5 port and enabled those deamons. |
A few other things ... should I run a higher securelevel? When installing the box, is using cfengine a good way to do it so that I have repeatability or is RCS/CVS/this log good enough? Should I change the internal IP address (to .1)? Obtaining a ports list (portversion -v) seems like a good idea too.
28 Sept 2004
Upgrade pkgsrc on both Surya and Laz via cvs. Found all out-of-date ports with lintpkgsrc -i and the ran make update in the affected packages directories to bring them up to date.
Thought about my ipsec VLAN. I currently use different hostnames for the ipsec IP, but it occurred to me that using a different TLD (seekingfire.sec rather than seekingfire.prv) has a different set of trade-offs.
25 Sept 2004
Made the same change to cvsup on Caliban, though the relevent file was /usr/local/etc/periodic/weekly/100.cvsup.ports due to it being a FreeBSD -CURRENT system with RCng.
24 Sept 2004
Started this log. Changed my cvsup scripts in /etc/daily.local to use -L1 instead of -L2; that should make my periodic email shorter yet still useful.
Removed serveral packages on Purple (libwmf, which necessitated: imagemagick, abiword, all of gnome, dia, and mc), an IRIX 6.5.4m host, with swmgr and then re-added them in an attempt to fix Abiword and Gnumeric (which suddenly died on the box a few days ago). Being able to put the http://freeware.sgi.com/Inst/ URL into the file location bar in swmgr is definitely much handier than downloading packages only to discover dependency issues the hard way.
