This doesn’t affect this particular site at all, but in case Dad or Kirsten check in, there seem to be issues with the djwudi.com\/hanscomfamily.com\/geekmuffin.com\/interalia.org server at the moment. Phil and I are trying to work on it and find out what the issue is — whatever it is, it popped up sometime during the night, when neither he nor I were doing anything with the machine. No current ETA on when we’ll be back up and running, though we’re trying to bring it back up as soon as possible.<\/p>\n
Update: Still down, still no ETA, and I need to get out of the house for a bit. I’ll get more info up when I can.<\/p>\n
This also means that any e-mail sent to my [\\@michaelhanscom.com]{.citation cites=”michaelhanscom.com”} address will not<\/em> get to me at the moment. Please use one of my other e-mail addresses if you have one of them and need to get ahold of me.<\/p>\n Just so I don’t lose track or forget about what we tried so far, a quick copy-and-paste of the work so far:<\/p>\n \n Phil: el problemo, senor<\/p>\n Michael: what’s up?<\/p>\n Phil: i think something has run awry in my crontab Michael: aah, is that what’s going on Phil: i suspect you’ll find tons of sa-learn processes<\/p>\n Michael: no ssh or httpd calls would answer Phil: i got ssh to work verrrrrry slowly. Michael: hrm Phil: gah. Michael: k Phil: i’m in Michael: :nods<\/p>\n Phil: other than a lot of sendmail and procmail processes, nothing seems to be wigging out.<\/p>\n Michael: took out whatever it was that was having an issue? or it just doesn’t seem to be “issuing” right now?<\/p>\n Phil: i took out what i -think- was causing it. no idea if that was it or not. Michael: okay Phil: SpamAssassin’s Bayesian filter learning tool. Michael: ah, okay<\/p>\n Phil: i’m still getting some false negatives that it should’ve caught. Michael: yeah – the UI’s frozen again Phil: goddamnit. what is going on? Michael: maybe it’s the machine? (though I hate<\/em> to suggest that) Phil: well, we’ll see; i sincerely hope that’s not it.<\/p>\n Michael: you and me both Phil: is it acting funny? fans any louder than usual?<\/p>\n Michael: doesn’t seem to be Phil: no, i’d wager something is getting overloaded. Michael: could I be getting hit by an attack of some sort, I wonder?<\/p>\n Phil: hmmm.<\/p>\n Michael: god_damn_ that thing takes a while to start up…lol<\/p>\n Phil: there’s a way to test that.<\/p>\n Michael: it’d have to be a pretty good attack to completely lock it up, though<\/p>\n Phil: during the brief time i can get on i’ll try to install snort.<\/p>\n Michael: it’s up<\/p>\n Phil: can’t get it to respond to ssh even now.<\/p>\n Michael: huh<\/p>\n Phil: n\/m Michael: okay<\/p>\n Phil: sshd probably wasn’t up yet<\/p>\n Michael: i’m logged in, have top -u 15 running in a term window<\/p>\n Phil: should see a wget process.<\/p>\n Michael: if it locks up, hopefully it’ll give me an idea of what the issue might be (maybe)<\/p>\n Phil: hopefully.<\/p>\n Michael: right now it’s mostly just top at the top of top (whee) Phil: those were me.<\/p>\n Michael: i’ve got my eye on you, young man…lol Phil: probably mail trying to connect.<\/p>\n Michael: and…freeze Phil: yeah, i noticed. hmm. Michael: i’m pretty sure mine’s empty…very sure, in fact Phil: and \/etc\/crontab just has system stuff in it.<\/p>\n Michael: back up<\/p>\n Phil: yep. i’m in. Michael: ? Phil: if you still want to, go for it<\/p>\n Michael: apache2? Phil: yes. Michael: hmm Phil: OK, I need to go find the startup item for apache2 and slay it.<\/p>\n Michael: it’s fixing a lot<\/em> of permissions<\/p>\n Phil: oh man, wait Michael: uhoh<\/p>\n Phil: any custom permissions I had set on my website are gonna be borked<\/p>\n Michael: oh shit<\/p>\n Phil: well, we’ll find out soon enough.<\/p>\n Michael: it seemed to be doing a lot in the \/System\/Library\/Perl\/ directory Phil: frozen again Michael: grrr Phil: probably the permissions check is taking a lot of cpu time.<\/p>\n Michael: gonna reboot again Phil: tell me about it. Michael: lol Phil: Do an archive & install, then. Michael: :nods definitely Phil: nor I.<\/p>\n Michael: if I do have to do that…think it’s worth putting panther on, rather than jaguar? Phil: if only for the fact that you’d have to configure postfix, i’m gonna say no<\/p>\n Michael: ah Phil: i noticed.<\/p>\n Michael: goddammit Phil: ridiculous.<\/p>\n Michael: fuck – locked up Phil: this is bloody odd. Michael: and i haven’t touched it<\/p>\n Phil: when it starts up, go to the apple menu, check about this mac, and see if anything on the amount of RAM or CPU speed looks unusual.<\/p>\n Michael: up Phil: yes.<\/p>\n Michael: yeah, that’s right<\/p>\n Phil: maxed out for this architecture<\/p>\n Michael: lol Phil: do tell.<\/p>\n Michael: login window could not find image named ‘bang’<\/p>\n Phil: ….?<\/p>\n Michael: :shrugs Phil: probably.<\/p>\n Michael: fuck, not that it matters Phil: i noticed…. so weird. Michael: lol okay Phil: we’ll find out. Michael: not thinking it’s a DOS or a hack or anything, simply because it’s behind my firewall, so huge amounts of traffic on my line should affect my connectivity\/speed on this ‘puter, too<\/p>\n Phil: not if they specifically attacked the webserver by IP address or name. Michael: nope Phil: eh, if it’s off, it doesn’t concern us. Michael: the box doesn’t seem to be restarting<\/p>\n Phil: …<\/p>\n Michael: i’m going to unplug it and let it sit for a few minutes<\/p>\n Phil: well, that’s not good.<\/p>\n Michael: given that this started happening when neither of us were doing anything on the box, and the fact that it seems to be a gradual thing – works fine for a bit, then gradually gets slower – and that it seems to be happening sooner on each reboot, and that it doesn’t want to reboot now… Phil: that’s not good.<\/p>\n Michael: which, realistically, is a possibility – this little G3 was used when I got it, and it’s been running as a server 24\/7 for, oh, probably around four or five years now Phil: Yeah, I can definitely understand that… I was gonna ask about that.<\/p>\n Michael: it’s an option, though, should the g3 not come back<\/p>\n Phil: Do you still have the hardware diagnostic CD somewhere for the g3?<\/p>\n Michael: i don’t think I ever had one<\/p>\n Phil: must be a new thing.<\/p>\n Michael: :nods Phil: well, we have an hour and twelve minutes to figure something out.<\/p>\n Michael: 1:12? lol<\/p>\n Phil: I’m going out to the Irish festival at that point.<\/p>\n Michael: aah Phil: amen. if I had a utilikilt i’d wear it today.<\/p>\n Michael: you should do it, man<\/p>\n Michael: not that you’d have it today, but still<\/p>\n Phil: yeah, i know. well, hey, my birthday’s coming up in a few months… we’ll see<\/p>\n Michael: excellent Phil: OK, it freezes up, you need to restart in single-user mode and run fsck to see if there are disk errors. Michael: and I do that…how, again? lol<\/p>\n Phil: click the link<\/p>\n Michael: I did Phil: it’s still detached from the network, right?<\/p>\n Michael: yeah<\/p>\n Phil: OK, let me find something that doesn’t suck i just noticed “check\/fix the filesystem”<\/p>\n Michael: lol<\/p>\n Phil: i think you need to hold down S while booting.<\/p>\n Michael: gaaa…doesn’t seem to be booting Phil: oh, maaaaan. that’s not good. Michael: :nods it got in last time, and I did a manual (correct) restart<\/p>\n Phil: iMovie froze, the finder crashed and wouldn’t restart, and everything rainbowed. Michael: :nods that’s what i’m thinking (grr) Phil: just remember to hold down C as you turn it on with the jaguar disk in the driver (or panther; if you want, i can look up postfix guides while you wait) Michael: right now…i’m a little annoyed, hungry, and want outside lol Phil: and really, from what i recall, setting up postfix wasn’t hard at all. Michael: as long as the websites survive, i’ll be satisfied<\/p>\n Phil: i think the config files made more sense to human beings<\/p>\n Michael: that’s my big worry, really Phil: i do too. y’know, if all else fails, stick the HD into the G5. I assume it has an expansion slot for a second HD.<\/p>\n Michael: yeah, but I can’t just drop the drive in Phil: oh, fuck, that’s right. dammit.<\/p>\n Michael: i’d need to drop the old drive(s) into ATA cases<\/p>\n Phil: if nothing else, even if the machine won’t boot, our data is still there.<\/p>\n Michael: yeah<\/p>\n Phil: you’re good, webmaster. go get food and sunshine; i’ll be doing the same myself shortly.<\/p>\n Michael: fuck – and i’m using michaelhanscom.com for most of my e-mail right now, too…lol Phil: will do. later man\n<\/p><\/blockquote>\n Update: Everything’s back up again. Phil tracked it down to a cron job that had piled a few thousand e-mail messages into his inbox, which caused everything to choke. Inbox is empty, cron is smacked back into submission, and all’s well that ends well.<\/p>\n iTunes:<\/strong> “Three Wishes” by Waters, Roger from the album Amused to Death<\/em> (1992, 6:50).<\/p>\n","protected":false},"excerpt":{"rendered":" In case Dad or Kirsten check in, there seem to be issues with the djwudi.com\/hanscomfamily.com\/geekmuffin.com\/interalia.org server at the moment.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2040],"tags":[25],"class_list":["post-2620","post","type-post","status-publish","format-standard","hentry","category-blog","tag-website"],"_links":{"self":[{"href":"https:\/\/michaelhans.com\/eclecticism\/wp-json\/wp\/v2\/posts\/2620","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/michaelhans.com\/eclecticism\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/michaelhans.com\/eclecticism\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/michaelhans.com\/eclecticism\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/michaelhans.com\/eclecticism\/wp-json\/wp\/v2\/comments?post=2620"}],"version-history":[{"count":0,"href":"https:\/\/michaelhans.com\/eclecticism\/wp-json\/wp\/v2\/posts\/2620\/revisions"}],"wp:attachment":[{"href":"https:\/\/michaelhans.com\/eclecticism\/wp-json\/wp\/v2\/media?parent=2620"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/michaelhans.com\/eclecticism\/wp-json\/wp\/v2\/categories?post=2620"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/michaelhans.com\/eclecticism\/wp-json\/wp\/v2\/tags?post=2620"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n Phil: and is chewing up way too many resources
\n Phil: on the webserver.<\/p>\n
\n Michael: i just had to restart it – it wasn’t responding to anything<\/em> when I woke up, except pings<\/p>\n
\n Michael: is it down again now?<\/p>\n
\n Phil: No responses again now.<\/p>\n
\n Michael: well, yeah, something’s borked – the UI is frozen
\n Michael: gonna restart it again<\/p>\n
\n Phil: let me know when you restart it, i’ll ssh in and nuke my crontab before anything launches.<\/p>\n
\n Michael: okay, login window is up
\n Michael: give it a shot<\/p>\n
\n Phil: gonna watch top and see what launches.<\/p>\n
\n Phil: but i recall something similar happening last time i tried to make a cron job for sa-learn.<\/p>\n
\n Michael: what’s sa-learn?<\/p>\n
\n Phil: It’s the “learn this now, dammit” tool.<\/p>\n
\n Phil: um.
\n Phil: i don’t think that was it. i’m not getting any responses on ssh again.<\/p>\n
\n Michael: wierd<\/p>\n
\n Phil: i know i haven’t actually changed anything…<\/p>\n
\n Michael: restarting again<\/p>\n
\n Michael: (and kirsten and dad too, I’d bet)<\/p>\n
\n Michael: i just sits in the corner and hums happily to itself (shrugs)
\n Michael: as opposed to Marvin, who sits in the corner and hums dismally to himself<\/p>\n
\n Phil: i don’t know what, though. hmmmm.<\/p>\n
\n Phil: there we go<\/p>\n
\n Michael: i’ve seen ssh, gzip, and tar pop up occasionally, but the drop right back down again<\/p>\n
\n Michael: imapd just hit thetop – bouncing between 10 and 25%<\/p>\n
\n Michael: dammit
\n Michael: the last thing at the top was just imapd at only 10%<\/p>\n
\n Phil: i’m going to nuke my entire crontab. nothing should be launching, but that doesn’t mean nothing is.<\/p>\n
\n Michael: okies, restarting<\/p>\n
\n Phil: ah HA
\n Phil: apache2 is starting.<\/p>\n
\n Michael: i was about to run a permissions check\/repair, just for grins and giggles…
\n Michael: oh, wait<\/p>\n
\n Michael: didn’t you kill that?<\/p>\n
\n Phil: precisely my point. i thought i did too.<\/p>\n
\n Michael: i wonder why it’s starting…and why it hasn’t been a problem until now
\n Michael: wow<\/p>\n
\n Phil: that was NOT a good idea.<\/p>\n
\n Michael: trying to stop it now
\n Michael: spinning rainbow cursor at the moment<\/p>\n
\n Phil: from CL too
\n Phil: so apparently apache2 is not the problem, but it is a problem.<\/p>\n
\n Michael: hmmm – the UI isn’t completely<\/em> frozen
\n Michael: if I move the mouse, the cursor will jump to another point on the screen after a few seconds
\n Michael: still, it’s quite unuseable<\/p>\n
\n Michael: christ, this is annoying…lol<\/p>\n
\n Phil: i want my email!<\/p>\n
\n Michael: if all else fails, there’s always the possibility of a reinstall
\n Michael: which, admittedly, could suck
\n Michael: system install, getting MT up and running again, making sure the websites don’t break, etc. etc., yadda yadda<\/p>\n
\n Phil: (If you have the disk space.)<\/p>\n
\n Michael: i do, there’s still gigs free
\n Michael: not the way I was planning on spending my saturday, though<\/p>\n
\n Michael: i’d have to figure out the new mail system (postfix instead of sendmail, right?)
\n Michael: hm<\/p>\n
\n Michael: okay
\n Michael: yeah – “starting apache 2 web server” is part of the startup window messages
\n Michael: and, we’re up again
\n Michael: holy shit – we’re locked up again
\n Michael: okay, the fact that that’s getting faster does not bode well<\/p>\n
\n Michael: y’know, if it was cloudy, i might not mind as much, but it’s a gorgeous day out there…lol
\n Michael: up
\n Michael: according to the apache2 control panel, it’s not running
\n Michael: even though the startup message was in the startup routine<\/p>\n
\n Michael: well, locking up
\n Michael: in the process
\n Michael: tried to load the sharing control panel
\n Michael: spinning rainbow cursor of death<\/p>\n
\n Phil: something has gone horribly wrong between yesterday afternoon and today.
\n Phil: i haven’t been around to do anything.<\/p>\n
\n Michael: 640Mb ram…trying to remember if that’s right<\/p>\n
\n Michael: brainless
\n Michael: hm
\n Michael: console log has some conflicts listed<\/p>\n
\n Michael: some startup items failed to launch due to conflicts
\n Michael: dammit…do I have ichat on that machine? lol<\/p>\n
\n Michael: i’m losing the UI again
\n Michael: it just keeps slogging down, slower and slower
\n Michael: until it doesn’t respond<\/p>\n
\n Phil: incidentally, the apache2 item should now start apache<\/p>\n
\n Michael: it’s working, it’s just responding like XP installed on a 286
\n Michael: making glaciers look positively snappy, in other words
\n Michael: i don’t get it
\n Michael: okay – restarting it, but i’ve unplugged it from the network
\n Michael: going to see if that makes a difference at all (though I’m not sure why it would)<\/p>\n
\n Phil: if that does it, you’re getting DOS’ed.<\/p>\n
\n Phil: well, some slowness, maybe.
\n Phil: but the webserver would see the majority of it.
\n Phil: …your windows box isn’t on the network, is it?<\/p>\n
\n Michael: well, yes, but it’s off
\n Michael: and on the few times I start it up, the first thing I do is apply any recent security updates
\n Michael: it’s as up to date as it can be without doing a daily security update check
\n Michael: okay, this isn’t good<\/p>\n
\n Phil: what?<\/p>\n
\n Michael: i’m really starting to worry that it’s a hardware issue<\/p>\n
\n Michael: the three years since i’ve been in seattle, plus a year or two in anchorage
\n Michael: it would suck<\/em> if that’s the case, though
\n Michael: i don’t have the funds for a replacement
\n Michael: and i could move things onto the G5, but i’ve liked having a server separate from my work machine<\/p>\n
\n Michael: either that, or I just didn’t get it with the machine<\/p>\n
\n Michael: yeah, I want to get out of the house at some point
\n Michael: shame to waste a perfectly good kilt day<\/p>\n
\n Michael: okay, booting<\/p>\n
\n Phil: http:\/\/www.macosxhints.com\/article.php?story=2004011205473937&query=fsck<\/p>\n
\n Michael: The specific sequence of how to do this varies depending on what version of OS X you’re running. Go to Apple Support for specific instuctions.<\/p>\n
\n Michael: got the startup chime, nothing else<\/p>\n
\n Phil: actually, though.
\n Phil: that might be a sign of a disk error caused by the forced reboots.
\n Phil: we had the exact same thing happen with a G4 at work.<\/p>\n
\n Phil: I’m going to suggest an archive and install.<\/p>\n
\n Michael: i’ll deal with that later, though<\/p>\n
\n Phil: k<\/p>\n
\n Michael: i may do panther…it’s got enough under-the-hood upgrades that it might be a good idea
\n Michael: just take a little poking around to get postfix up<\/p>\n
\n Phil: i did it at WWDC the day i installed panther.<\/p>\n
\n Michael: i’d hate to end up nuking everyone’s sites (again)<\/p>\n
\n Michael: different drive specs
\n Michael: serial ATA on the G5<\/p>\n
\n Michael: argh
\n Michael: ah, well
\n Michael: it’ll wait for a day
\n Michael: or a few hours, or whatever
\n Michael: anyway…i’m wandering – enjoy the irishfest<\/p>\n