Rebuilding djwudi.com

This is going to be my repository for keeping track of what I do to install and configure the djwudi.com server. As such, it’s likely to be filled with all sorts of geeky bits of no interest to anyone except me. Feel free to ignore it. :)

20031015 2315: Install OS X 10.2

The first bit is fairly obvious.

Installation options: all localization options, extra applications, etc. are off. I’m installing merely the core OS and the BSD subsystem. As this is now going to be a dedicated server, rather than a combination server and workstation, I don’t need the extra goodies such as iTunes, iPhoto, yadda yadda yadda.

20031015 2352: Reconfigure home network

Apartment Network

Something’s going goofy here. The G5 sees the ‘net fine, the G3 suddenly isn’t. Odd — it did last night after a fresh system install with the same settings. Going to have to track that down soon — hopefully it’ll cure itself after a reboot, as I’ve got some more installations to go. In the meantime, my current network setup is shown in the graphic.

[Update:]{.underline}

Figured out the ‘goofyness’. When entering the DNS servers, make sure you get all the numbers entered correctly. It helps.

20031016 0019: Install developer tools

All options (including the BSD SDK, which is off by default) are on.

20031016 0054: Install all necessary software updates from Apple

Installing: IE 5.2.2 Security Update, Mac OS X Update Combined 10.2.8, QuickTime 6.3, Safari 1.0, StuffIt Expander Security Update 7.0, and Java 1.4.1.

Not installing: iMovie 3.0.3, iPhoto 2.0, iPod Software 1.3, iPod Software 2.0.1, iTunes 4.0.1, or iCal 1.5.1.

20031016 0202: Fine-tune initial setup

Adjust all system prefs to taste (Energy Saver needs to be set to never go to sleep), enable file sharing, web sharing (Apache), SSH access, and FTP access, verify that the webserver is responding (it is, though nothing’s there yet), nod in satisfaction, yawn, and go to bed.

20031016 2319: Set up and configure sendmail

Most if not all of the following commands need to be executed as root. All usernames, domain names, and variables listed below as are I set them for my server. If anyone else is going through this page as a reference, your variables will need to be adjusted for your system.

  1. Start sendmail automatically at system boot^1^.
    • Edit /etc/hostconfig: change MAILSERVER=-NO- to MAILSERVER=-YES- and set HOSTNAME=-AUTOMATIC- to HOSTNAME=djwudi.com
  2. Make sendmail play nice with Mac OS X’s permissions^2^.
    • cp /usr/share/sendmail/conf/cf/generic-darwin.mc /etc/mail/config.mc

    • Create the following script, save as /etc/mail/update, and make it executable (chmod g+x /etc/mail/update or chmod 654 /etc/mail/update):

      #! /bin/sh
      
      if [ /etc/mail/config.mc -nt /etc/mail/sendmail.cf ]
      then
          echo Regenerating sendmail.cf
          m4 /usr/share/sendmail/conf/m4/cf.m4 /etc/mail/config.mc > /tmp/sendmail.cf
          mv /etc/mail/sendmail.cf /etc/mail/sendmail.cf.old
          mv /tmp/sendmail.cf /etc/mail/sendmail.cf
          /System/Library/StartupItems/Sendmail/Sendmail restart
      fi
      
    • Edit /etc/mail/config.mc and add the following line just after define(PROCMAIL</code>&hellip;: <ul> <li><code>define(confDONT_BLAME_SENDMAIL’, `GroupWritableDirPathSafe’)

    {=html}
    <!-- -->

  3. Run the update script:
    • ./update
  4. Tweak netinfo per Apple’s suggestions^2^.

    • niutil -create . /locations/sendmail
    • niutil -createprop . /locations/sendmail sendmail.cf /etc/mail/sendmail.cf
  5. Define hostnames to accept incoming e-mail for^1^:

  6. Edit /etc/mail/local-host-names and add:

    djwudi.comgeekmuffin.comhanscomfamily.com

      </li>
      <li>Restart sendmail:
        <ul>
          <li><code>ps -ax | grep sendmail</code></li>
          <li><code>kill -HUP xxx</code> (where <em>xxx</em> is the process ID of whichever sendmail process ends with <code>-q1h</code>)</li>
        </ul>
      </li>
    </ul>
    
  7. Set e-mail aliases^1^.

  8. Start NetInfo Manager.

  9. Unlock it.

  10. Click on / > Aliases.

  11. Create a new folder (leftmost button or, in the menus, Directory > New Subdirectory).

  12. Rename the new directory webmaster.

  13. Insert a new property (in the menus, Directory > New Property).

  14. Download and build the IMAP server^1^.

    • curl ftp://ftp.cac.washington.edu/imap/imap.tar.Z > imap.tar.Z
    • uncompress imap.tar.Z
    • tar xf imap.tar
    • cd imap-2002e/
    • make osx SSLTYPE=nopwd SSLDIR=/usr SSLCERTS=/etc/sslcerts
    • mkdir -p /usr/local/bin
    • cp imapd/imapd /usr/local/bin/imapd
  15. Configure the IMAP server^1^.
    • Set up the security certificate:
      • mkdir -p /etc/sslcerts
      • openssl req -new -x509 -nodes -out /etc/sslcerts/imapd.pem -keyout /etc/sslcerts/imapd.pem -days 3650
      • Follow the prompts and insert the correct information when required.
    • Set OS X to answer to IMAP requests over SSL port 993.
      • Edit the /etc/inetd.conf file and add the following line at the end of the file:
      • imaps stream tcp nowait root /usr/libexec/tcpd /usr/local/bin/imapd
      • Restart the inetd daemon:
        • ps -ax | grep inetd
        • kill -HUP xxx (where xxx is the process id of inetd)

At this point, sendmail works for sending messages from the server, and I can log into the IMAP server and check my messages using Mail.app on my G5. Unfortunately, I can’t figure out how to send mail from a machine other than the server — any settings I use result in errors of one sort or another. For now, I’m going to stick with what I have, and come back to tackling IMAP at another day.

20031017 1039: Continue to configure mail services

  1. Allow djwudi.com to catch mis-addressed email^2^.
    • Edit /etc/mail/config.mc and add the following line just after where we added ‘DONT_BLAME_SENDMAIL’ earlier:
    • define(LUSER_RELAY',local:djwudi’)
    • Rebuild and restart using the update script (./update)
  2. Allow relaying from trusted hosts^2^.
    • Edit /etc/mail/access to include my G5 by adding the following lines:
      • 216.231.44.207 RELAY
      • dsl231-041-022.sea1.dsl.speakeasy.net RELAY
    • Compile for use with sendmail:
      • makemap hash /etc/mail/access < /etc/mail/access

And that solved my problem from last night where I couldn’t send mail from my G5. Rock on — I’m learning things bit by bit. Fun!

20031017 2137: Finalize tweaking sendmail (for the moment)

  1. Tweak the sendmail update script to ease work down the road^2^.

  2. Add the following two ‘if/fi’ commands to the script shown above (20031016 2319 item 2). Running the final script will then check to see if the sendmail.cf, aliases, or access files have been updated since it was last run, and if they have, it will rebuild and restart sendmail.

    if [ /etc/mail/aliases -nt /etc/mail/aliases.db ]
    then
        echo Updating aliases
        newaliases
    fi
    
    if [ /etc/mail/access -nt /etc/mail/access.db ]
    then
        echo Updating access
        makemap hash /etc/mail/access < /etc/mail/access
    fi
    
      </li>
    </ul>
    

20031017 2206: Set up user accounts.

If you don’t know how to do that without bulleted and numbered steps, you probably shouldn’t be reading the rest of this webpage. ;) In any case, there are now user accounts for myself, dad, and Kirsten on the server.

20031017 2235: Start tweaking the webserver.

  1. Enable SSI^3^.
    • Remove the # characters (uncomment) the following two lines in the /etc/httpd/httpd.conf file:
      • # AddType text/html .shtml
      • # AddHandler server-parshed .shtml
    • Find the Directory directive for /Library/Webserver/Documents and add Includes to the end of the Options line.
    • Save httpd.conf and restart Apache.
  2. Enable PHP^4^.
    • Edit the /etc/httpd/httpd.conf file and uncomment the LoadModule and AddModule lines that handle PHP.
    • Add the following two lines (I added them just underneath where we uncommented the SSI AddType lines):
      • AddType application/x-httpd-php .php
      • AddType application/x-httpd-php-source .phps
    • Save httpd.conf and restart Apache.
  3. Allow serving SSI and PHP files by default along with HTML^5^.
    • Edit /etc/httpd/httpd.conf, find the DirectoryIndex line, and add index.php and index.shtml to the end of the line. Now, when no filename is specified, Apache will default to each choice in order — first looking for index.html, then index.php, then index.shtml.
  4. Things that I’m not going to play with yet, but will come in handy later: Custom Error Pages, .htaccess information, and password protecting directories are all covered in the document referenced at footnote 5.

20031018 0013: Back up a bit and go for better upgrades!

Well, here’s a nice find. I was poking around for other good Mac OS X apache/perl/php/sql etc. resources on the ‘net, and happened across Server Logistics, who offer pre-compiled OS X .pkg installers for Apache 2 (rather than 1.3.27), PHP4 with more added extentions, Perl 5.8.0 (rather than 5.6.0), [mod_perl 1.99_07], MySQL 4.0.15, and a few other packages that I probably don’t have a need for.

So, it looks like my next step is upgrading all of those packages. At least I discovered this while I was still fairly early on in the installation and configuration process!

[Update:]{.underline}

Okay — all of the above listed software packages have been installed on the server.

20031018 0211: Install MovableType

  1. Download MovableType^6^.
  2. Install MovableType following the provided instructions^7^.

20031018 1432: Whoops

Well, during the MovableType install process, I realized that I needed the DBD::mysql perl module installed to allow MovableType to talk to the MySQL database. A few hours of playing then convinced me that the one downside to the Server Logistics MySQL package was that it used non-standard installation locations, which caused issues with getting DBD::mysql installed. I fought with it for a while, until I got to the point where it was easier to just give up.

My next approach was to remove the Server Logistics MySQL package, and replace it with another one that I’ve used in the past^8^. However, that installation apparently didn’t like some of what was left over after removing the prior installation, and it told me to kiss off. Not in so many words, of course, but that was the result.

So, just to be on the safe side, I’m starting over. OS X is re-installing (again) now, and I’ll go through everything I’ve detailed above (again). At least this time it’ll be easier, as I won’t have to muddle my way through figuring it all out again. I’ll set everything up the same way I had been, only using the MySQL package I’m more familiar with instead of the new one that caused me issues.

Ah, the joys of geekdom…

20031019 1258: Starting over again

So, I started over. Yesterday got OS X installed and updated. Today so far, I’ve installed the OS X developer tools, and set up and configured both sendmail and imapd. This all goes much faster since I thought to write it all down the first time!

Now I’m at the point where, the first time through, I started tweaking the webserver setup, only to get distracted partway through by finding the Server Logistics packages. This time, I’m going to start by installing their packages (except for the MySQL package), then install MySQL from the package I’m more familiar with, then continue on and see where things go from there.

20031019 1451: Last few installs before MovableType (I hope)

  1. Install lynx^9^.
    • The downloadable installer puts lynx in /usr/local/bin/ rather than /usr/bin/, and isn’t seen by the default shell after an installation. I solved this by creating a symbolic link to lynx (ln -s /usr/local/bin/lynx /usr/bin/lynx).
  2. Install wget^11^.
    • Same caveats as with lynx.
  3. Install ncftpget^12^.
  4. Update CPAN^10^.
    • perl -MCPAN -e shell
    • Follow the questions at the prompts. All defaults should be acceptable.
    • Once setup is done, at the CPAN prompt, type install Bundle::CPAN
    • After CPAN updates, type reload cpan
    • (Optional: at the CPAN prompt, type r to get a list of installed modules that have been updated. For any modules that you want to update, just type install [module name] to update them to the most recent versions.)
    • At the CPAN prompt, type install Bundle::DBI
    • At the CPAN prompt, type install Bundle::DBD::mysql
  5. Install Image::Magick^13^.
    • Well, that doesn’t seem to work (at least according to mt-check.cgi). Moving on…
  6. Install Fink^15^.
  7. Install NetPBM^14^.
    • Well, this is nice. The OS X binary package for Fink now comes with a GUI application for managing Fink packages called ‘Fink Commander’. Using that, installing NetPBM was a single-click operation, and I’m now installing the Fink package for Image::Magick also, to see if that works any better than my first attempt did. One way or another, I’ll have image manipulation available for MovableType!
    • Image::Magick threw a fit because I don’t have an X11 window manager installed. No biggie, I’ll just go with NetPBM.

20031019 1636: Install MovableType

  1. Download MovableType^6^.
  2. Install MovableType following the provided instructions^7^.

Footnotes/References

  1. O’Reilly Network: Setting up a Site Server with Jaguar
  2. O’Reilly Network: Configuring sendmail on Jaguar
  3. O’Reilly Network: Apache Web Serving with Jaguar, Part 2
  4. O’Reilly Network: Apache Web Serving with Jaguar, Part 3
  5. O’Reilly Network: Apache Web Serving with Jaguar, Part 4
  6. MovableType.org: Download
  7. mtinstall – Installing MovableType
  8. Marc Liyanage – Software – Mac OS X Packages – MySQL
  9. Lynx text based web browser
  10. Installing Perl 5.8 on Jaguar (scroll down to ‘Testing Your Installation with CPAN’)
  11. Apple – Downloads – Unix & Open Source – wget 1.8.1
  12. NcFTP Software: Download
  13. Marc Liyanage – Software – Mac OS X Packages
  14. Fink – Package Database – Package netpbm
  15. Fink – User’s Guide – Install

Mayday! Mayday!

Back in August I described my experience level as “somewhere between ‘power user’ and ‘geek’ — in other words, I know enough to be dangerous, but not enough to get paid for it.” While it was said somewhat tongue-in-cheek, it’s come through with a vengeance this week — especially the dangerous part.

What started as simply trying to install a MovableType plugin has, thanks to my own bumbling incompetence, rapidly devolved to the point where I may not be able to resurrect my server to its prior state without starting completely over from scratch. While I haven’t lost any information from any of the three sites I hosted (the sole consoling factor to any of this), I have managed to utterly and completely destroy all the various little connecting pieces of software that tied it all together.

Something in the old httpd.conf file (the configuration settings file for the Apache webserver) was causing issues after the system reinstall, to the point where CGI executables weren’t running, and Apache itself was sometimes refusing to start up. I’ve just reverted to the default httpd.conf file, and I’m going to need to go through the old and new files line-by-line, enabling features one by one, until I find whatever setting(s) is (are) causing the problems.

MySQL needs to be reinstalled. Once it’s reinstalled, I need to see if I can find the old MySQL database that MT was using, and then see if I can get the new MySQL installation to access it. If I can’t, then MT won’t be able to access any of the old posts, and we’ll have to start over from scratch there.

Perl is missing various libraries that MovableType uses. Off the top of my head, the Perl -> MySQL bridge that allowed MT to read the MySQL database that stored all the weblog entries for six (?) weblogs strewn across the three sites; and the image processing libraries that allowed MT and the Gallery photo album software on the hanscomfamily.com site to automatically generate thumbnails for uploaded images. There are probably more missing too that I just haven’t run across yet.

In other words, to borrow an old military acronym, everything is currently FUBAR.

What I really want to do right now is call in “sick” to work so I can dedicate the day to working on the system. Unfortunately, neither my bank account nor my slightly overdeveloped sense of responsibility to my job will let me do that, so things are just going to have to stay the way they are for the moment.

The worst thing about all this is that it’s not just my own site. If it were only my own personal playground, than while it would certainly be frustrating and aggravating, at least that way I would only be causing issues for me. However, since I have been hosting the sites for both my family and a friend, I’ve now managed to knock them offline too. Bleah. Not a good situation.

Anyway. Whining about it here isn’t really going to do much other than let me vent my frustration for a bit. Work beckons, and then this evening, I’ll be diving back into everything to see what else I can break.

Dammit

Well, I managed to break my server. Attempting to log in to MovableType results in long strings of errors — apparently perl is completely hosed. Time to re-install Jaguar and hope that I don’t manage to nuke the websites that are on there.

This has not been my best couple of nights.

Update: OS reinstall is done, and all three websites are visible again. MT functionality hasn’t been restored yet, though — that’s my project for tomorrow night. Right now, it’s bedtime.

Not a good evening

I host three sites on a computer here in my apartment — my old website (djwudi.com), my family’s website (hanscomfamily.com), and my friend Kirsten’s site (geekmuffin.com). All three sites have been getting hit over the last few weeks with the comment spam that is rapidly becoming such a hassle.

Tonight I attempted to install the MT-Blacklist plugin that has recently been released as a tool to combat these comment spammers. Unfortunately, after installing it, when I attempted to access it, I got the following error:

An error occurred: Can’t locate Storable.pm in @INC (@INC contains: /Library/WebServer/CGI-Executables/mt/extlib /Library/WebServer/CGI-Executables/mt/lib /System/Library/Perl/darwin /System/Library/Perl /Library/Perl/darwin /Library/Perl /Library/Perl /Network/Library/Perl/darwin /Network/Library/Perl /Network/Library/Perl .) at /Library/WebServer/CGI-Executables/mt/lib/MT/PluginData.pm line 9. BEGIN failed–compilation aborted at /Library/WebServer/CGI-Executables/mt/lib/MT/PluginData.pm line 9. Compilation failed in require at /Library/WebServer/CGI-Executables/mt/extlib/jayallen/Blacklist.pm line 18. BEGIN failed–compilation aborted at /Library/WebServer/CGI-Executables/mt/extlib/jayallen/Blacklist.pm line 18. Compilation failed in require at /Library/WebServer/CGI-Executables/mt/mt-blacklist.cgi line 29.

Well, that’s no good. I dinked around with fink for a while, attempting to get the mysteriously missing ‘storable.pm’ installed, only to continually get error message after error message. Eventually giving up in frustration, I decided to attempt one of the other, lower-tech methods of combatting the spambots — simply renaming the scripts that handle comment and trackback submission. However, when I attempted to do a rebuild on my old weblog, I ended up getting the same error message.

This worries me. It would appear that this ‘storable.pm’ is required for MovableType to function at all. However, now all of a sudden, I don’t have it, and I have no idea why. So now, I’m faced with reinstalling OS X, fink, and whatever other packages I had installed on the server — and I just hope I can remember them all — and hopefully do so while still managing to keep the information for all three weblogs. I could actually live if I lost what’s on my old weblog, as most of it is already imported into this weblog, and I’ve got the export files saved already, but I don’t have either a recent backup of the box or backups of the hanscomfamily.com or geekmuffin.com directories (yes, I know, bad sysadmin).

I think I’ll be able to reinstall without losing everything — but then, I thought things were hunky-dory up until this point, too. So I’m a little concerned.

End result — it’s past my bedtime, I’m tired, more than a little frustrated, and ready to go to bed. Hopefully things will look better when I get back to poking around tomorrow evening.

(I don’t think that MT-Blacklist caused any of the problems, in case anyone is wondering. I think it’s just either my goofing something up, a random server glitch, or a combination of the two. No worries on the MT-Blacklist front in and of itself.)

Easy MovableType to TypePad redirecting

Since this weblog used to be managed using MovableType on my personal server, and I’m working on moving all of my old posts over to this weblog (only one year’s worth of posts left to go!), I’ve ended up with most of my posts duplicated in two spots on the ‘net. I’m also still getting a lot of hits to my old site (and the occasional comment) thanks to all the search engines that still point there.

I’d been planning on diving into the arcana of the Apache mod_rewrite module — it’s a very powerful way to tell your server “if someone asks for this page, send them to that page instead” — to redirect all the hits to this new address, but then tonight I discovered a much, much easier solution.

Needless to say, as soon as I get those last years’ worth of posts transferred over, I’ll be setting this method up.

Here come the ads

My “Bookshelf” (books) and “Noises” (CDs) lists are back on the site, now that TypePad has given us the necessary tags to work them into our templates. Since I didn’t want to go back to having sidebars on the page, but still wanted to incorporate them into the front page somehow, I’m currently experimenting with an “ad banner” style layout.

Between the 2nd and 3rd posts on At the very bottom of the main page, there’s now the “ad banner” box, displaying the most recent addition to my book and music lists. For books, this is whatever I’m currently reading, and for music, I’m more or less randomly choosing a CD every so often to pop in there. The far right side is simply a link to Amazon. As always, any purchases from these links funnel a few pennies my way. It may not be much, but every little bit helps!

Notable me

TypadistasNifty — I just got picked as a ‘Notable’ site on the Typadistas directory!

And while it’s really, really geeky, I love the fact that she complimented me on my source code (hey, like I said, it’s really geeky). I actually put a bit of effort into making sure that my code is clean, well-structured, and easily readable — not only does it help me when coding and debugging, but I figure it might also help others looking for examples (which is a large part of how I learned in the first place). Always nice to know that someone appreciates that!

www.michaelhanscom.com

One of the features of the TypePadPro” level that I’ve been looking forward to is domain mapping — the ability to assign a domain name I own with my TypePad weblog. Last night I noticed that domain mapping beta testing was in progress, so I sent a note to let them know that I was interested. Lo and behold, I got my response this morning, made a few clicks to my domain configurations…

…and it worked! Eclecticism is now residing at www.michaelhanscom.com! Even better, the old address of djwudi.typepad.com will still work as a backup, so any links out there pointing my way will still work, without any issues whatsoever.

So, feel free to update your bookmarks to point to www.michaelhanscom.com (or don’t, whatever suits your fancy). I know I’ve been bouncing everyone around in the past few months — from djwudi.com’s ‘The Long Letter’ to djwudi.typepad.com and now to michaelhanscom.com — but this should be the last move for the foreseeable future.

What of djwudi.com, then? Well, I’ll likely leave my DJ Wüdi propaganda over there, as well as having it available for whatever other little projects I want to play with when I’m in the mood to geek out, so it won’t be disappearing. Just in case you were worried. ;)

Working out the bugs

I’ve been getting some great feedback on the new design, and it’s very appreciated. The kind words on the new look are always flattering, and pointing out areas that are confusing is wonderfully helpful. Things that make sense to me as I’m putting it all together don’t always fly in the real world, and I’m never upset by constructive criticism!

A ‘home’ link has now been added to the navigation bar for all the sub-pages. While I’d had the ‘eclecticism’ title linked back to the home page, it wasn’t terribly obvious, so this should clear up any confusion there. Besides, a little redundancy never hurt.

I’m going to need to do a little tinkering to the display of the comments. I decided to break with convention a bit and put the byline of each comment above the post, rather than below, which seems to be a tad disorienting. Breaking conventions is all well and good — doing so at the cost of usability isn’t. Fixing that is high on the priority list.

Next on the priority list will be adding a bit more space between individual posts on the main page and comments in the comment threads. I’ll need to figure out the best way to do that — because I’m using a display: inline; declaration for the h3 tags to set the border just around the text rather than across the width of the div, simply adding a margin-top: 10px; argument won’t work. I could simply add one or two p or br tags to add some lines of whitespace, but that introduces some unnecessary (purely presentational) code, which I’m trying to avoid, so I’d like to come up with a better solution than that. We’ll see how that goes.

How this page looks in Safari

Right now, the lowest priority is fighting with the skyline image at the top. If those of you that are seeing problems with the display of the image could let me know what browser/version/OS/resolution you’re using, as well as telling me that it’s ‘off-center’, it’d help greatly. I’m using Safari 1.0 on Mac OS X, at 1024×768, and the header looks fine to me. I also checked it in Camino (which should match with Mozilla or Netscape, as they use the same rendering engine), and it was good there. It was only in IE/Mac OS X that I saw any issues (and I haven’t looked into that yet). Unfortunately, my PC is dead at the moment, so I can’t test the site on PC browsers from home, but I’ll certainly be looking into it from work.

Anyway, I’m quite gratified that the design seems to be fairly well received, and that any bugs that have been mentioned so far are actually fairly minor. It’s about time I started exploring different ideas, and you all are helping me iron things out a lot. I’ll buy you a drink next time you’re in town. :)

Well, would'ja look at that?

And here we go, folks — step one of the new design. It’s not completely finished yet, as all I’ve got active at the moment is the primary content, but the rest will follow soon enough.

Update: Okay, I should have thought to check this first, but this redesign has just proven — again — that Internet Explorer sucks. This may or may not get fixed in the future — I’m tempted to just leave it as-is. I’m doing things correctly, dammit, and it’s not my fault that that program doesn’t do what it’s supposed to. Grrr.

Update: That’s it — we’re live. The only page I have yet to dink with is the ‘About’ page, but considering it’s 4:22am, I need to get to bed. All pages linked in the navbar now work, and there are even more choices for RSS feeds available (Full posts with comments, full posts without comments, and excerpts only). I still haven’t looked into the IE wierdnesses, but that will come. Maybe.