Technical Notes and Solicitations

Posted on May 25, 2007    Posted by John Scalzi

For all of you wondering how I might possibly have blown up the site yesterday, here’s what’s going on.

This blog, like most blogs, has a SQL database, in which all the data pertaining to the blog resides. The permitted size of that database is 100 MB, and currently the database is well above that size. The technical folks at 1&1, my host provider, say that it’s not like I’m going to get in trouble for this, but that the database itself has an increasingly large chance of becoming corrupted as time goes by. I’m not geek enough to know anything about MySQL, so I can’t judge the accuracy of this statement; be that as it may I’d rather not have the Whatever go kerplooey.

At the same time the Movable Type infrastructure of Whatever is becoming increasingly unstable, partly due to age and partly due to the fact I am apparently not competent enough to update the software, and every time I try I break something with the way it works and I am not savvy enough to fix it when I do. Basically the place is coming down around me, albeit in slow motion, so none of you on the outside would notice.

This presents both problems and opportunities. The opportunity lies in that this would probably be a fine time to do a significant revamp of the Whatever and either do a complete, clean installation of MT, or look at some other option that is robust enough technically to handle 25,000 visitors a day. The problem lies in dealing with previous entries (and attendant media and their URL paths and etc). The last time I did a major revamp of the site, back in 2003, my solution to this issue was simply to leave out everything I had written prior to the switchover. I don’t think that’s an optimal solution now.

Underpinning all of this is the database issue; I think it would be lovely to leave the present database as is, as an archive, and plug in some sort of blogging software to access the entries/comments therein while the “new” Whatever runs off a new database. The problem with this is that I don’t know how to do it. I tried doing it with a WordPress install I did on my scalzi.info domain (very easy to do, incidentally — so much easier than MT), but was confronted with two problems: Just making the WP install write to the current database does not make it display the information therein, and using the “import” function is not useful, in no small part because I can’t get my MT install to export a significant percentage of the blog posts using its export function. I’ve either screwed up the software in some fundamental way or it’s simply that the Whatever is too damn big (the third option is that I’m incompetent in using the software, which is, as noted earlier, a very big consideration).

So, in sum: This is what I’m thinking about going forward —

1. Looking at blogging/community software to maintain and build out Whatever/Scalzi.com and possibly some of the other domains I own (which is to say, possibly adding the option for a place for folks to post their own entries, etc) — something that can point to more than one database for additional iterations of the software would be optimal;

2. Creating a static archive of Whatever posts to this point (i.e., no additional comments or entries into that database) that is easily accessible — and if at all possible still maintaining the previous URLs, so everything isn’t broken;

3. Finding someone who is more technologically competent than me to help me build this out and install it for me. No, I wouldn’t expect this work to be done for free.

I’m taking suggestions on how to accomplish all three, so if you have ideas for software, how to build out an archive (or even if it’s necessary), or if you feel you’re just the sort of geek who can help me with this sort of project, go ahead and comment in the thread or drop me an e-mail.

(CAVEAT: Please don’t comment in this thread just to comment, or to offer “advice” that’s not actually on point, i.e., don’t be “helpy” when I actually need helpful here, because that’s just going to irritate me. There will be other comment threads to be silly or tangential. Please make this comment thread on point. Thank you.)