Welcome to my blog.
I normally get around 10,000-11,000 readers a day poking their noses in here. By blog standards, that's a bit hardcore — but it seldom spikes over 15,000 visitors, even if I post something interesting enough to get BoingBoing'd or front page coverage on Hacker News or Slashdot. The most visitors I'd ever seen until yesterday were 120,000 on two consecutive days, when I bloviated about the iPad and struck a nerve.
It's a good thing I upgraded my server last month ...
In the past 24 hours, the blog entry I posted yesterday has had 1.44 million readers. No, I'm not making that up; I wish I was. I burned through about a year's worth of bandwidth in one day. The comment system turned to sludge and I had to switch it off for a while. The hits are still incoming (many thanks to Neil Gaiman for tweeting the URL to his million and a half followers!) and I've taken down all the comments (and switched off comment posting) on that article, reducing its page size from 250Kb to 56Kb in the faint hope of not having to, like, pay for the consequences of the entire internet suddenly deciding to flashmob my server.
I don't have an exact breakdown of what just happened, because my logfile analyser is still churning, from 2am last night. That's because the Apache logfile today is over 500Mb — it's normally about 150Kb.
1. Static HTML, with no graphics, is king. If you accidentally break a major news story, it will save your ass. They say one picture says a thousand words, but a thousand words of HTML is about 7-8Kb; a picture can easily be ten times the size. And dynamically generating content on the fly is expensive; while my server could probably keep up with 10k visits/day, 100k would kill it. By pre-generating the pages, my blog was able to keep up with millions of visits/day — as long as it wasn't running CGI scripts to regenerate the comments.
2. Even a PC with mundane desktop specs (dual 2.4GHz Athlon, 2Gb RAM, 1Tb disk space) can serve pages at an insane pace today — it may be a budget box, but it kicks sand in the face of the serious production servers I was working with at Datacash in 1998-2000.
3. You can optimize the hell out of CGI scripts but that won't do you much good if your load spikes by two orders of magnitude. So: aim to bulletproof your server for a one order of magnitude load spike (to cover every case but the once-a-decade flashmob), and be prepared to degrade service gracefully (e.g. by disabling comments) if you get that mobbing.
4. Tune Apache now — you won't get a chance to do it when the stampede arrives!
We now (I hope) return you to your regular scheduled service. If things go dark, it's probably because my hosting company pulled the plug. If they didn't, I'd like to take this opportunity to plug the hell out of ByteMark!