January 2009 Archives

I've been feeling a bit low, lately; staying at home, not working very effectively, feeling burned-out and demotivated.

In part, it's the effect of winter up here north of the 60th parallel — Edinburgh is north of every city in North America except maybe Anchorage; we're about fifty miles north of Moscow, and it's dark for up to 18 hours at a time. But I've got a SAD lamp for that, and besides, I'm used to it.

More interestingly, I was chatting to some friends yesterday and they were noting that everybody seems to be low right now. ("Everybody" for values of everybody in Edinburgh that they'd spoken to, that is.) The economy in the UK is, according to the IMF, due to take a bath in 2009, with shrinkage of 2.8% predicted — the worst recession since the 1930s — and this is making everyone feel a bit grim and pessimistic. Retail sales have fallen off a cliff and large retailer managers were predicting last month that 10% of retail staff nation-wide would be out of a job within 12 months. The housing market is something that, as a homeowner, I don't even want to think about. Our currency nearly collapsed last week, falling to its lowest level in 23 years. Everybody seems to know someone who has lost their job, or worse, to have lost their own: there's an pall of "who's next?" hovering in the air. Even for those of us who, like me, know we've got a solid job through 2009, it's gloomy, and your mood appears to be strongly influenced by that of your second-order friends and acquaintances.

Anyway: is this actually a common phenomenon in economic depressions? That is: are they characterised by pervasive emotional depression (as well as economic malaise) among the population at large? (I don't know anyone who's old enough to have experienced the 1930s depression as an adult ...) And if so, do we have the technology today — in the form of SSRIs and other medication — to fix Depression 2.0 on a global scale?

Academic group blog Crooked Timber kindly asked me last year if they minded me participating in a seminar on my writing.

The results are now up on the web. Here's the intro. Here's Maria Farrell on why she thinks you should read my work. Paul Krugman (yes, that Paul Krugman) discusses the Merchant Princes books. John Quiggin discusses finance and the singularity; Ken MacLeod explores Saturn's Children: Brad DeLong riffs off Asimov's three laws, corporate personhood, and slavery; John Holbo says something I didn't quite catch; and Henry Farrell gives "Halting State" a going over. Finally, I got a chance to respond (Part 1, Part 2).

Lots of stuff here, especially if you want to see what other people make of my work.

Incidentally, that previous blog entry reminds me: people regularly ask me, "well, why don't you use (Windows | Microsoft Office | [insert program here)? Everybody else does, and it would make your life so much easier." Or they ask me "why bother using Linux? It's so much easier to use Windows." And so on.

Well. Why do I swim against the tide?

A good starting place is to read this Guardian opinion piece by the chief executive of the British Library. She starts off: "Too many of us suffer from a condition that is going to leave our grandchildren bereft. I call it personal digital disorder. Think of those thousands of digital photographs that lie hidden on our computers. Few store them, so those who come after us will not be able to look at them. It's tragic." And then she goes on, just barely scraping the surface of a dismal phenomenon that I've been aware of since the late 1980s — purely by accident.

My history with computers starts thus: in 1981, I acquired a Sinclair ZX81. And I played with it for a bit, before deciding it sucked. (I sold it and used the proceeds to buy a programmable scientific calculator, which I still own.) It wasn't until 1985 that I saw a computer that did what I wanted and that I could actually afford to buy (on a student's shoestring): the Amstrad PCW 8256. I loved my PCW, and lavished on it a memory upgrade, and a second 3" floppy disk drive (not 3.5"; the PCW ran on a unique flavour of media that nobody else used), and then — quite radical in those days — a hard disk that cost me two weeks' wages on my first post-graduation job. Then in 1987 I got a much better job, bought myself an IBM PC clone, and ...

But enough about the hardware history. The software history is quite different.

The Amstrad PCW came with a word processor called LocoScript. LocoScript got me through my final year at university and, coincidentally, was what I wrote my first professionally published short stories on. But it was very limited — if your files went over about 40Kb in size it slowed right down, and there was no word count facility. I realized early on that I wanted something better, and I bought a copy of Protext (before LocoScript 2 came out, which fixed most of my complaints). As I was also teaching myself a bit about programming (and from 1987 doing a night school course in computer science) I stopped using LocoScript, and simply started new work in Protext.

When I switched to an IBM PC clone (an Amstrad PC 1512) it had 5.25" floppy disks, rather than the eccentric 3" disks used by the PCW. And the PCW had no serial port (the serial port accessory cost all of £66, a fair bit of money in 1987). So I copied my writings onto a couple of 3" floppes and sent them off to a bureau who, for a small fee, returned them to me — along with a brace of 5.25" floppies. Then I started hunting for a WP for the PC.

Being halfway to broke at the time (I'd just bought my first flat, and interest rates went up roughly 4% over the next year, doubling my repayments) I poked around various shareware libraries first. Along the way, I settled on PC WRITE 2.4 for a while, and wrote a novel on it. (Various ideas, reworked in other forms, ultimately ended up in "The Atrocity Archive" many years later). I was reading magazine reviews, and had a fair idea of what bugged me about existing word processors; so when Borland Sprint came out, I coughed up the relevant amount of money and bought a copy of that, which served me well for roughly five years, by which time Windows 3.0 was sitting on my 386, and I'd acquired a student copy of Word 2.0 for Windows while at University.

... Are you noticing a pattern yet?

In the space of six years, I went through five word processing packages. Being naive at the time I didn't export my files into ASCII when I moved from CP/M and LocoScript to MS-DOS. I learned better, and when I switched from Sprint to Word I halfway ASCII-fied those files; they're a bit weird, but if I really wanted to I could get into them with Perl and mangle them into something editable. Along the way, I lost the 3" floppies from the PCW. Then I had a hard disk die on me — in those days, the MTBF of hard drives was around 10,000 hours — and it took the only copy of most of the early work with it.

Score to 1993: two years' work is 90% lost. And a subsequent five years' work is accessible, kinda-sorta, if I want to strip out all the formatting codes and revert to raw ASCII.

In 1992 I got a Mac (a Macintosh LC, with all of 4Mb of RAM and a 40Mb hard disk). I was also working in a UNIX shop, where text processing was an important part of my job and SGML was the coming thing. And I began to notice something ...

Every time Microsoft bought out a new release of Word, they introduced a new file format. The new version of Word could read documents created by about the last three versions, plus RTF. If you were in business and needed to exchange electronic documents with business partners, you had to upgrade in lockstep so that you could read the files they sent you. This was used quite coldly as a marketing tool, to compel the herd to buy new copies of a word processor — which, by then, was a mature technology. The upgrade cycle was about 18 months to two years long, and I suspect it had more to do with accounting and depreciation rules (so that a corporate customer for MS Word licenses would only have one generation of the software depreciating on the books at a time) than with development time. The upshot was that, unless you took precautions, your documents would become inaccessible due to designed-in obsolescence within about 4-6 years.

I am not in a business with a 4-6 year document retention cycle. I am in a business where I hope that what I wrote ten years ago will still be accessible a century hence. Microsoft's policy was deliberately destroying my life's work.

Of course, Microsoft was not (then) in the business of selling software designed to meet the requirements of novelists; it's in the business of making money by selling software to offices where the average document has a life of a couple of months to a couple of years, and where paper files are routinely destroyed after 5-10 years to save archival storage space. And realistically, how do you go about selling a mature product (word processors) into a market like that? Well, a simple solution is to get the users to give you their data — and then charge them rent for accessing it. Microsoft charged rent in the form of payments for regular rolling upgrades. Now they're pinning their hopes on Cloud Computing, where all your data will be stored in a nebulous cloud somewhere on the internet — sort of like Google Docs and Sheets with a Microsoft tax on top (Google monetize it by advertising, of course).

It's not just word processing. I briefly looked at Microsoft Outlook as an email client, once. It turns out that Outlook stores email in a proprietary data format that only Outlook can easily read. Needless to say, I wouldn't touch it with a barge-pole. Ever since I first got email in 1989, my acid test for an email system is "can I get at the content from outside?" To put this in perspective: last month I rediscovered a DC 6150 tape cartridge containing a backup of emails I'd sent and received in the period 1991-95. I'd thought it lost forever, and indeed, I had no way of reading it. But thanks to a friend of mine who did, we were able to retreive the contents — mailboxes stored in MMDF and Mbox formats (MMDF is similar but not compatible). Both are still in use, and still readable, to this day, using open source clients — or even a text editor (they're simply long text files with individual email messages separated by a header). The tape had been written using tar, a UNIX archiving tool that's been around since the late 1970s.

I can't really blame the big corporations for wanting to seize all our data and charge us for access (either a monthly fee, or by forcing us to pay attention to adverts); corporations behave the way they do for structural reasons. (It's like the fable of the Scorpion and the Frog.) But I don't need to cooperate with them.

As a matter of personal policy, for those activities that involve creating data, I aim to use only software that is (a) cross-platform, (b) uses open or well-published file formats, and (c) ideally is free software.

This is in some ways a handicap; Thunderbird (my mail client of choice) and OpenOffice aren't as colourful and feature-rich as, say, Apple's Mail.app or Microsoft's latest Word. However ...

Firstly, they run on Macs, Linux systems, Windows PCs, and even on some other minority platforms. This protects my data from being held to ransom by an operating system vendor.

Secondly, they use open file formats. Thunderbird stores mailboxes internally in mbox format, with a secondary file to provide metadata. (This means I can claw back my email if I ever decide to abandon the platform.) OpenOffice uses OASIS, an ISO standard for word processing files (XML, style sheet, and other sub-files stored within a zip archive, if you need to go digging inside one). I can rip my raw and bleeding text right out of an OASIS file using command line tools if I need to. (Or simply tell OpenOffice to export it into RTF.)

Thirdly, they're both open source projects and thus the developers have no incentive to lock me in so that they can charge me rent. I don't mind paying for software; where an essential piece of free software has a tipjar on the developer's website, I will on occasion use it. And I'm writing this screed on a Mac, running OS/X; itself a proprietary platform. But the software I use for my work is open — because these projects are technology driven rather than marketing driven, so they've got no motivation to lock me in and no reason to force me onto a compulsory (and expensive) upgrade treadmill.

I'll make exceptions to this personal policy if no tool exists for the job that meets my criteria — but given a choice between a second-rate tool that doesn't try to steal my data and blackmail me into paying rent and a first-rate tool that locks me in, I'll take the honest one every time. And I'll make a big exception to it for activities that don't involve acts of creation on my part. I see no reason not to use proprietary games consoles, or ebook readers that display files in a non-extractable format (as opposed to DRM, which is just plain evil all of the time). But if I created a work I damn well own it, and I'll go back to using a manual typewriter if necessary, rather than let a large corporation pry it from my possession and charge me rent for access to it.

I get mail. Usually I don't answer it in public, but I figure this one might contain useful information, so (with slight editing) here it is:

Hello, I am a member of SFWA. One of my fans pointed me this way and said you had been talking about Linux and EeePCs and what version of Linux you loaded on your machine.

I have a 900A with Linux (as well as a 701 and a 1000HD with XtraPrissy) that I am slowly conquering (yes, I am one of those slaves to Windows OS who has worked her way up from DOS 3.5 to Vista and will admit to loving CE, even thought I understand it really stands for "crippled edition," and gotten very disheartened along the way, and upon learning that Windows was planning to rule the world with Cloud Computing) and starting to fall in love with. So I have decided that Linux might be the way to go — assuming I can learn its ins and outs.

At any rate, I would be interested in learning about which Linux you chose to install (I am finding Xandros for Eee PC unchallenging, and it's advance mode has sent the wordprocessor into crashing regularly), so if you could point me to something on the web that would enlighten me, I would most appreciated it.

I'm going to start by heartily seconding the comment about Xandros on the Eee. Xandros is a commercial derivative of the main "purist" free software distribution, Debian. Debian isn't very newbie friendly — its userbase consists mostly of engineer/sysadmin/CS student folks who tend to roll their sleeves up and take the engine apart. Xandros didn't so much tame the beast as lobotomize it and give it a Mickey Mouse hat to conceal the bandages. While they've planted a brightly coloured user interface on top, and cunningly contrived to make it very difficult for meddling idiots to render their machine non-bootable, they've also stuck it with a number of obsolete versions of key programs that you need to get real work done.

To cut a long story short: for an Eee, I'd recommend a version of Ubuntu, tailored to support the Eee hardware. Ubuntu is another Debian derivative, but this time vastly smoother and more polished than Xandros — and developing rapidly, with a huge user base and active updates to the latest applications. I'd also recommend the Ubuntu Netbook Remix user interface — an application launcher and desktop that does much the same as the Xandros desktop, but is actually useful. This isn't part of the standard Ubuntu distribution, and for full hardware support on the Eee (for example, to ensure wifi, ethernet, and sound work properly) you'd normally need to install some extra drivers. However, the Eee is so popular that there are a couple of sub-distributions out there especially tailored for it.

These are (in no particular order): EeeBuntu, and Easy Peasy (formerly Ubuntu-Eee)

Both of these distros do much the same: they take Ubuntu (typically trailing the current release by a minor version number), add Eee support, and package it for installation on an Eee. EeeBuntu also provides two variants — one with the Netbook interface, and one with the regular Ubuntu desktop. (Note: I can't get to www.eeebuntu.org right now, probably due to a blocked interwebtube thingy.)

To install them: well, you'll either need an external USB CDROM drive, or a 1Gb or larger USB memory stick. You follow the standard Ubuntu installation instructions — but use one of the Eee-specific distributions rather than the standard Ubuntu disk image. Note that before doing this, I'd double-check that I knew where my Xandros emergency restore DVD disk is! Installing Ubuntu will nuke the pre-existing Xandros installation completely, and if you want it back you'll need to reinstall it from the DVD. (For this reason I recommend that Eee owners get themselves a cheap USB CD or DVD drive. Mine, a slimline USB-powered DVD-RW with no external power brick and a colour scheme matching my Eee 1000, cost about £35 from Hong Kong via eBay, but you can find cheaper ones if you're less fussy.)

Note that Ubuntu tends to assume that you want to use a standard set of applications: OpenOffice for office documents, Firefox as a web browser, and Evolution as an email client.

I don't use Evolution (I use Thunderbird, because I can haul my profile between Linux and Mac, or even Windows), so the first thing I did on installing Ubuntu was to yank in Thunderbird (and a few other useful odds and ends). The tool you use to install and remove software on Ubuntu is called Synaptic, and that link leads to a basic user guide. Don't be surprised if, when you request it to install one program, it warns you that it's going to install a bunch of other stuff — Synaptic's back end, apt, is designed to keep track of what's on your system and make sure that if you install something, all the prerequisites it needs to work are also installed at the same time.

One gotcha relates to OpenOffice: Ubuntu 8.10 (the official current release) provides OpenOffice 2.4. The current OOo release is 3.01; this should show up in Ubuntu 9.04, due out in April 2009. However, the Easy Peasy distribution (which is based on Ubuntu 8.10) adds in OpenOffice 3.0 and Firefox 3. This is slightly unexpected (these tailored distributions usually lag behind the mainstream), and if you're planning to schlep documents between an Easy Peasy based Eee and an Eee running Xandros — which is still stuck on OOo 2.0 — you'll need to keep an eye on the file format you're saving documents in.

Final caveat: I'm not using either of these distributions on my Eee 1000. I'm using stock Ubuntu 8.10 with a patched kernel and a set of packages that provided the Netbook remix interface separately. On the other hand, Easy Peasy and EeeBuntu weren't available when I set this machine up. I think they'll do what it says on the tin, but I haven't verified this.

Anyone got anything to add on this topic? (Suggestions that my correspondent should upgrade to (Vista | OS/X | FreeBSD) or buy a real computer will be mocked, mercilessly.)

I just want you to know that some time this week I rolled past 600,000 words of Merchant Princes saga, and I can feel every damn one of them weighing down on my back.

40,000 words to go then I can give it a rest for a while and maybe do some blogging or whatever it is that sane people do with their lives when they're not working their way up to the climax of a 1600-page saga.

In other news: I want you to try to remember that Barack Obama is not the messiah. He's just a very skilled, very intelligent politician. If you expect him to walk on water, feed the masses, and raise the dead you will almost certainly be disappointed. Remember that: almost certainly.

Existential horror: it's not just for breakfast.

H. P. Lovecraft didn't invent horror, but he pretty much pioneered the first open-source horror mythos; a universe mind-bogglingly ancient and vast (he had Edwin Hubble's cosmology to work with, not Bishop Usher's), populated by mind-numbingly alien beings, around whose feet we are as dust. And in this eschatology Lovecraft found room for a particularly chilling apocalyptic resonance; for one day, when the stars are right, that which is not dead but sleeping will awaken and return to earth, there to impose its unspeakable and nightmarish will upon those of us who survive. Or something like that.

Sort of like this.

It occurs to me that the return of the Old Ones in the Lovecraftian mythos shares quite a lot of things with the other hoary staples of western mythology; of Armageddon and Apocalypse, of the fear of total nuclear annihilation that those of my generation grew up with, even — to pull a science-fictional twist — of the singularity (which isn't known as the rapture of the nerds for nothing).

There are some subtle differences, of course. As with the singularity, what comes after the stars come right is inconceivable by default: we are no longer the dominant species in the intellectual food chain, able to map out and name the universe around us — we are, in fact, as dust beneath their feet. In contrast, the end times of Christian eschatology are fairly extensively described (idyllic heavenly whatever for the godly, eternal boiling sulphur for the rest of us). Thermonuclear armageddon tends towards the justified-punishment-of-the-sinners in fiction; what comes after is described in gruesome detail (for example in Threads or A Canticle for Liebowitz).

But is it good horror?

My take on horror is that it is a branch of literature that reflects the human condition distorted almost out of recognition. It's there to tell us something about ourselves — frequently something unpleasant (although there is scope for redemptive messages if the author is so inclined.)

You can apply horror to just about any other fictional form by spray-painting it with a thin wash of atomized blood. Weirdly enough, in this respect horror works just like humour; you can write humorous fantasy (a hat tip to Sir Terry here), or humorous romance, or humorous hard-boiled detective stories. So why not layer the two tints? Spray horror atop humour, or vice versa, to add a sick laugh at the apocalypse in order to bring it back to earth in the fertile soil of human concerns. It worked for Stanley Kubrick in Dr. Strangelove, one of my favourite movies; it's also the material I was working with in "The Atrocity Archives" and "The Jennifer Morgue".

The former of those novels started out with a dry run in 1998, a short story titled A Colder War. There's nothing terribly funny about "A Colder War": I was groping in the dark for a way to express the alienating horror of nuclear annihilation that I'd grown up with, and Lovecraft's monsters came perfectly to hand. The existential dread they evoke is not so alien to those of us who lived through the original Cold War. But I couldn't write a whole novel in that key — there's no redemptive message in a holocaust without survivors. Hence the leavening of dark humour in "The Atrocity Archives", and the somewhat lighter tone of "The Jennifer Morgue".

For some reason I don't want to examine too deeply, I'm rather attached to those two novels — to the extent of having written a third, and having vague plans for at least two more. But schoolboy humour isn't enough to sustain or motivate a series work, so I've been floundering around looking for thematic lessons to bolt atop the structure of the Laundry series. And it occurs to me that the Lovecraftian apocalyptic singularity is underexplored. In a nutshell, it poses this question: what happens when we take the human condition, and twist? You need a topping of gallows humour just to keep it in perspective: humour is a brutal necessity when you're confronting the horrific on a day to day basis (as anyone who hangs out with medics can probably attest).

What's the role of humour in this universe? Well, one might ask what Stanley Kubrick intended when he turned "Dr. Strangelove" into a theatre of the absurd: absurdity is generated by dissonance between a situation and its meaning, and Kubrick used it to viciously anatomize the process of atomic annihilation and hold up the petty and banal motives of its perpetrators to ridicule. But "Dr. Strangelove" didn't laugh at what came after the bomb — it ended, on a double-blind ironic note (singing "We'll meet again" to a background of mushroom clouds). The bomb was the punch-line of the joke, not the set-up. What happens in a survivable apocalypse? Lovecraftian apocalyptic fiction never actually explores the consequences of the Old Ones returning, let alone the human wreckage left behind in the aftermath. It's like the Singularity in SF, circa 2000 — off-limits to exploration.

(Clears throat.) This isn't a manifesto. It's just an explanation of what I've been writing, and what I plan to write more of. It's probably best described by a portmanteau word: Strangelovecraftian (or, if you're in a hurry, Strangecraftian) fiction. It's goal is to use the eschatalogical horror of the Mythos much as recent SF has used the Singularity, to shed light on the human condition under circumstances that warp the soul.

And now, if you'll excuse me, I've got to go find a cave to gibber in ...

I was going to post something intelligent today, but my brain crashed and the words refused to line up.

Just so you know: I aten't dead yet, and normal service will be resumed shortly.

Because it's Friday, and we have a video camera, here's Mafdet, wondering why we're poking a box in her face instead of giving her all the cat treats in the observable universe:

(If you're wondering whether this is a dry run to see if I can videocast from my office, the answer is "definitely maybe".)

It's officially published today. Yes, you can buy it now (if you dare). Despite the bizarre misconceptions held by Amazon's stock tracking database, this is not, in fact, a large print edition; so if you read it on the bus the annoying guy in the seat behind you will not be able to read it over your shoulder! Fact.

In other news: I do not do Facebook, but I am reliably informed that there is some kind of fanatical minion indoctrination site over on Facebook. And you can find news of upcoming author events there, like, er, this one.

(What, you don't do Facebook either? Well: I'll be doing a Q&A, maybe a short reading, and a signing at Pandemonium Books in Cambridge, MA at 7pm on Tuesday February 10th.)

In other news: it's open season for Hugo award nominations (if you're a voter from last year or a member of this year's worldcon). I've got two items which are eligible this year: a novelette, Down on the Farm (published online on Tor.com), and the novel Saturn's Children (extract here). If you nominate either or both of these works, I promise not to spontaneously combust from sheer excitement. Fact.

Finally, in case you've had a particularly shouty day and are tired of being shouted at, here's a video of a hamster eating popcorn on a piano.

* Metacrap — why ubiquitous, complete metadata about our surroundings is an impossible dream. (Jacqui Smith take note.)

* Related: 10% of records in one HMRC database are incorrect. Money shot: "The frameworks database only contains quite simple information — first, second and surname, title, sex, data of birth, address and National Insurance number."

* These people produce trend maps like these: 2009, 2008, 2007. What's your take on next year's trends?

* The carbon footprint of a cheeseburger. (An oldie but a goodie that bears repeated re-reading).

* More on desktop fabricators. (A fab shop just opened up on The Grassmarket in Edinburgh's old town — bespoke model manufacturing, presumably aimed at architects and mechanical engineers initially.)

* First Falcon 9 launch vehicle nears completion (That's the smaller, cheaper, faster space program on the pad, for only £120M per flight.)

* Soviet Military maps of Great Britain.