I'm cheating here: if you want to understand "Singularity Sky" as published, you need to read this earlier piece I wrote (which uses it as a springboard for discussing why I killed off the Eschaton novels after book #2).
What I'm going to add here is merely the history of the project. Which, with 20/20 hindsight, was a nightmarish mire of despair ...
Rewind to 1996. I was living in Edinburgh, working for a web consultancy that was in the process of going bust. The year before, I'd handed in the way-overdue manuscript of "The Web Architect's Handbook", a non-fic get-rich-quick scheme originally proposed in 1993 which crashed and burned in the market because folks who weren't working 60-80 hours a week as CGI app troubleshooters got their how-to books out first. I was writing about an article a month for "Computer Shopper" (the British mag, not the Ziff-Davis title) and I'd sold one short story that year — a reprint. I was 32, I'd sold about a dozen or two short stories, signally failed to sell a novel while everyone else I knew who'd begun selling short fiction through Interzone at the same time had become a household name (Pete Hamilton, Steve Baxter, Paul McAuley ... do I need to continue?), and was having a crisis of confidence.
What do you do when you have a crisis of confidence in what you've seen as your vocation (write science fiction) since age 15? You either give up completely, or you double-down.
Contemplating the smoking wreckage of my first decade of writing and selling SF, I concluded that I was Doing It Wrong. I'd been selling short stories to British magazines and anthologies, hoping to build a name and visibility and acquire an agent and a publisher for the novel manuscripts I was producing at a rate of, roughly, three a decade. But the British short fiction market was ... well, nobody paid much attention to it. And my experiences with British literary agents were, shall we say, not terribly good. (1996 was the year my second agent fired me.) So: if one wants to write SF and do nothing else, it follows that one needs to be successful enough to earn a living at it, which means cracking the North American market, because as Willie Sutton said when a journalist asked him why he robbed banks, "that's where the money is". (Not that there's much money in SF publishing anywhere, but there are more readers in the 350-million strong market of the USA and Canada than in the 60-million strong market that is the UK.)
So, I worked up a task list. Item: sell stories to the Big Three magazines (Asimov's SF magazine, Analog, F&SF). Ideally get shortlisted for the Hugo and Nebula. (Yeah, right. As if that'll ever happen.) Write novels. Each novel must be #1 in a series in a different sub-genre, but don't write #2 next — go do something different while #1 is slumbering in a cobweb-afflicted slushpile.
(Digression time: Received wisdom in the 1980s was that unsolicited novels would be glanced at and returned or looked at more intensively within 2-3 months. But the 1980s saw the advent of affordable word processors, and you do not want to know what the combination of word processors and schizophrenics with hypergraphia did to your average publisher's open slushpile. By the late 1990s turnaround times for submissions were up to a year and climbing—and submitting a novel to more than one publisher simultaneously was and remains a big no-no: it's a small world, editors at rival houses talk to each other, and they don't like time-wasters. If you want to run an auction you need to get an agent, which is a whole 'nother story.)
My end goal was to get a publisher interested to the point of issuing a contract — then, rather than signing it (a mug's game), to sign up with a young, hungry literary agent who would smoke out the booby traps hidden in the legal boilerplate and hopefully extort a bigger advance. (Note: the issue of getting a literary agent is one of the most vexatious Catch-22 situations I can think of. I'll try to remember to rant entertainingly about it at a later date.)
I began writing short stories: 3-4 a year. And the first place I sent them was Asimov's SF Magazine. Where Gardner Dozois read them and knocked them back, but with an actual written explanation rather than the regular checkbox form. (Asimov's back then were averaging 30 submissions a day. Getting an actual human-drafted rejection letter meant you were in the top 10%.) After a while I got dispirited and began sending them to a local British SF magazine first, which was a mistake: that's why "Antibodies" and "A Colder War" came out in Spectrum SF (which you haven't heard of) rather than debuting in Asimov's and maybe making a splash.
In the meantime, I set to work on The Novel Project. I finished "Scratch Monkey" in 1993. I'd then written another novel-shaped object, which in 1996 I took out behind the barn and shot — the recyclable bits turned up later in "The Atrocity Archives" and "Antibodies", but John Jarrold's lengthy rejection letter (he was editorial director at Earthlight, Simon and Schuster's UK SF imprint) shone a sufficiently bright light on precisely why it was unpublishable that even I couldn't kid myself that it was salvageable. So: what to do?
Enter the idea of the ACME General Purpose Space Opera Universe. This being 1996 or thereabouts, I needed to figure out a way to dodge the implications of the Singularity for space opera that Vernor Vinge had so irritatingly pointed out to us in "A Fire Upon The Deep". So I decided to embrace it with open arms, give it a name and a face, and make it a plot McGuffin. Hence the Eschaton and it's human agent (Martin, initially a bit of a cipher), and the rather more interesting interstellar arms control inspector Rachel. Who were investigating a tap-dancing-around-the-rules gambit being contemplated by the space navy of a backwater empire, in a novel titled "Festival of Fools".
I'd been reading too much David Weber at the time, and noting the uncritical enthusiasm with which readers seemed to receive his tales of Napoleonic Navies in Spaaaaace. Alas, I'm prone to tearing apart my toys, and after a couple or eight books in the series I began to ask questions. Such as, why are the opponents in this sub-genre of space opera always evenly matched? Surely in a diverse space operatic universe you'll occasionally see a Napoleonic space navy run into a nuclear-powered hunter-killer submarine? Or the equivalent of wooden tall ships encountering an unarmed modern ice-breaker ... on whose decks some desperate amateur has parked a TOS-1 Buratino? (Wooden ships and thermobaric warheads just don't mix.)
You can also note my nascent aversion to absolutist monarchism coming out here. We in the modern world have a technical term for absolutist monarchies of the kind who deploy Napoleonic Space Navies (or would, if NSN's existed): we call them hereditary military dictatorships. Poster child: Kim Jong-Un. Or maybe the House of Saud: I digress. Let's just say that the political systems in most military space opera really suck.
Final ingredient: if you're going to play in the David Weber sandbox, or even satirize it effectively, you need a military campaign. I decided to pick on the most barkingly insane naval expedition of recent history, the voyage of the Russian Baltic Fleet commanded by Admiral Rozhestvensky in 1905, during the Russo-Japanese war. TL:DR; the Imperial Japanese Navy bushwhacked the Russian Pacific Fleet in the opening weeks of the Russo-Japanese war (to be fair, the Russians started it: "what this country needs is a short, victorious war" as one of the Tsar's ministers put it). They then laid siege to Port Arthur. The Russian response was to tell Rozhestvensky to do the impossible — to sail his fleet of superannuated and obsolescent coal-burning battlewagons from their port on the Baltic (north of Europe) the long way round to the Sea of Japan to lift the siege. Wikipedia says, "Under Admiral Rozhestvensky's command, the Russian navy holds the record of sailing an all-steel, coal-powered battleship fleet over 18,000 miles one way, to engage an enemy in decisive battle." What wikipedia doesn't say is just how dumb this was, because the battle in question was the most decisive victory in a fleet engagement since the Battle of Trafalgar in 1805—for the Japanese.
Seriously, go read that wikipedia article. When you're done, go read the wikipedia article about the Dogger Bank Incident as well, then boggle — why on Earth did the Russian navy expect to encounter Japanese torpedo boats in the North Sea of all places? (Nearly lighting the fuse for the first world war nine years early, for the British government was Not Amused ...)
Anyway. I wrote 130,000 words about the most insane, doomed space operatic navy campaign in future history. And then I realized I had a Problem. The fleet was shortly to arrive at it's destination ... and I hadn't bothered to give them an adversary. Trust me, plot-driven mil-SF, even the satirical sub-species, doesn't thrive without an enemy to go up against.
It was August 1997 and I was wandering around Leith with friends, looking for a pub, and as I frequently do, I was chewing over my dilemma in the presence of an audience. And as I recall (warning: my memory of conversations held 15 years ago in the presence of alcohol is less than reliable) I was muttering aloud along the lines of, "what I need is a threat they don't understand, one that they can't understand." And a friend said, "something like the Edinburgh Festival Fringe?"
The whole reason we were wandering around Leith in search of beer in the first place was that we'd been driven out of our normal
den of iniquity pub by the presence of Festival-goers. Edinburgh in August is a city on the receiving end of an alien invasion spearheaded by unicycling mimes and bagpiping elephants. Add the fleeting twilight nights (we get maybe 4 hours of complete full dark at that time of year) and the pervasive random weirdness—you can go shopping dressed as a Dalek or a 17th century French aristocrat and nobody will blink at you—and it seemed like the perfect metaphor for what the New Republican Navy was going up against. Result!
So I ripped the guts out of the novel — I think I ditched 80,000 words, a full-length novel's worth in its own right a decade or two earlier — then wrote the opening sequence, complete with the rain of telephones (because I'd just acquired my very own first personal cellphone) and it was the sort of thing that something like the Festival would do that would be totally incomprehensible to the NR.
And from there it was downhill all the way, until I finished the novel in 1998.
Now for the confessional about what happened later, and my own fit of unprofessional behavior.
Intending to have a crack at the US market, I sent "Festival of Fools" to an editor at Tor who I happened to have met and who I thought wouldn't leave it on the slushpile unread. Unfortunately my Tor kremlinology was underdeveloped in those days. I was aware that individual editors run their own lists. I was aware that I was submitting to a fairly senior editor. What I didn't recognize was that Patrick Nielsen Hayden was Tor's editorial director, massively overworked at all times: and that his backlog was a thing of legend, spoken of in hushed voices in the halls of New York Publishing.
And after about a year Patrick stopped answering my 3-monthly email queries.
Fast-forward to April 2000, and the British Eastercon, held that year in the Central Hotel in Glasgow. I mostly recall that convention as being rather badly programmed. (At one point I was sitting in a bar, kibitzing on a three-way discussion of the state of Scottish SF between Andrew J. Wilson (SF reviewer for The Scotsman), Ken MacLeod, and Iain Banks ... none of whom had been offered a seat on any of the panel discussions at the con, much less a chance to discuss the state of Scottish SF in front of an audience with some advance warning.) But the bad programming had a silver lining, as it turned out. I ran into an old friend, Ben Jeapes. These days Ben is best known as a British author and academic publisher. But back then, he was making a decent fist of setting up a small British SF press, Big Engine. And some years earlier we'd workshopped together. I was grumbling about the lengthy slushpile delays to Ben, and he said, "well, I see where you're coming from: how about letting me see it? If Patrick wants it he can have it, you sent it to him first: but I wouldn't mind reading it. It sounds interesting." So I sent "Festival of Fools" to Ben, and he said, "there are some changes I want making, but if you make them, I'll buy it." And I sent a "sorry, but if you don't tell me you want it I'm going to withdraw it from submission" note to Patrick. Who didn't reply.
(Many years later I mentioned this to a different Tor editor. And she said, "oh yes, I remember that! Patrick was going to buy it and hand you over to me to edit. But he never quite got round to making the offer." I like to think that, if we live in a multiverse, somewhere in one of the other multitudinous trouser-legs of time there is a world where my primary SF publisher is Tor and my editor is Teresa Nielsen Hayden. But that's not how things turned out ...)
Anyway: the consequences of my fit of unprofessional behaviour were that I landed a rather poor contract from a British small press. But I had enough wits not to sign immediately. Instead, I went looking for an agent based in New York. And found one at exactly the best stage in the literary agent life-cycle from an author's point of view: a publishing insider (actually a senior editor at an SF imprint, changing jobs because being an agent is easier to combine with having babies — it's easier to work from home) with no existing clients above me on the totem pole. (I can guarantee you that the best way for a new author to get a literary agent's undivided attention is by opening with "I have an offer from [PUBLISHER] sitting here and I haven't signed it yet ...") Caitlin rolled up her sleeves, went to work, removed the landmines from the Big Engine contract, then auctioned North American rights to the book. (Tor — in the shape of the inimitable David Hartwell, who is my editor for the Merchant Princes series — and Ace turned up to bid: Ace's opening offer was slightly better and Tor didn't counter-offer.) Then headaches happened.
Editors like to edit, unless they're too busy managing production workflow on a list that handles hundreds of books a year. Ben edited "Festival of Fools" his way. But Ace also wanted to edit it. One edit went into British English; the other went into American English. The two drafts began to diverge subtly. Ace decided to retitle the novel, because Richard Paul Russo had a book out titled "Ship of Fools" and they didn't want to confuse everybody; could I come up with a new title, and by the way, could it have the word "Singularity" in it? (The Singularity was hot in 2001.) But Ben wanted to stick with the original title. Checking copy-edits and page proofs is time consuming for the author, and doing it twice for subtly different versions of the same book is ... well, you don't get paid any extra money for doing it twice, and it's a lot of extra work. Finally, the publishing schedules began to diverge. Big Engine was running into cash flow trouble. Ben recognized this, and finally took the decision to shut down his company ... one month before "Festival of Fools" was due to be published in the UK, and about a month or two after a very similar but non-identical novel titled "Singularity Sky" debuted in the United States.
Caitlin had done her work on the contract properly: we re-acquired the British and Commonwealth rights to "Festival of Fools" without any great difficulty. She then sold the American variant, "Singularity Sky", to Orbit, which is why the British edition came along 1-2 years after the American publication and, to this day, I am effectively marketed in the UK as an imported American author. And why I always, ever since, have made a point of trying to use American spelling in my novels and ensure that only one publisher gets to edit any given novel.
Final note: I wrote FoF from 1996 to 1998. Gave it a polish in 1998/1999 before submitting it to Tor. Had to edit it again in 2000 for Ben, then again for Caitlin before the US agent-mediated auction. Then copy edits for both Big Engine and Ace in 2001. Ace published it in July 2003; Orbit in 2004 (if memory serves). Upshot: it took 7 years from writing to first print publication. And this is now not unusual for a first novel. The subsequent books came along much faster ...