Charlie Stross: February 2010 Archives

I'd like to tackle two common misconceptions about publishing in this piece. Firstly, a lot of people who should know better — business journalists covering the publishing industry, for example — seem to think that authors sell the copyright on their books to their publishers. And secondly, a lot of readers think that if a book is available in print in the English language in, say, the United States, they ought to be able to buy it anywhere in the world. This might be true in a practical sense, but in a legal context it's anything but — and with more and more ebook readers trying to buy titles internationally and running slap-bang into software-enforced geographical blockades, it's time to explain why.

Unfortunately I'm not a lawyer. I'm going to invite a couple of folks with law degrees who specialise in intellectual property law to kick the tyres on this post, but it may still contain inaccuracies — if you're selling a book of your own you must not rely on me for legal advice because I'm not qualified to give it. Also? It's incomplete, and merely represents a worm's eye view of book contracts from the perspective of an author of commercially published genre fiction.

(And it's so damn long that I didn't want to clog the front page of my blog up with it. So to continue reading it click the link below ...)

It is a common misconception — to paraphrase a commenter in the previous post on common misconceptions about publishing, that "the only two people that matter are the author and the reader (one puts creativity in, the other money: the rest add cost)".

This is a bit like saying that in commercial air travel, "the only two people that matter are the pilot and the passenger (the rest add cost)". To which I would say: what about the air traffic controllers (who stop the plane flying into other aircraft)? What about the maintenance engineers who keep it airworthy? The cabin crew, whose job is to evacuate the plane and save the passengers in event of an emergency (and keep them fed and irrigated in flight)? The airline's back-office technical support staff who're available by radio 24x7 to troubleshoot problems the pilots can't diagnose? The meteorology folks who provide weather forecasts and advise flight planners where to route their flights? The fuel tanker drivers who are responsible for making sure that the airliner has the right amount of the right type of fuel to reach its destination, and that it's clean and uncontaminated? The designers and engineers at Boeing, Airbus, Embraer, or the other manufacturers who build the bloody things in the first place ...?

I hope you can see the point I'm trying to make. To be direct: a manuscript is not a book. The author's job is to write the manuscript. The publisher's job is to turn a series of manuscripts originating from different suppliers into consistently produced books, mass-produce them, and sell them into distribution channels.

Note the phrase "series of manuscripts". Small outfits like Golden Gryphon Press (who published the first two of my Laundry novels in hardcover) work on a handful of individual titles (in GG's case, 1-3) in any given year; each book gets special attention and is handled as a separate job. Larger publishers — be they recently-graduated-from-small-press outfits like Subterranean Press or operations like Tor operate on a production line basis. Tor publish 250-300 books a year. Each incoming manuscript goes through a series of production stages, and if there's a hold-up they lose their slot in the publication queue (unless the book is very, very special, for major-bestseller values of special).

What is the production workflow for a book by a professional author working under contract with a publisher?

1. The author writes a manuscript. Note: a manuscript is not a book. It's a bundle of pages of written text (or a text file on a computer: most big publishers these days insist, for no sane reason, on receiving a Microsoft Word 2003 .doc file — probably because it's the commonest format people use with support for italics, underlining, boldface, and headings). The author may suggest a title (the publisher doesn't have to accept their suggestion). The manuscript has been worked over and polished by the author to the best of their ability, but will inevitably contain typos, grammatical infelicities, continuity errors, factual errors, internal inconsistencies, and muddy bits where the author was typing on autopilot. (And these are the good, publishable manuscripts from authors with a track record. For some insight into how bad the bad stuff can get, read this and weep.)
2. The author or their agent send the commissioned manuscript to their editor. (I'll write about agents and acquisitions in a future entry.) Note: outsiders often have some strange ideas about what editors do. These days, an editor is a middle manager. Their job is, in conjunction with a marketing manager, to determine what titles to acquire for their list; to negotiate deliverables and deadlines with authors: to provide high-level guidance and feedback to authors: and to ride herd on the production pipeline so that Sales and Marketing receive each quarterly or triannual batch of new titles on schedule. Editors not infrequently enjoy editing, but there's a lot more to the job than that. Here's one editor's perspective on how you wind up in the job.)
3. The editor reads the MS. If it's in need of more work, they tell the author what they want doing. ("It needs a purple singing dinosaur and a surprise ending. Oh, and get rid of the DEATH guy, he's a downer.") But more often than not, they accept the manuscript. This notifies Accounting to pay the author (or their agent) the second tranche of the book advance (I'll talk about royalties, advances, and how authors make money in a future posting), the Delivery and Acquisition money. They then work out where to slot the MS into their production queue.
4. Marketing. In general, the marketing spend on a book is around 5% of the dollar value of the books the publisher expects to ship. It may be more if the publisher is trying to build this particular author's sales. An editor talks about marketing, including where the money goes. Note: this activity starts when the book is delivered, before it is published — but the budget is based on expected revenue which won't be known until the orders are in. So to some extent the degree of marketing push depends on the editor's (or marketing director's) best guess as to how well the book will do.
5. Scheduling. This is hammered out in the rough before the contract commissioning the book is even signed. Generally, large publishers like authors to deliver one MS per twelve months, like clockwork, because their production line is geared to turn manuscripts into published books in twelve months. However, the clock only begins ticking when the MS is accepted by the editor: so the editor looks at the calendar about 12 months ahead, works out where there's room to slot the book in — ideally in a month preceding the summer or pre-Christmas sales peak (if the book's likely to sell well) — then works backward to sort out when the various steps in the process must take place. Note that it doesn't take 12 months to produce every book — but 12 months is ample time to process the slowest book in the queue, by the recluse who doesn't use email, writes with a quill pen, and is always late reviewing the page proofs.
6. Copy editing. The MS is sent to an external copy editor (usually a freelance editor specializing in the genre in question). The copy editor's job is to keep the author from embarrassing themselves in public. To this end they need to regularize spelling and typography and nomenclature, spot inconsistencies and obvious errors, fix spelling mistakes, and telepathically read the author's mind to ensure that the manuscript reflects the author's pure and original intent rather than looking as if a barrelful of monkeys had a fight for ownership of the keyboard. Traditionally, copy editors work on a paper MS using red pencils to indicate their changes; larger publishers are now switching to all-electronic workflow, hence Word change tracking and notes. The result, a Copy-Edited Manuscript or CEM, is returned to the editor within 2-4 weeks.
7. The author reviews the CEM. (For some reason my publishers' editorial assistants seem to love to send me CEMs for review the week before Christmas, with a due-back deadline of January 4th. It's especially fun when two publishers pull this stunt simultaneously. Perhaps they think nobody loves me and I welcome the distraction ...?) Alas, copy editors are not, in fact, telepathic. Sometimes they get stuff wrong — rarely, they get lots of stuff wrong. So when the editor receives the CEM they bounce it to the author for review. The author can override the copy editor's changes and, if necessary, add corrections of their own. There are usually multiple changes per page — ranging from trivial stuff (serial comma policy at $PUBLISHER differs from author's usual style) to "you've referred to your hero as Joe sixteen times and Jim fourteen times in the MS — which is right?" The author typically has up to four weeks to review the CEM and return it to editorial. If it's on paper, the author must use a different colour of crayon from the copy editor, otherwise whackiness ensues. Electronic workflow involves change tracking and lots more irate marginalia, followed by hoping that Word or OpenOffice don't corrupt the file.
8. Advance Reading Copies. If the book is going to be promoted heavily, the editor arranges a production run of ARCs. These are trade-paperback-sized print-outs of the manuscript, sometimes with rudimentary typesetting, and a non-glossy cardboard cover. They go to reviewers (the lead time for a book review in Publisher's Weekly, Kirkus, or another news outlet is several months), other authors (if a cover quote is being solicited), and sometimes to bookstore buyers to promote a title that's expected to sell well. Note that despite its crude appearance, an ARC costs a lot more to produce than a finished hardback — it's laser-printed and hand-bound, in small numbers, so not all books get ARCs. Someone — typically the office intern or editorial assistant — has to mail the ARCs out.
9. Book design, cover design, front and back flap copy, and cover artwork. The editor pulls together a description of the book and/or the original manuscript. These are used to brief the art director, who if necessary commissions an artist to produce a painting. (Cover paintings may not be necessary if the fashion is currently for abstract/design-driven covers in marketing.) The art director also works out a suitable cover design for the book. A marketing/blurb writing person/editor also writes the flap copy (the text on the back of the book that makes you want to read it). The goal of book design is to motivate shoppers unfamiliar with the author to pick the book up. Nothing more and nothing less. (Retail psychology tells us that shoppers who handle a product are more likely to buy it. Existing loyal fans will buy it anyway. So the book design is aimed at appealing to new readers.)
10. CEM delivered to Typesetting. Publishers these days own neither internal typesetting departments nor printing presses. What happens at this point is that the copy edited manuscript is sent to an external typesetting bureau, where a typesetter with a workstation running either Quark Publishing System or (increasingly, these days) Adobe InDesign sets up a Book project, imports the styles specified by the publisher's standard book design, and imports and reflows the CEM — making sure that all the changes are incorporated into the DTP file. If the book is still being produced with paper-on-pencil workflow, at this point the poor bloody typesetter has to go through it page by page entering manual corrections (and undoubtedly injecting new and creative errors of their own).
11. Marketing copy. The editor develops a pitch for the book that will motivate their marketing and sales force. At a company like Tor, internal meetings are held between editorial and sales several times a year, at which the editors present their new and forthcoming titles and explain their selling points, target audience, and special angles. The sales team then go forth and push the book at store buyers (who choose which titles to stock in retail store chains) and wholesale buyers (ditto for wholesale channels). The picture at other publishers may be different; for example Ace (part of Berkley publishing group, part of Penguin, turtles, recursion, etc.) have a smaller number of higher level marketers who liaise directly with the buyers for the big chains (Barnes and Noble, Borders). This stage is critically important, because the retail and wholesale buyers place their orders before the size of the print run is finalized.
12. Review page proofs. The typeset file is used to generate a PDF image of the book, as it will appear on paper. This is sent to editorial, who send a copy to the author and, hopefully, one or more proofreaders. At this point, the job of the author (and the proofreaders) is to correct errors introduced during the typesetting process, and possibly typos they didn't spot earlier — but not to make substantive changes. Again, around one month is allowed for reviewing page proofs and returning the marked-up chunk of dead trees (or an annotated PDF file, or a list of page/line number/error tuples) to editorial, who pass it back to the typesetters for fixing. (Parenthetical note: there is no such thing as a clean page proof. Authors are blind to their own persistent spelling idiosyncracies, typesetters get stuff wrong, and so on. But in general, the more eyeballs — and more proofreading passes — a set of proofs receive, the fewer errors will be left in the finished book.)
13. Collate advance orders and order the print run. The sales folks have spoken to the big chain buyers and wholesalers, who guess at how many books they can sell and place orders. If these are trade books they are sold typically on 90- or 120-days net credit, sale or return. (That is: if the bookseller orders a carton of 24 books, they must either return unsold stock or pay up the agreed wholesale price when invoiced after 90 or 120 days. So they're running on credit from the publisher.) If they are mass market books they are sold on net credit, but instead of being returned for credit, the wholesaler/retailer returns the stripped covers and pulps the book block. (Which means that MMPBs unsold after 90 or 120 days are treated like spoiled grocery stock. The mass market pipeline is absolutely insane, but it mirrors how magazines are sold; it was brought in during the 1930s/1940s to sell cheap paperbacks and arguably kept the publishing industry from collapsing, but today ... let's just say I haven't met anybody who's in favour of it. Note that although mass market paperbacks are all C-format in size, not all C-format paperbacks are "mass market" books; in the UK, the sale-or-strip channel died in the late 80s/early 90s, and today — as far as I know — all British C-format paperbacks are sold on sale-or-return terms.) On the basis of these advance orders, and experience which tells them how many extra copies to print for late orders/specialist stores, the publisher orders a print run from a print shop.
14. The print run: this is the moment of truth. (If the editor has coughed up a $100,000 advance against a 10% royalty on cover price, they need to ship $1M of books (cover price); if the advance orders add up to 2500 hardbacks, senior management are going to start asking difficult questions.) Some rough figures: a typical first SF/fantasy novel in hardback from a major US publisher will ship 3500-5000 copies. Anything over 10,000 is nudging towards the bestseller list; a fiction book that shifts 25,000 in its first month in hardcover is going to be in the New York Times top 20. Mass market paperback runs are much larger — 15,000-30,000 — but 25-50% will be pulped and the profit margin on them is vanishingly narrow compared to those juicy hardbacks. Oh, and the long tail applies: the top 3 on the New York Times bestseller list easily outsell the next 30 combined, and #4 to #30 combined probably outsell the next three million.
15. The printing process may include multiple stages: ordering of paper (different grades of paper have different properties and costs), then printing of book blocks (containing the pages of the books), trimming, printing of colour cover flats, folding and binding, or printing of signatures, stitching and trimming, binding, printing of dust jackets, and folding (depending on whether paperback or hardback, with variations depending on whether the hardbacks are saddle-stitched — as is still common in the US market — or perfect (glue) bound — as is the case for most British hardbacks). At least the old process of cutting photographic plates is obsolete; most commercial printers these days have machines that take in a properly imposed postscript or PDF image and act (from the user's point of view) like a gigantic Postscript printer (fed by barrels of ink and giant rolls of paper). It takes 2-4 weeks from ordering the print run to the shipping pallet of finished books to appear on the publisher's warehouse loading bay, depending on how busy the print shop and how complex the manufacturing process are.
16. Shipping. Someone in a warehouse has to ship out several hundred or thousand cartons of books to stores, and a smaller number of larger palletloads to wholesalers.
17. Invoicing and accounting. I'm going to tip-toe past this particular pit of festering insanity. Let's just say that if you're shipping to dozens of wholesalers, a handful of large bookstore chains, and literally hundreds of small bookshops, it gets hairy fast. (Multiply by 300 titles a year with an average in-print life of 5 years if you're someone like Tor.) Note also that the contracts the publisher signs with their authors dictate that the authors will be paid a royalty (percentage share of the proceeds) based on the number of books sold and the sales channel through which they are sold. The authors almost certainly have a contractual right to order a third-party audit if they think you're screwing with their figures. And if the figures are out, you (the publisher) pay for their audit, in addition to the money you owe then.

After I hand in an MS, I expect to do another 3-6 weeks' solid work on the book before it is published — mostly in the CEM-checking and page proof-checking stages. After I hand in the MS, I expect my publisher to put in ... well, Tor produce 300 books a year with 60 staff, so it's about ten person-weeks per book in house, but this doesn't include the external copy editor, proofreader, typesetter, printer, and other outsourced tasks, which probably double it again. Overall, the process of turning a manuscript into a book is estimated to cost $7000-$20,000 — an amount comparable to the author's likely earnings from the book. In fact, the actual division of labour on a book is split roughly 50/50 between the author and the publisher.

In summary, while it's true that the author is the one with the creative input, they only do about half the work. And the other half of the job is not optional. The reason publishers exist is to provide for division of labour; if I did the other 50% to bring my rough manuscripts up to published-book-quality, I'd only be able to write half as many novels.

I'm back home, I'm over the jet lag (for now), and I'm looking for something to write about.

It struck me, reading the comments on my various postings about the Amazon v. Macmillan spat in January, that many people don't have the first clue about how the publishing business works — or even what it is. Publishing is a recondite, bizarre, and downright strange industry which is utterly unlike anything a rational person would design to achieve the same purpose (which I will loosely define for now as "put authors books into the hands of readers while making a profit, to the satisfaction of all concerned"). So over the next few blog entries I'm going to make some notes about what's going on ...

Misconception #1: The publishing industry makes sense.

Most discussions of publishing take it as axiomatic that there is a thing called the publishing industry and that the entities within it look similar and work pretty much the same way. Nothing could be further from the truth.

As an author of commercial science fiction and fantasy novels, which is a highly restrictive category I mostly deal with a very specific type of publisher: a mass-market commercial fiction publisher — as opposed to, for example, a University press, a small press, or a vanity press. (NB: the word "press" is often used to mean "publisher", even in this day and age when almost all publishers have outsourced the inky job of running a printer to someone else.) Here's how the mass-market commercial fiction publishers are structured:

At the top of the food chain, there are the big publishing conglomerates; companies like Holtzbrinck, Hachette Livre (itself the publishing arm of Lagardère Group), or News Corporation. You (as a reader) do not buy books from these people, and I (as a writer) do not sell books to these people. Rather, we interact with the publishing groups that they own. They're colossal multinationals, and during the 1980s they went on a buying spree, acquiring smaller (often family-owned or private) publishing companies in a giant game of publishing pokemon. Now they operate as an umbrella, setting goals and corporate policy. If you've wondered where the idiot push for DRM on ebooks comes from, it's from the top down — from the board level of companies that own film studios, communication satellites, newspapers, and a huge chunk of EADS — from executives who know next to nothing about the business of publishing commercial fiction (because it's usually below their radar).

At the level below the multinationals, we see the individual publishing houses owned by the dinosaurs; outfits like Little, Brown (owned by Hachette) or Penguin Group (owned by Pearson PLC). These are still large multinational organizations, but at least they're in the business of publishing books — fiction and non-fiction. Policies originating at this level usually make sense, from the point of view of a publishing industry executive who has risen to boardroom level through the industry during the 1970s to 2000s.

At the level below the multinational publishing houses, we see the individual imprints. Imprints are the vestigial remains of formerly-independent publishing companies, often family-owned or private, which were eaten by the big publishing houses during the takeover wave of the 1980s. Today they maintain a twilight residual existence within the cubicle farms and office suites of the publishing houses, because they're brand names recognized by the reading public. Ace Books, whose logo appears on the spine of my SF novels in the USA, is such an entity. Founded by Aaron A. Wyn in 1952, Ace published until his death in 1967; it continued with financial troubles for another five years before being sold several times, ultimately ending up as part of Berkley Publishing Group. If you turn to the bibliographic information page in one of my Ace novels, you'll see their name; they're another, somewhat larger imprint, existing within Penguin Group. These days, "Ace" is a brand name BPG stamps on their SF novels; "Roc" is the corresponding brand name they stick on Fantasy; "Berkley" goes on non-genre fiction (such as William Gibson's latest novels), and so on. Policies originating at this level come from editors and marketers who interact directly with customers and suppliers (authors), and usually make sense (when not overriden from higher up the pyramid).

So: for my American SF titles, I deal with the editor in charge of the Ace imprint, within the Berkley publishing group (division) within Penguin Group, which is owned by Pearson PLC (who also run educational software companies and publish The Financial Times). It's turtles all the way down!

But this is not how the publishing industry is structured. I've been lying to you, by over-simplifying. The truth is a lot more complicated. What I've outlined above is merely the hierarchical structure of the major commercial trade publishing houses.

In tomorrow's thrilling installment, I'm going to outline the alternatives to the big media conglomerates. And once I've got that out of the way, I can start describing just what it is that I do to earn a living — which, you might be surprised to know, is nothing to do with writing.

Here's an interview the Tech4Thought folks recorded with me at Boskone a couple of weeks ago. (Also of interest: they interviewed Vernor Vinge at more or less the same time.)

In other news: The Hugo Award nominations close on March 13th. If you're eligible to nominate (i.e. if you were an attending member of last year's worldcon, or are a supporting or attending member of this year's worldcon), I encourage you to go and nominate; not enough folks do so.

In case you were wondering, this year I won't have a novel on the Hugo shortlist — not unless a miracle happens; I only had one novel out in 2009, and the penultimate fifth book in an ongoing series isn't a likely candidate for any award. However, I have three other eligible works; Overtime (a novelette, published by Tor.com), my novella "Palimpsest" (collected in "Wireless"), and ... this blog! Yes, this blog is eligible for a Hugo nomination in the "best semiprozine" category. (At least, I think it is, and a former Hugo administrator I asked agreed; turns out that averaging over 10,000 daily readers for months on end makes the cut.) Obviously — I'm not going to pretend otherwise — I'd be very pleased if any of these works ended up on a Hugo ballot; but it's not going to break my heart if they don't. I've had a good run on the ballot (nine years in a row in one category or another: six consecutive years in the "best novel" category). I'm more concerned that people should nominate, and vote, and continue to make the Hugo awards a product of public participation.

International air travel — as a user experience — sucks. I should know; I'm just back home in Scotland after a nearly two week trip to the USA, and I'm off to Japan in April. (My carbon footprint, let me show you: the domestic/automobile side of it is positively hair-shirt green in its miniscule dimensions, but the air travel part is bloated.)

Right now, I'm jet-lagged. Which is annoying, because I'm supposed to be writing a novel: but while I'm kinda-sorta back on local time, past experience suggests that if I try to work, my written output will resemble that of a fourteen year old with ADHD — anything I do in the next day or so would have to be junked or re-written so thoroughly that it would add more time to project completion than it would detract.

Jet lag is an insidious and unpleasant condition, aggravated by economy class travel and overnight flights: in my case, taking off around 10:40pm in New York, landing at 11:20am in Amsterdam, after six and a half hours or so in flight. The night is shorter by six hours than it should be, due to time zone differences. The cabin crew are scheduled to feed and water the cattle 30 minutes after take-off (allowing an hour for dinner), then 90 minutes before landing (with a light meal). In-between, the cabin lights are dimmed ... allowing for a grand total of four hours of darkness in which to sleep, if you can, in a not-too-wide seat. I lucked out on this trip (we were bumped from regular economy to KLM's economy-with-15cm-of-extra-legroom, normally a €60 extra) and was able to nap a couple of times; but it's no way to spend a night, and I've never in my entire life been able to stay awake for more than 28 hours at a stretch: I need my sleep.

West-to-east travel across time zones makes for the worst jet lag; you lose hours out of the middle of the night, and unless you can afford business class (I can't, usually) you don't get a seat you can sleep in. The end result is something like an 18 hour day followed by a 24 hour day, with some uncomfortable napping in hours 16-20. To minimize the damage and get onto local time at the destination as fast as possible, my preferred strategy is to catch a 2-3 hour afternoon nap on the day of arrival, then stay up until regular bedtime. Trouble is, after that nap I wake up feeling as if I've got most of the symptoms of flu (except for the fever and severe joint pains), and it lasts for days.

To add to the fun, my local airport doesn't serve many destinations outside the EU. I therefore need to connect via a major hub. In this case, back in November 2009 when I booked the flights (before Mr Explodeypants screwed the experience for everyone) I booked via Schiphol, Amsterdam's airport. Schiphol is actually east of my destination on the west-to-east flight; but I had a reason. Firstly, Schiphol is not Heathrow (or Gatwick) — travel via the British intercontinental hubs is a hideous experience, and I'll go to some lengths to avoid them. In terms of flight time, Amsterdam's about half an hour further from the US than London, and half an hour further from Edinburgh. But in the departure area there is a hotel which you can book for a four-hour slot, for about the price of access to a business class lounge. This gets you a compact but nicely designed room with an en suite shower/toilet bathroom and a really nice mattress. I planned for a four-hour connection, precisely so that I could catch a nap after the overnight flight: it worked well enough that I'd do it again.

A secondary aspect of the experience is dehydration, of course. Like many forty-something males I take medication for high blood pressure; consequently, I drink like a fish. Air-conditioned cabins pressurized to the equivalent of a 4000 metre mountaintop are very dessicating: at the end of a long flight my skin feels like leather and I have a mild headache, and that's even after managing to grab a litre bottle of water to supplement the ration that comes with dinner. Alcohol is pretty much right out, unless you can grab an extra water supply: it's a mild diuretic, your tolerance at altitude is reduced, and if you indulge to even a normally tolerable extent you'll end up with the hangover from hell.

Perhaps the worst part of the experience of flying these days is the security theatricals. No surprise in the attentiveness of the screeners at Schiphol being dialed right up to eleven; but I was somewhat shocked by the poor quality of the security at JFK.

Here's the rub: security is a state of mind, not a procedure. Procedures can't cope with attackers, because they're inflexible. If you search passengers for guns, someone will carry a knife. If you search for knives, someone will sew themselves a set of underwear full of PETN. And so on. To deal with a threat — say, someone who wants to attack your air travel infrastructure — you must look for the attacker, not their tools, because they can change their tools at will to exploit weaknesses in your procedure for identifying tools.

JFK is wide open to terrorists intent on causing mass casualties.

Why?

It took me about 60 minutes to shuffle through the scrum for security clearing, around 8pm on a Thursday. This isn't an accident of timing: at that time, we had about four or five 747-loads of passengers trying to get to their gates. Among them was the last El Al flight of the week to Tel Aviv that can arrive before sunset on Friday (i.e. the sabbath); guess what ethnic profiling of passengers for that flight would suggest? Also among them: the last 747s for Europe from New York on a regular midweek day. All these flights were crammed to the gills; so we had a couple of thousand passengers milling around in a zig-zag maze of rope barriers, waiting to squeeze themselves through the metal detectors and their bags through the X-ray machines. Keeping an eye on this mass, from outside: about two cops and two soldiers. At the gate, security was cursory at best: the only passenger I saw being hand-searched was the wheelchair user (who obviously couldn't go through the gate), and I saw no sign of bags being re-checked or hand-searched either: the TSA staff were clearly under immense pressure due to the workload, and were therefore going through the motions — performing their checklist at maximum speed (look for guns, look for knives, look for things that look like TSA-standard training bombs), rather than being thorough.

Let me wear my Osama bin Laden hat for a minute:

Suppose I wanted to attack the US air travel infrastructure. I can't get bombs or box cutters onto planes reliably. But I can kill lots of passengers! All I need to do is to buy a maximum-size carry on bag (US dimensions: 7" x 13" x 20") and build the biggest, heaviest bomb into it that I can wheel behind me. It's not weight-constrained for hand luggage: there's probably room for about 10 kilos of PETN (or similar) and 5 kilos of metal shrapnel in such a bag.

All I would have to do then is buy a ticket (return, please: one-way with no checked baggage is now flagged as something to watch) and go queue. Then, when I get to the middle of the crowd, detonate the device. (For added horrors: have an accomplice with a similar device hang back, to detonate their bomb amidst the fleeing survivors.)

My point here is that security checkpoints are a target, too, because they slow down travellers and cause crowds to form, and another term for "crowd" is "convenient target". And because the attacker has not been separated from their weapon at the point when they reach such a target, it's the logical weak point for causing maximum damage.

Schiphol — Amsterdam airport — gets the security screening right, or at least less wrong than JFK and most other airports. Rather than having a hideous bottleneck between check-in and the departure area, security screening is carried out at each depature gate, with a separate metal detector and X-ray belt; no huge crowds form in unsecured areas. On US-bound flights, someone who clearly isn't a minimum-wage drone checks ID documents and asks a couple of questions that seem to me to the aimed at flushing out anyone who is disturbed or tense — a crude form of profiling. This was in place before December 26th, 2009; Schiphol has much tighter security than many European or American airports. (Which is another, albeit minor, reason why I prefer to use it as my hub for long-haul travel. There's just one black mark against it, namely the lack of soft drink/water vending machines — or even toilets &mash; in the gate waiting areas after security: you can't buy extra fluids to take aboard your flight.)

Anyway. Never mind the jet lag and other minor inconveniences: I resent the way the inconvenient, intrusive, and annoying security procedures currently being forced upon us actually make air travel less safe. And I resent the way that the political syllogism — something must be done; this is something; therefore this must be done — is used to justify nonsense like expensive and buggy teraherz radar booths as a panacea in place of plain old-fashioned intelligence-led policing.

(And now, if you'll excuse me, it's time for my afternoon post-jetlag nap.)

As you probably guessed from the silence, I'm still away from home: in New York right now, flying back across the North Undrinkable tonight, and probably jet-lagged for the next two or three days. You wouldn't want to see me blogging while jet-lagged: it's kind of like blogging while drunk, only not as (unintentionally) amusing.

(I'm in New York for meetings with my agent and editors, plus general purpose sightseeing and maybe just a smidgeon of research for future fiction.)

Normal service will be resumed eventually: meanwhile, I'm going to try to line up more guest bloggers for next time I'm away for a protracted time (meaning, April: I'm a guest of honour at HAL-con in Omiya, Japan, and I'm not flying all that way just for a weekend).

Hi: this is Charlie checking in again. I'm in a hotel in Boston, winding down from Boskone. Among other things I participated in a number of panel discussions, including this one: a reappraisal of the Singularity in SF. The moderator was Alastair Reynolds; the other panelists were Vernor Vinge and Karl Schroeder.

My attempt at using a voice recorder was an abject failure — I manged to leave it on pause the whole time — but all is not lost, thanks to Mike Johnson, who video'd the whole thing and put it on Vimeo. In case this sort of thing amuses you, here is a bunch of SF authors who ought to know better, kicking the tyres, twenty years on:

The Singularity: An Appraisal from Michael Johnson on Vimeo.

"Hexapodia is the key insight." (You've probably seen that video already, but I couldn't resist sharing. Just in case.)

In about 48 hours I'm off to Boston and New York for a week and a half. Needless to say I won't be blogging much (although I'm hoping to record and podcat the panel I'm on at Boskone — a retrospective appraisal of the Singularity in SF with Al Reynolds and Vernor Vinge).

But surprise! The blog won't be left to the tumbleweed.

Because I travel too damn much, I've decided to invite guest bloggers to take over Charlie's Diary while I'm away. (I've also recruited a couple of moderators to ride herd on the discussions in my absence.)

First up is the — what's the right adjective here? Fantastic? Feisty? Fragrant? Formidable? — Elizabeth Bear, who has won twice as many Hugo awards for her SF as me. If you haven't heard of her, start here. (Better still, buy her books.) She's got my soapbox for the next week. I'm looking forward to seeing what she does with it!

In my most recent posting on this topic I noted "Amazon surrender", and cited a New York Times article as saying that Amazon had agreed to re-list the Macmillan titles they'd dropped.

As of this morning, five days later, my own Tor books are still not available from Amazon. I'm hearing lots of reports from other Tor authors, too.

Amazon lied.

They lied about other things, too; in their press release they lied like a rug about Macmillan's negotiating position, mischaracterising it in the worst possible light from the point of view of onlookers. They lied by falsely positioning themselves as the defenders of cheap $9.99 ebooks and Macmillan as some kind of capitalist oppressor; the truth is that many ebooks sold via Kindle cost well over $9.99, while Macmillan were proposing to sell some titles for under $6.

Amazon lie by omission. They lie like politicians in an election campaign. And you've got to ask, if they're selling such a good product, and if they're such good-hearted folks, why do they need to lie?

I'd like to note in passing that Amazon has a long history of bullying — from union-busting among its employees, abusive treatment employees (arguably in violation of health and safety standards in the UK), to punishing authors by de-listing their books as a way of applying negotiating pressure to publishers; they did this to Hachette in May 2008, and the people who got hurt the worst were the authors and their readers. In fact, Amazon are the WalMart of the book trade.

Meanwhile, things have been moving rapidly this week:

Hachette are switching to the agency model, too — which means Amazon will be fighting a war on two fronts next week. (Are they going to de-list close to a third of all the books they sell before this is over?)

HarperCollins are also queueing up to renegotiate terms with Amazon. That's the publishing arm of NewsCorp, who I am not terribly keen on — the third of the big six publishing groups. (What does it say about a retailer that half its suppliers are really unhappy with the terms it has imposed on them separately?)

Macmillan's CEO had this to say about Amazon, in a public letter to his authors and their agents: "A word about Amazon. This has been a very difficult time. Many of you are wondering what has taken so long for Amazon and Macmillan to reach a conclusion. I want to assure you that Amazon has been working very, very hard and always in good faith to find a way forward with us. Though we do not always agree, I remain full of admiration and respect for them. Both of us look forward to being back in business as usual." (I don't think I'd have been that polite.)

There's a lot more stuff going on behind the scenes. In particular, it looks like the industry-wide shift to the agency model, catalysed by Apple, has finally cracked open the door to a renegotiation of royalty rates payable by publishers to authors for ebooks. The traditional 10% royalty rate on hardcovers didn't come out of nowhere — it reflected a 50/50 split in the profits (the other 80% of the cover price going on production, printing, and distribution). With much lower printing and distribution costs, it looks like royalty rates of 25% or 30%, on a much lower overall price, may be where things are going under the new model. (Hint: this isn't finalized yet, which is why Macmillan aren't promising readers the moon on a stick — unlike Amazon.)

Finally, let's look at the authors, because we're the small mammals who get steamrollered when the dinosaurs start stomping on each other.

Novelist Cat Valente explains lays out why she sides with her publisher: "the costs of publishing an ebook are not zero. That is, if you have any interest at all in a quality product. No one goes around suggesting that everyone should become their own autonomous cheesemakers and cheering the death of the cheese industry. Why? Because that would result in a lot of shitty cheese."

John Scalzi explains what's been going on this past week. He also explains why publishing will not go away anytime soon in most amusing fashion.

Susan Pivar, former music label exec (and author) compares and contrasts the Amazon/Macmillan dust-up to how the music industry (mis)handled things.

And finally, I commend to you this blog posting by a guy whose first novel came out from Tor the very week Amazon decided to delist all Tor's books. Talk about depressing ways to start (and quite possibly finish) a career.

The rumble is on-going, but on Monday I'm flying out to Boston and New York for about ten days. This means that your questions here are likely to go unanswered and I won't be posting for a week or so ... but I have a surprise waiting for you on Monday!

The phrase "Information wants to be free" gets 92,000 hits on Google.

As wikipedia notes, Stewart Brand (then editor of Whole Earth Review, and a clueful chappie) uttered the fateful words at the first hacker's conference in 1984:

On the one hand information wants to be expensive, because it's so valuable. The right information in the right place just changes your life. On the other hand, information wants to be free, because the cost of getting it out is getting lower and lower all the time. So you have these two fighting against each other.

He subsequently reformulated it slightly:

Information Wants To Be Free. Information also wants to be expensive ... That tension will not go away.

The condensed version — and it's almost always quoted in the condensed form, much as those who quote "my country right or wrong" almost invariably miss out the second clause ("when right, to be kept right; when wrong, to be put right") — is an attractive slogan because it's simple, but ambiguous. It's worth noting that English, the language in which "information wants to be free" was coined, makes no distinction between two usages of the word free: free as in "civil liberties", and free as in "no payment expected". "Information" is also ambiguous; Brand's explanation of the coinage uses the word "information" to mean both content and bandwidth, but this nuance is lost in the shortened form.

Richard Stallman also reformulated it in a way that puts a different spin on it:

I believe that all generally useful information should be free. By 'free' I am not referring to price, but rather to the freedom to copy the information and to adapt it to one's own uses... When information is generally useful, redistributing it makes humanity wealthier no matter who is distributing and no matter who is receiving.

Stallman's reformulation is transformed into a statement of political aspiration by the substitution of "should" for "wants", and it's an aspiration that I can't object to on moral grounds (although in practical terms, I see obstacles).

Here's the rub, though: it's a slogan. Stripped of their social context, slogans are a great way of motivating people — but they don't put food in your belly. More to the point, slogans with ambiguous wording or implicit subtexts are lethally sharp double-edged swords. "Let a hundred flowers bloom" had [probably] unplanned consequences; so, too, does "information wants to be free". It has, as Jaron Lanier notes in this interview, become the governing ideology of a large and vocal segment of internet users:

He blames the Web's tradition of "drive-by anonymity" for fostering vicious pack behavior on blogs, forums and social networks. He acknowledges the examples of generous collaboration, like Wikipedia, but argues that the mantras of "open culture" and "information wants to be free" have produced a destructive new social contract.

I'd put it another way: it's the tragedy of the commons in action.

There's a big difference between a gift economy (where items are given freely, as gifts) and a theft economy (where items are taken without offer of recompense, be it monetary or participatory). While "information wants to be free" remains a valuable insight, the freetards who are its loudest proponents these days seem blind to the flip side of the coin, which is the obligation to create and release information of use to others.

I'm big on obligations, because it seems to me that they're the flip side of rights. The right to not be murdered in my bed imposes on me the obligation not to murder other people in their beds. Human beings are social animals; we do not exist in isolation, and if we desire some specific behaviour from our peers, we, too, are required to abide by it. The alternative is tyranny, a state in which some individuals are exempt from ordinary rules and may exercise their liberty at the expense of others.

So it follows that if you want information to be free you are taking on an obligation to make information, and give it freedom. An obligation to work to better the lot of humanity, not to merely sponge off the labour of others.

Next time you hear someone invoke "information wants to be free" as a justification for demanding free-as-in-no-payment-expected content, ask them: precisely what content have you released for free lately?

Note: buying an ebook, stripping the DRM from it, and uploading it to usenet does not count (unless they're the author). The point is recipricocity in creation.

Finally: this posting was prompted by the fact that just before the Amazon/Macmillan custard pie fight broke out, my editor and I at [name of publisher withheld] were discussing the possibility of releasing another of my novels as a free download. I'm now wondering — in view of the huge number of Kindle cheapskates yelling that $14 is too much to pay for a bestseller on the day of publication, it must be $9.99 or less, even if the publisher can't make a profit at that level and subsequently publishes fewer books — whether this would merely encourage the perception that my books are valueless. I'll probably get over my dog-in-the-manger mood soon enough — I tend to take an optimistic view of human nature — but right now I'm a bit annoyed. (PS: yes, I use open source software, and yes: I have in the past written and released software under an open source license — it's on CPAN if you want to look for it. See "obligation", above.)

What have you created and released lately?

The New York Times reports that Amazon has conceded to Macmillan:

"We have expressed our strong disagreement and the seriousness of our disagreement by temporarily ceasing the sale of all Macmillan titles," Amazon said. "We want you to know that ultimately, however, we will have to capitulate and accept Macmillan's terms because Macmillan has a monopoly over their own titles, and we will want to offer them to you even at prices we believe are needlessly high for e-books".

Which is highly interesting: as Bruce Schneier puts it, "in a supply chain, profit — and power — tends to flow to the most constrained member of that chain." In an airport food court, most of Starbucks' or MacDonald's profits flow towards the airport, for real estate in that situation is constrained; in ebook publishing on the Kindle platform, profits flow towards the platform owner (i.e. Amazon).

I'd rate this as a temporary setback for Amazon; in the war to define the internet book distribution chain, Macmillan have merely clawed back some of the territory they've lost over the past five years and bought themselves some time in which to try and get their ebook sales up and running on a profitable basis. In the long run, though, if they don't make it work within a couple of years, expect to see more battles (and possibly a new CEO: there are many firing crimes in publishing, but pissing off a major distribution channel in order to win a concession and failing to exploit it successfully has got to be one of them).

Longer term the publishers badly need to reconsider the entire idea of selling ebooks wholesale or on an agency basis via internediaries. This goes against everything they've ever done before — but the correct model for selling ebooks (profitably and at a fair price) is to establish a direct-to-public retail channel, like Baen's Webscription subsidiary. Oh, and once you're there, you can ditch the annoying DRM.