Publishing's big bang

[ Site Index] [ Linux Index] [ Feedback ]

Publishing's Big Bang

Book publishing used to be a staid, conservative business. Not any more; the internet is changing everything and seeding a new industry.

If you're reading this article you almost certainly belong to a self- selected group: you're interested in, and probably work with, information technology. We wake up every morning and going to work with computers, software, and the internet, and it's part of the job. Change is good and new technology makes everything run smoothly. In particular, the internet is a new kind of glue for the economy, making business run faster and more efficiently and opening up new opportunities.

At least, that's the conventional wisdom. In the staid world of book publishing the explosive growth of the internet has come as a rude shock. New technologies like Print on Demand, coupled with direct distribution via dot-coms like Amazon promise to destroy the traditional relationship between publisher, distributor, and bookshop; meanwhile, e-books (books you can download and read on a laptop or Palm-sized device) completely upset the economics of producing books. Then there are the warez d00dz, scanning copies of their favourite books and posting them on the net -- a growing threat to the economic basis of writing as a profession.

Everything is up in the air at present, as authors, publishers and distributors try to figure out whether they're still going to have a job in five years' time. Nobody's sure, yet ...

What the literate Warez D00dz are scanning today

In the past couple of years, usenet (the huge distributed conferencing system that runs on the internet) has exploded in volume. These days, 95% of the traffic on usenet has nothing to do with discussions of software, needlepoint, or whatever people talk about on a bulletin board: it consists largely of messages with software attachments. Download and open some of these attachments and you'll find they contain copies of commercial software -- often with registration code cracked. The software industry as a whole is used to this; it's a minor nuisance compared to the commercial forgers who produce and sell duplicates of expensive products in bulk.

A year or so ago a couple of new binaries-only newsgroups showed up: with names like "alt.binaries.warez.e-texts" they didn't fit the traditional mold. Instead of software, people were anonymously posting books via the newsgroups. Some of these were legal; Project Gutenburg has a long and honourable history of taking out of copyright works, scanning or transcribing them, and making classics available for free via the internet. But a lot of the books showing up on the e-text warez newsgroups were a long way from being out of copyright.

Copyright on books is established internationally by the Berne Convention; it runs for the duration of the author's life and for seventy years after their death before the work enters the public domain. If copyright is owned by a company, it lasts for the lifetime of the company, and then another seventy years. For the duration of copyright control, the author (or whoever they assign their rights to) has in theory got authority over who does or doesn't publish the work -- an action which is loosely defined as making copies of the book available publicly. Posting copies of it on usenet is a pretty clear violation of copyright, and the likes of Tom Clancy and Stephen King are still alive. So what was going on?

Like so many other head-scratching problems associated with the internet, the e-text piracy groups arose as a result of a collision between technology and consumer demand. On the one hand, the price of optical scanners crashed by an order of magnitude during the 1990's (and cheap, effective optical character recognition software became available, allowing scanned bitmaps to be converted into text files semi-automatically). This was combined with the advent of cheap pocket computers with sufficiently good screens to make reading an entire book possible. On the other hand, the publishing industry had lost track of the ball.

What publishers do

Every year, roughly 40,000 books are published in the UK. Book publishing is an odd business, very unlike the magazine business. This magazine in your hands is a group effort. Editors commission articles from journalists, who supply the articles to a deadline. The commissioning editor then squeezes the supplied copy so that it fits the layout and page plan for the issue. Each issue is assembled on a production line, so that after the sub-editors have run the editorial content into the issue the adverts (acquired by a separate advertising department) can be added. The finished magazine (in the form of a monstrous DTP file with associated images and fonts) is typeset and printed. Several tens of thousands of copies of the magazine are then sent to a distribution company (the largest in England is W. H. Smiths' distribution arm -- in Scotland it's John Menzies) and shipped to retail outlets in time to go on sale.

Books don't work that way. A book is usually written by a single author, or consists of a set of chapters commissioned by a single editor. Books are the ultimate brand-name product; when a customer goes into a bookshop and asks for the latest novel by Stephen King, they won't be satisfied with another random novel published by the same company. Rather than being assembled on a production-line basis, books show up as and when the author is able to deliver them.

Within the book publishing business there are a number of different types of publishing mechanism. In fiction, it's a buyer's market: everyone seems to have a novel in their desk drawer, and publishers can pick and choose. What a publisher wants is a solid list -- a stable of authors, each producing a book every six months to three years, each of whom can build up a consistent following of readers. When a book arrives the editor may suggest some changes to the author, then slot it into their publishing schedule -- which prints and distributes books at regular intervals, dependent on the publisher's capacity. (Note that this is the picture for established authors. For first-timers it's a lot harder; between 99.5% and 99.8% of unsolicited manuscripts are rejected by publishers.)

In non-fiction, things work a bit more like the magazine world. Authors and publishers agree on the outline content of the book before it is commissioned -- then once the manuscript is completed (and hopefully passed by the reviewers) it gets slotted into their publishing schedule.

The important thing to understand here is that after the book is published, a couple of pallet-loads end up in a warehouse somewhere. Boxes periodically get removed and sent to those same mass-market distributors, who supply stock to bookshops. When the pallet-load is exchausted, the book is out of print; unless the publisher feels like printing more copies, there are no more to be had. And if the pallet sits around too long, or the copies the distributor sent out don't sell fast enough, the left-overs will be returned to the publisher -- "remaindered". This became a particular problem in the 1990's in the USA, where a decision by a court in New York State determined that tax was payable on stock stored in a warehouse for more than one financial year: it meant that books could no longer be warehoused cheaply, and for this reason the whole publishing lifetime of a book might be compressed into a matter of months rather than years.

This is the way the publishing cycle used to work. if you wanted an out of print book, or one that had been remaindered, you were out of luck. If you wanted a book by an obscure author you were out of luck, too, unless you could find a bookshop that still had a copy sitting on their shelves, unreturned. And if you wanted a copy you could search electronically, or read on a palm-sized device that would fit handily in a pocket, you could go whistle.

Amazon and the web revolution

The first crack in the distribution chain can be attributed to one man: Jeff Bezos. In June 1995, Bezos created a new company, specifically to make an end-run around the distribution bottleneck. In March 1996, their web software and distribution network was ready to roll, and as Bezos says, "anybody who would've predicted what has happened today would have had to have been put in a padded room with a strait jacket immediately." That's because in just two years Amazon's revenue grew from $825,000 to $600M.

Bezos sat down and asked himself what the best product to sell online was. He says: "I made a list of 20 different products and force-ranked them according to several different criteria. I was looking for something that you could only do online, something that couldn't be replicated in the physical world."

In the end he picked books. "Books are incredibly unusual in one respect, and that is that there are more items in the book category than there are items in any other category by far. There are more than 3 million different titles available and active in print worldwide. ... There are lots of categories where selection is proven to be important: books, in particular." isn't a bookstore; it's a database. With more than 4.2 million titles in their inventory by early 1999, there was no way that Amazon could actually have warehouses with that inventory in stock. Instead, Amazon focussed their business on getting access to the inventory databases of publishers, manufacturers, and other distributors. When you search for a title on Amazon's web site, you're actually searching across the stock databases of every publisher who Amazon has an agreement with. When you order a book, Amazon either despatch the order from their own warehouse (which stocks the most popular items) or orders it from the publisher -- it then gets routed through their system and is bundled with any other items you've requested.

Bezos explains: "online, you can have this vast catalog of millions of titles, whereas in the physical world, the largest physical superstores are only about 175,000 titles, and there are only three that big."

This business model has two huge advantages over traditional retailing. For starters, leaving stock sitting in a warehouse is inefficient; it represents money that isn't flowing through the business. Dell grew to their pre-eminent position in the PC market by recognizing this and shifting to "just in time" manufacturing: the components for your new Dell PC aren't purchased until you confirm the order for the PC. Amazon works the same way, applying just-in-time methods to selling books. Secondly, by not being limited to the stock in their warehouse, Amazon can afford to advertise and sell a much wider range of goods than any conventional business. Just trying to imagine a warehouse with 4.2 million different types of item in it beggars the imagination: the management and stocktaking overheads of such an operation would be prohibitively expensive, but Amazon doesn't need to operate it.

The point to note here is that Amazon is a threat to two different types of business -- to the small, specialist bookshop (which cannot compete on range of products), and to the distribution chains (because Amazon can go direct to the publishers, by-passing the distributors who normally parcel books out to bookshops). And the thing that makes its model possible is the wild profusion of products -- the fact that there are three million different titles in print, and more being published all the time.

Soft copy

Of course, e-commerce websites aren't the only threat to the traditional bookshop. When you buy a book, only 10% or so of the cover price goes to the author, and even the publisher only picks up 15% or thereabouts; the rest of the cover price is split between the distributor, the bookstore, and production costs. The nuts and bolts of book production didn't change much between 1880 and 1980 -- but now it's shifting rapidly as two different technologies become available: e-books and print-on-demand.

An e-book is a book delivered as software. Some early experiments notwithstanding, the first success of e-book marketing only came with the widespread adoption of the CDROM; capacious enough to include graphics and animations, CDROMs were a practical medium and found a niche as a lightweight replacement for the multi-volume encyclopaedia. But CDROMs still cost money to print: with discs costing 20p or so in bulk, and packaging costs on the same order, they're not that much cheaper than paper books (where 15-20% of the actual price reflects the expense of paper, ink, and printing overheads).

Early e-book schemes foundered on the need for a cheap, reliable reader machine: but with the explosive growth of the PDA market since the introduction of Palm's Pilot in 1996, this has stopped being an issue. A Palm Pilot with 8Mb of RAM has enough memory to store a dozen novels, and its battery life is more than adequate: if you already own one and commute to work by train -- or just can't be bothered carrying a large lump of dead tree around with you -- you're a potential market for the likes of Peanut Press or Baen Books.

Peanut Press is structured as a conventional publishing house. They buy republication rights for existing paper books from their primary publisher, and re-sell the books online; you pay by credit card and can then download one of their encrypted e-books into you Palm Pilot (or Windows CE palmtop). The encryption key is the credit card number you used to purchase the book -- it needs to be re-entered whenever you open the book on a new palmtop, as a deterrent against copying -- and the price is about the same as the current paper edition of the book.

In contrast, Baen Books of New York are taking a more adventurous approach to e-books. Baen are one of the larger publishers of science fiction in the US, releasing four or five titles per month. Because of their tight genre focus and emphasis on a house style, Baen have staked out a particular territory within their market (at the more action-oriented end of the genre). Rather than seeing e-books as something to be licensed out to an external publisher, they view their e-books as content and have established something more closely related to a magazine: the Baen WebScription. Rather than buying an individual title online, and downloading an encrypted version of it into your PDA, Baen publish an entire month's list as a single package -- for just ten dollars, rather than the price of the paper copies. (Which, as they do hardbacks, can be rather steep.) Essentially the webscription system is a monthly fiction magazine in which the content consists of novels from the publisher's list.

Moreover, Baen's e-books aren't encrypted: taking an optimistic view of their readership, Baen work on the assumption that customer loyalty will keep unauthorised copies off the net: so far they seem to be right. Although software to crack Peanut Press e-books has been released, Baen's unencrypted e-books seem not to have been targeted by the warez community -- the theory is that they're sufficiently cheap for there to be no incentive for piracy, even though their books are released in unencrypted HTML and RTF versions.

The economics of e-book publication are very different from those of paper publication. Typesetting and editorial costs are similar, and book marketing costs the same in any medium, but there's no wood pulp to print, bind, stack on pallets, ship to a warehouse, store, and distribute. Instead there's a web server to maintain (with a gateway to a payment service provider), and bandwidth for downloads. Here, the superiority of e-books to paper from a publisher's point of view becomes glaringly clear: the bandwidth to ship a one megabyte e-book (assuming you're buying an E1 line at UK prices) costs roughly ten pence, while in the US it's nearer to two pence. In contrast, the cost of paper and ink and printing is nearer to a pound for a paperback, two to three pounds for a hardback. A webscription or Peanut Press e-book can deliver the same net profit to the publisher and author at a fifth the price of a paper copy -- if it can find the market.

Which is the hard bit. Because only a small minority of readers have PDAs yet, so only a small minority can buy and read e-books -- even if they wanted to. So far, the e-book market is in its early stages.

The six hundred pound software industry gorilla has only just begun to sniff at this new market. In late 2000, Microsoft released Microsoft Book Reader, an e-book reader package that included some nifty anti-aliasing (to make text easier to read on screen) and a copy protection/encryption mechanism. "We're not going to compete with anyone in the content creation and distribution business," says Dick Brass, vice president for technology development. Microsoft has set up strategic alliances with Barnes and Noble (a large American bookstore chain) and Simon & Schuster (a large publisher). "There are guys who are in the business of selling content. The format is a plumbing issue. We happen to be really good plumbers." But Microsoft still hasn't released a copy of Reader for it's own Windows CE platform: and in the e-book world, cross-platform readability is important. A paper book can be read anywhere, any time, and readers don't seem eager to jump aboard e-book systems that they can only read on one particular machine.

It comes as no surprise to learn that Baen's webscription service, while not losing money, isn't hugely profitable -- and companies like Peanut Press, Fatbrain, iBooks, and their relatives haven't followed Amazon's meteoritic stock market example. If the much-heralded convergence between cellular telephones and PDAs occurs, bringing better screens and more memory to the phones, this may change rapidly: but for the moment, e-books haven't set the market on fire. Yet. The early indicators are that the likes of Stephen King aren't about to leap into the water; after setting nerves on edge throughout the publishing industry by putting out the first chapters of his book "The Plant" on the internet, King retreated, saying that it wasn't selling well enough to continue with -- despite 40,000 downloads in the first two weeks.

Print on demand

There's one new technology that really is rattling a few cages: and that's Print on Demand.

In the old days of publishing, typesetting a book entailed making lead plates containing an image of the page: ink was transferred from these to the paper directly. This was superseded by the offset litho press, in which a film image of the page was the final stage in pre-press production: but film images are bulky and expensive to make and store. Setting up a press was expensive and time-consuming, so books were printed in batches of several thousand at a time. With the development of Postscript, printers acquired a much cheaper storable format for books: and in the 1990's heavy duty printers arrived that could take a postscript image in at one end and spit out books at the other, at the rate of one or two copies per minute. With bindings printed separately, it's possible for a print-on-demand publisher to produce and warehouse very small numbers of books, re-printing individual copies whenever one is sold.

Print-on-demand technology attacks the warehousing link in the publication chain. Instead of that pallet-load of books, gradually gathering dust as it sits in a warehouse for years (and eating up the rent), all the publisher needs to store is a set of postscript images -- and fifty to a hundred books will fit on a CDROM.

A number of publishing people spotted the logical convergence between print-on-demand and the new distribution media provided by companies like Amazon back in 1997-98. John Betancourt, an SF book packaging specialist from New York, set up Wildside Press that year: his goal was to buy up the rights to out of print books, or books by dead authors, as fast as possible. A book goes out of print when it's no longer in the warehouse: but by arranging to have books scanned or re-typeset to disk, Betancourt was able to put hundreds of books back into print and make them available via Amazon and Ingrams (the largest US book distributor). The cost was minimal -- with no investment in warehousing, printing plant, or editing, Wildside Press was rapidly able to build up an inventory of hundreds of out of print works.

Because there's a huge range of books on the market, most of which sell slowly, it's hard to predict where a bestseller will come from. Traditionally, publishers took two approaches: follow the established bestselling authors, and keep a huge "mid-list" of promising newcomers in print in the hope that one of them will take off. Wildside branched out, spawning new imprints such as Cosmos Books. Editor Sean Wallace writes: "Cosmos is centred on promoting British and Australian science fiction and fantasy, whether in the form of original novels, reprints, or anthologies. Traditional publishing in the United Kingdom has failed British authors by not utilising the local talent to its fullest. We are striving to correct that imbalance by concentrating on working with British authors in order to fully maximise their marketability."

The first stage in this process is to get books into print: and without the internet this would be much harder. Wallace explains: "Access to the internet does make the system move faster - it cuts down on the production time by an enormous factor, and it usually takes only four to six weeks for a book to appear in the distribution system."

The printing side is automated, too. "Our printer is LightningSource, which uses proprietary printing equipment designed by IBM. Books are printed and bound on-site within 48 hours of an order being received. Since LightningPrint is owned by Ingram Books, the largest distributor in the world, the LightningSource system has been fully integrated into Ingram's distribution system."

The economics of print on demand are very similar to e-books: "there is little actual investment in physical inventory. Every time one is ordered, one is generated. In terms of viability, print-on-demand books are potentially hugely successful. Wildside's best-selling titles are selling hundreds of copies per year. We expect this number to continue to increase as more readers become aware of us. We also do a lot of direct marketing -- we sent out 60,000 catalogs in 2000."

Best sellers, selling hundreds of copies per year? This doesn't sound like a traditional best seller! But these books are ones that have been out of print for years -- with sales of hundreds per year it's not worth cranking up a traditional press, but with print on demand these books can stay available indefinitely.

And the sting in the tail of this situation is that bestsellers are unpredictable. The Lord of the Rings was in print for twelve years, selling less than two thousand copies, before it suddenly caught fire and became a runaway success. The economics of traditional publishing (with the overheads of warehousing and distribution) make this sort of phenomenon rare; but the print on demand business works in the other direction -- if one of the Cosmos Books titles suddenly takes off it will be evidence that Wallace and Betancourt are right, and the traditional publishing business model is obsolete.

The future

The future for book publishing is uncertain. The conventional distribution chain is disintegrating, under the corrosive impact of the internet: retail sales, warehousing, and distribution are all under attack by technologies that promise to put the authors in direct contact with their readership. While this is bad news for publishers, it's potentially good news for readers -- no more out of print books, e-books costing less than paper ones, and anything that's ever been published available at the click of a button on a website.

The only thing that isn't sure is who, apart from Jeff Bezos, is going to make money out of the new publishing business. And who's going to be looking for a new job ...

[ Site Index] [ Linux Index] [ Feedback ]