Back to: Things publishers can't do (yet) | Forward to: The permanent revolution

Why I don't self-publish

In the previous thread, one of the commenters noted: Of course, OGH has previously estimated that disaggregating the publisher's job would land him with 0.5-equivalent of management work, leaving us (and him) with only 0.5-equivalent of an author. That doesn't mean it can't be done, but there'd have to be a pretty good reason...

Let me expand on that, in case anyone isn't convinced.

Left to my own devices, in a good year with no major disruptions (which, alas, don't come along as often as I'd like) I can write around 200-240,000 words of finished fiction — a pair of 330 page novels or one big doorstep plus a novella. This assumes I'm working on lightweight novels that flow easily, or that my drill sergeant muse is standing on my shoulder shouting "gimme chapters, worm!" in my ear through a megaphone. (A tough one can flow at half the speed — "Rule 34" took 18 months to write, for example.)

The specialist SF trade fiction publishers I know have a production ratio of roughly 6 novels/year for direct employed members of staff. That is: Baen (10 folks) produce 60-odd novels, Tor (50 folks) produce 300-odd books.

However a modern trade-fic publisher is an organization dedicated to handling the work-flow of book production. Over the past 30 years they've ruthlessly outsourced everything that isn't a core part of the job of publishing — including many tasks that an outsider might think were core competencies. Copy editors work freelance, paid by the book. Proofreaders ditto. Typesetting is carried out by DTP agencies. Printing is the job of a printer, not a publisher.

The stuff that remains in-house is editorial, marketing, accounting, and (occasionally) sales. "Editorial" in this context means workflow management — someone to ride herd on the pool of copy editors and proofreaders and to make acquisition decisions (in their spare time). "Marketing" includes book design, blurb writing, ARCs/review copies, presence at trade shows, glad-handing the big chain buyers, commissioning advertising, organizing signing tours and author promotion, and so on. (There's also a "production" side, sometimes subsumed under editorial, whose job it is to organize typesetting and printing and the business of turning the manuscript into a physical product. Generating ebooks slots into this workflow in place of "send PDF file to printer, order x thousand copies").

Part of the workflow involves running the copy-edited manuscript past the author for comment, and letting the author rub their bleeding eyeballs over the page proofs. This isn't part of the 6 books/year/employee, but goes on the author's side of the balance sheet in addition to "write the book in the first place".

So, I estimate a book takes roughly 2 months of publishing company employee labour to produce. Plus another 4 weeks of author eyeball time (which is that part of the publisher workflow the author undertakes — see previous paragraph).

When you add it all up: if I'm as efficient as a trade publisher, it would take me roughly 3 months to produce a book that also took me 6 months to write. More realistically, I'm likely to be less streamlined and efficient than a publisher who specializes in this job. This supposes I'm sufficiently plugged-in to commission my own copy-editor, book designer, cover artist, and typesetter. I then have to handle the contractual, accounting, and tax side of things. Ebooks are not VAT-exempt in the UK, so there's quarterly VAT accounting. Sales through Amazon.com outside the UK are liable for US tax withholding unless I jump through various international tax-treaty related hoops. HMRC are likely to sit up and pay attention if I suddenly switch from being a trade supplier to a direct seller, so I'd better have my books ready for inspection at any time.

I reckon 3 months of management time per self-published book is probably an optimistic, low-ball estimate. I should also note that, as a worker, my strength lies in generating crazy new ideas and hacking out new stories, not in sales accountancy, project management, and attention to fiddly details. (This is probably true for many if not most authors of fiction.)

Anyway: this is why I don't self-publish. Yes, I could do it. But it'd suck up a huge amount of time I would prefer to spend doing what I enjoy (writing) and force me to do stuff I do not enjoy (reading contracts, accounting, managing other people). The only sane way to do it would be to hire someone else to do all the boring crap on my behalf. And do you know what we call people who do that? We call them publishers.

144 Comments

1:

I think the main problem over this topic was splitting the discourse in things you as a writer with a personal set of preferences and quirks and workstyle are comfortable or uncomfortable doing, from things that in general are a bad/good idea due to the economic situation/copyright laws/relationship with readers/technology constraints.

If the discussion in the previous topic was about if self-publishing in the current enviroment can be a good idea is one thing, if the discourse was how you can improve your personal work-output money-input rspecting your personal preferences, it's a different situation because subjective factors start to come into play.

2:

That's the exact paradox of starting your own business. If it works, you end up running a business instead of doing whatever it was you started the business to do.

3:

Note I titled this entry "why I don't self-publish", not "why you shouldn't self-publish".

4:
The only sane way to do it would be to hire someone else to do all the boring crap on my behalf. And do you know what we call people who do that? We call them publishers.

Well, yes and no. We call them publishers when they've licensed the rights and take most of the profit. We call them employees when they work for us, and get paid a salary, and optionally a smaller percentage of the profit.

I respect your desire to write more, and that is a perfectly acceptable reason to stay with a publisher. That can be the end of this comment, right there.

However, you'd probably earn between 2x and 6x the royalties if you self-published.

That's easily enough to hire an employee to deal with all the aspects of self-publishing and an accountant to deal with the taxes.

I think the impact is likely to be more smaller. You've depicted the publishing work to be 50% of the writing work. I suspect with the right help, it's likely to be 20% of the writing work. Maybe that's worth it, maybe it isn't.

5:

I just want to point out a couple of bizarre factoids that make this accounting interesting.

It's not that I disagree with any of the above numbers, but I personally can gin out a book cover in an afternoon that satisfies me, and in another afternoon I can gin out a book design that also satisfies me. I also have a low typo count to, by copy editing myself with two beta readers. That took less than a week for 100,000 words.

Now, what I can't do is create a cover that screams mainstream science fiction--this requires art skills I don't have, and a bigger set of specialized fonts than Microsoft gave me. Similarly, book design with the stylish fonts and custom thumbnail section dividers is something I haven't tried yet, although I plan to do so in my next self-publishing effort.

In other words, there's a lot of effort that goes into making a book look like a mainstream science fiction book. While I won't argue for a second that this effort isn't important for marketing, I will argue that Amazon has made a fortune by asking whether this extra effort can be profitably eliminated.

Is all this effort necessary to produce commercial SFF? Or has everyone been conditioned to think it's necessary? That's worth thinking about, because I don't hear a lot of love for most mainstream science fiction covers. Michael Whelan artwork, yes, that's great, but when a book cover doesn't turn into a proper Amazon thumbnail or crudely mashes T&A buttons in ways that annoy over half the audience, you've got to ask if that's money well spent.

6:

We call them publishers when they've licensed the rights and take most of the profit.

Actually, no they don't.

The author makes around 10% of the net sale price as profit. So does the publisher. Another 10% goes on production overheads, and the other 70% ... goes to middle-men like Barnes and Noble, Amazon, or Waterstones.

(Why won't this myth lie down and die? The idea that authors get ripped off by traditional publishers is only plausible if you have no idea how the business works.)

If I thought self-publishing was the way to make 2x to 4x as much profit, I'd do it in a shot. Alas, it ain't.

7:

There's one more very big thing that publishers do. They spread the risk. If Charlie has a bad product for a year or two, he might just go under. Those employees he has hired want to get paid no matter what his sales are.

But a publisher can afford a bad book or few per year. The cash flow from the successful ones fund the occasional turkey. Plus the middling ones that while not turkeys don't make much money.

Charlie How do advances in the fiction book publishing world work if you don't come close to your sales targets? Do they try and claw them back or write them off?

8:

How do advances in the fiction book publishing world work if you don't come close to your sales targets? Do they try and claw them back or write them off?

The author-publisher contract contains a lot of boilerplate covering this. There are things I can do that make me liable to repay the advance if I don't hold up my end of the contract -- by not delivering the book on schedule, by not performing agreed tasks (e.g. editorial-requested changes), and so on. (In practice, the publisher would have to be really pissed at me to activate these clauses: if a book is a month late arriving but is in time for production and looks good, who cares? Especially as it'd permanently burn the author/publisher relationship and generate bad publicity in the trade press.)

However, if I fulfill my side of the contract and the book tanks through no fault of my own, they write off the advance: it's their loss.

(They might subsequently choose not to make me an offer for the next book, or to offer a reduced advance, but that's not the same.)

A point worth noting: publishers calculate the anticipated profit and loss for a novel before they make an offer of an advance. If they're doing their job right, the book will slide into profitability shortly before the advance is fully earned-out. So the author may not earn any more royalties and the publisher may have overpaid on the advance, but they're not out of pocket.

9:

Well said Charlie

10:

I appreciate and respect the presentation that you have made. It is certainly reasonable to think that you can pump out a lot more books per year if you do the writing, and publishers handle "all that other stuff."

The "nuance" that could change the picture would be if the publishers were to then take an excessive portion of the profits falling out of the resulting "pumping out a bunch of books."

If your part of producing the book costs you six months, and you could double your profitability by taking on the burden of the other three months of efforts, then that might well be a good deal. (And if you're lucky, you might be able to pass some of that three month burden to a friend/spouse/nephew, so that the value "stays in the family.")

With some of the recent stunts that publishers have been trying to pull, it wouldn't shock me to find that self-publishing is a good deal for a lot of would-be new authors.

11:

"The author makes around 10% of the net sale price as profit. So does the publisher. Another 10% goes on production overheads, and the other 70% ... goes to middle-men like Barnes and Noble, Amazon, or Waterstones."

Can I just torrent your books, send you 30% of the sale price and have you pass half that along to the publisher, therefor giving both of you an extra 5%?

12:

Your point about self-publishing being a good deal for would-be new authors is worth noting.

The biggest enemy any author has is obscurity: if people don't know you're out there, they can't buy your books!

In my case, I'm [obviously] not new and not obscure. But if you can't convince an existing publisher to take you on and promote your work, you may be trapped in a catch-22 situation: no visibility means no sales, no sales mean no visibility.

The worm in the apple of self-publishing for people in this bind is quite simple: we are very bad at critically analyzing the quality of our own work. I'm currently working on my twenty-something'th novel (if you ignore the dozen or so unpublishable ones I wrote while I was teaching myself how to do this thing) and I am still uncertain as to whether it's any good at this stage. Newbie writers frequently over-estimate the quality of their own work, and try to get it published much too soon; I know I did (and I collected a fat file of rejections in my teens and 20s for stuff that, with 20/20 hindsight, was frankly unpublishable).

So it's a bit of a gamble. A new author might be able to short-circuit the lengthy process of breaking into the traditional publishing business and go straight to the top. But that's about as likely as a lottery win. More likely, they'd overestimate the quality of their writing, screw up the publishing side of things (amateurish cover design, no or poor editing and proofreading) and either vanish into the sea of self-published amateurs, or worse, get a negative reputation.

13:

Given an ebook-only format with distribution through Amazon, a lot of this production work could be skipped, though the end-product would be much, much rougher: Think spelling mistakes, some awkwardness on layout, crappy cover images, etc. The major problem is the marketing side. Many authors have seen success in the ebook/amazon route, but for everyone managing a decent living there are likely 100-1000 languishing in obscurity.

Even if an author could pump out a manuscript that was without need of any sort of editing, and was also a graphic design genius that could whip out beautiful covers in an hour, a product != sales & revenue without a way to get people to purchase it. The marketing side is probably the hardest portion to self-publishing.

14:

@DavidL:

Publishers share risks, yes. The good ones also provide editing (my books are better, more logical, and more fully developed for editing. I do not produce flawless work, and editing is a different skill set than writing and not all writers have it), copyediting, proofreading, marketing (I can email my publicist and have HER send books to reviewers and contests: average time saving, half an hour per book!), translation services, emotional support, book/cover design, and somebody to blame when sales tank. They also provide a legal department whose services I can use--as does my agent.

I do some self-publishing, with a collective of other authors. (Shadow Unit: http://shadowunit.org/) So I know exactly how much work it is.

Charlie is not overstating the time investment in the production end of a book. And that's without even considering the money/people management aspect of it, and the marketing aspect.

The thing is, if I wanted to be a marketer, I could be making a six-figure income as a marketer. If I wanted to do sales, I could do sales.

I spent fifteen, twenty years working on my writing skills so I could write for a living. It's what I'm good at.

Self publishing is a totally valid way for some writers to get their work out there, and indie creators do some amazing innovative insanely wonderful things. And I suspect that a portion of my work will always be handled that way--but working with a traditional publisher has real benefits as well.

Advances, for example. Getting paid 33% on signing allows me to pay my bills WHILE I WRITE THE BOOK. Not to be sneered at.

15:

Can I just torrent your books, send you 30% of the sale price and have you pass half that along to the publisher, therefor giving both of you an extra 5%?

<tongue-in-cheek>Sure. Just first come up with an accounting package so I can show the tax officers (and the publisher's accountants) where the money is coming from?</tongue-in-cheek>

Seriously, you just invented a new distribution channel, is all. Unfortunately most people won't use it. The reason everyone self-pubs via Amazon is because the readers have been trained that Amazon is where the books are; I could self-publish direct from my own website, but I'd get maybe 10% of the sales, if that, simply because people go to AMZN to buy books, not here.

(To make matters worse, AMZN's contract forbids selling ebooks for less money via other outlets. So I couldn't, say, sell ebooks for $10 via AMZN and for $5 from my own blog.)

16:

You're also assuming that an the final product would be as good. I'm thinking that 'editor' sounds like a job skill in itself that is pretty different from 'author'.

I suppose a 'self published' author could sub-contract that work out, but they would still have more editorial control which may not be good for quality.

And then it just comes back to the risk thing. What you're calculation actually is is how much of the profit the publisher takes is for taking (and spreading) the financial risk. Both taking and spreading risk are profitable activities, you'd need to know the cost outsource those as well to have a real comparison.

17:

And, following on from this argument, asking your friends "what do you think of 'The Covenant-Hikers Guide to Lost Dragon at Dune's Foundation' (based on an idea by Dave Langford)?" is likely to get "I don't want to hurt your feelings" responses rather than honest ones.

18:

Spending more time doing management stuff than writing... This is actually what scares me most in self publishing.

19:

'With some of the recent stunts that publishers have been trying to pull'

Yeah, the unscrupulous publishers meme is a strong one. Perversely, it seems to work in favour of actual unscrupulous publishers eg 'Let us help you self-publish'.

I get this from recent Scalzi blog posts. By the by, with his recent experimental effort, the individual episodes look very professionally produced indeed, a joy to the eye.

20:

With some of the recent stunts that publishers have been trying to pull, it wouldn't shock me to find that self-publishing is a good deal for a lot of would-be new authors

I don't really see how a new author (I.e relatively inexperienced) could take on all the specialised roles that a publisher provides. OGH's 3 month estimate of the extra work needed is the ideal, it implies that the work can be handled just as well as the people who do it now. This is probably unlikely as current workers benefit from career experience, training, appropriate resources and contacts.

It might be possible but it seems far more risky (in terms of chance of doing the job poorly) and expensive (in terms of the time it takes) than conventional methods. Given that if it is accomplished as good as a publisher in the ideal time it only generates 50% more revenue what are the chances that it will result in a net gain from someone who has to learn from scratch?

21:

Most publishers are not unscrupulous, most of the time.

There are a small number of bottom-feeders who exploit the vulnerable; Preditors and Editors keeps track of them, and is a vital resource if you're trying to get published in genre fiction. (Remember Yog's Law: Money flows towards the author. If it doesn't, something is wrong.)

Less frequently, a committee inside a publishing conglomerate -- usually so far removed from the sharp end that it's staffed entirely by non-editorial people -- comes up with a really stupid idea that loses track of the usual risk/profit balance that keeps everyone happy (by trying to grab the entire pot). All of the big six five are prone to this particular kind of committee-induced insanity; it usually goes away after their noses are rubbed in their stupidity in public. (Most recent example.)

Sometimes a small publisher runs into difficulties -- cash flow can be a killer if they try to expand too fast, or a key employee goes batshit insane and has a breakdown, or one of their distributors goes bust on a Friday while owing them the money to pay their printer the following Monday. (That latter one is why Tor, previously independent, was sold overnight to St Martin's Press. It sucked, but not as much as going bankrupt would have.) If this happens to a small publisher, the authors -- as minor creditors -- often get the shitty end of the stick: for example, here in the UK the tax man always gets first dibs on the proceeds from a liquidation.

But, in general, publishers are honest because their business model is at its most profitable when they develop a mutually profitable repeat business relationship with their suppliers (the authors).

22:

I'm not sure where the money goes, but I'm sure it's a different economic landscape for self-publishing vs. traditional.

From March 2012 to February 2013, I sold 18,998 self-published books.

The total gross sales (e.g. price to end buyer) was $68,825 and I received $36,268 in royalties. That’s $1.90/book sold and royalties as a percent of gross was 52%.

When I deduct my expenses of $1,920 (editing and cover art; probably low because I have a book designer in the family), my net profit is $34,348.

That leaves profit as percent of gross sales at 49.9%, and my personal profit per book at $1.80.

That's a 5x difference from 10% of net sale price.

Then again, I'm now going the traditional pub route to get wider distribution and avoid getting locked into Amazon as my only effective channel. I'm not trying to make the argument for one versus the other as the right choice, just pointing out that there's appears to be a big economic difference between being your own publisher, even if you hire to do it, and giving it to someone else.

23:

if you ignore the dozen or so unpublishable ones I wrote while I was teaching myself how to do this thing

For what it's worth, incidentally, I liked Scratch Monkey a whole lot and I would pay money to read more of the "unpublishables". Consider that a binding offer if you like.

24:

Looking at Kindle ebooks of the obviously self-published sort, I'm pretty sure I am better than that.

I am also pretty sure that I am not a consistently good writer.

So maybe I have fewer illusions than most self-published authors. Likely return, bugger all. And a metric shit-load of hassle, starting with having to sort out the international tax situation for an unknown return.

I am not the writer a vanity press makes a profit from, and some of the appeal of self-publishing is, I think, rooted in the same soil as vanity publishing.

I don't know publishing, but I have some hard-won knowledge about business. My writing efforts are fun, sometimes. The settings are an escape, occasionally (I have had some great dreams). And there is that sense of making a V-sign in the direction of the school-teachers of my past.

Kindle Direct Publishing? My answer would be seriously unprintable.

25:

Your editing is on the low side; I've paid for professional copy editing and, with proofreading, I'd expect to pay 2-4 times that much (albeit expecting a very high standard of work). This doesn't include high level editing, either -- again, a different skill set.

For sales of 19,000 books ... if that was hardcover, the royalties would be on an escalator and somewhere above 10,000 sales they'd be paying 15%, not 10%. I expect to make around $1 per book in profit; if I had that kind of run, I'd be looking at $1.50, unless they were being sold at deep discount.

As things stand, I think I can admit in public to making more than $32,000 per book. Quite a bit more.

26:

Some of those stunts are well-documented, and when they got publicity and informed criticism, the publisher concerned backed down rather hastily.

Writers Beware reports the climb-down

Writer's Beware is a website that reports on the lurking dangers of getting published, and I don't think it is good that a major publisher gets a report.

I am wondering just where the idea came from, and if the people responsible knew what they were doing.

And the apparent attitude to submissions could be a long-term killer for publishers who want to be the only one getting a particular submission, and then never seem to get around to reading it.

Publishers can do some apparently stupid things.

27:

"Scratch Monkey" was far and away better than the other stuff. Trust me on this. Subsequent books that were unpublishable have already been sliced, diced, and recycled in stuff you're familiar with (hint: "A Colder War" and "The Atrocity Archives" didn't come out of a vacuum); their original forms weren't published because, well, Reasons. Prior to "Scratch Monkey" there's just a wilderness of learning exercises, except for the short fiction and one novel that might eventually get re-visited and re-written from scratch (anyone up for Charlie Stross does magical realism in "Vurt" mode?).

28:

Publishers can do some apparently stupid things.

Here's my working hypothesis as to why this happens:

"Publishers" are corporations. Some of their people are engaged in the business of publishing. Others are engaged in the iron law of bureaucracy. Revenue comes entirely from the first group, which is a good thing, because it tends to prevent the latter from dominating the corporation. But occasionally the latter think, "why can't we do that?" and give it a go, in a crappy cargo-cult fashion. They invariably get their heads handed to them on a plate once their initiative goes public, but because they're organizational game-players nobody on the practical side of the house gets a look-in, much less a veto, before they go public.

Note that the bigger the bureaucracy corporation, the higher the frequency of these idiotic screw-up attempts at re-inventing the business from the ground up (badly).

29:

That's why I'm grateful to Steve Jackson Games, who suggest topics for books that will sell, check my prose and my rules, do page layout, procure cover and interior art, get printing done, and get my books to distributors and sell them via their online ordering process—or, these days, do pdf-only versions of a larger number of books. In theory I could learn to do those things; in practice they'd take up a lot of time and boredom, and I'd likely enough do some of them badly enough to lose money. Being able to do just the part that I can do well is a boon.

30:

OGH: anyone up for Charlie Stross does magical realism in "Vurt" mode?

I'd buy it. I'd buy almost anything you might write. (Just don't do paranormal romance, OK?)

31:

IMO Charlie does "other people's styles" very well, sometimes better than the originals (BASHFUL INCENDIARY for example) so I'd buy it too.

Also, IMO there's still space for more paranormal romance, but at a better Charlaine Harris or Laurell K Hamilton writing level, rather than at a Twitlight level, mkay?

32:

I recently stumbled over this: http://www.perceptualedge.com/blog/?p=1521

He both self-published and wrote for O'Reilly, and was not amused. I wonder if non-fiction might be a different beast entirely: Because let's face it, your typical editor will have a very tough time when the topic is "Efficient Design Patterns for embedded systems in C++" or something similar, and the same holds true for layout (don't you dare touch my code layout!).

And then there is the problem of getting a foot in with a publisher as a fledgling author: Shamus Young self-published The Witch Watch, and it was rather decent historical fantasy. Not the best book I've ever read, but still far better than the NYT #1 best-seller Anathem that I opened up yesterday and put down again after fourty pages of constant eye-rolling; He makes up new words at a silly pace, even for things that have names (e.g. mass [prover], camera [speelycaptor], movies [farspark OR speely, depending on codec!] endorphins, watches, Pythagoras). I get that plants on another planet need a new name, but video cameras do not! And don't waste the first thirty pages of your book with people talking about winding up clocks and dusting floors, because that's fucking boring.

33:

A couple of ideas:

Charlie, would it be possible to have somebody you work with at your publisher do a guest-post on the publishing and answer questions? (I suggest warning your moderators ahead of time so they can schedule extra time for the flaming/trolling which will ensue) It would be really great to get insight into the industry from other parts in the workflow. Perhaps also your/an agent, too, who can describe the process from their vantage point.

Does general editing occur before/after/during copy editing?

For books which are expected to be e-book only (or at least initially), would it be possible to use rapid versioning and crowd-sourcing to handle some part of copy-editing? If so, would that be able to cut out cost or turn-around time? Find 10 spelling errors and get access to the next book 3 weeks in advance. Hey - if it "works" in the software industry...

34:

@32: In theory, technical publishers have specialist editors, copyeditors, book designers, etc. (although cost-cutting and the assimilation of academic by for-profit publishing are threatening this).

I know Mr. Stephenson and his books slightly, and I think that like late Heinlein he would benefit from an editor who made him trim down his books and restrain his artistic experiments. Unlike the late Heinlein, he is good enough to pull off many of those experiments, but they still irk.

35:

Ha. Somebody doesn't get Anathem. Film at 11. Sorry for the snideness, but there are those of us who didn't care for the longhand Newtonian stuff (I mention this as a possible test for not just loving Stephenson uncritically) who truly appreciate Anathem. I've read it five times so far, and it's close to being a book I can read twice in a row. Far and away Stephenson's best work, and I don't say that lightly.

Unless you're just a slowish reader (I wouldn't urge this on my wife, whose native language isn't English), I recommend you push past the start - don't skip it! - and let the story take hold.

36:

I'm with Paws on OGH's ability to adopt other authors forms...I still find myself laughing at ideas in Trunk & Disorderly themed from PG Woodhouse.

I loved Jeeves & Bertie and the takeoff is perfect.

I'm also with Sean on Stephenson, too clever and especially too prolix by half.

-- Andrew

37:

I reckon 3 months of management time per self-published book is probably an optimistic, low-ball estimate.

Having self-published 2 books, I can say 3 months is about the median length of time for post-production, but it all depends on the length and complexity of the work.

My most recent book is a novella, which took about 6 weeks in Post. The next book will take longer because it's a big novel (+/- 135K words) so I'm setting aside 4 months for Post. Since I do my own graphic design*, I'm able to shave a few weeks/thousand dollars off the overhead, but that's far from typical.

And of course my advertising budget is (fishes in pockets) $0.35, there's absolutely no guarantee that I haven't wasted nearly 2 years of creative energy.** The upside is that I sometime this fall, I'll be able to hold the physical book in my hands and say, "I made this." So there's still some reward.

  • Highly ill-advised unless you also have a BFA in graphic design, and even then quality can be iffy. ** Since a secondary goal is to hook a publisher with the shiny well crafted result, there really is no wasted effort, but still.

--Keith Edwards

38:

All this is in essence an argument about Adam Smith's Pin Factory.

I am not going to claim I understand all that he is getting at, mind you, but it all comes down to the division of labour, so that eveyone benefits from specialisation. And the problems come from those greedy sods who think that the labourers don't deserve to be paid.

[Much ranting deleted, concerning large companies and tax avoidance, amongst other things.]

39:

Oh my god yes, take my money.

...semi-relatedly to this whole discussion, I wonder if there's any way to determine precisely -which- crazy ideas would work well with your potential audience other than actually writing the books and working it all out in retrospect. Still wouldn't help a new / first time author trying to build an audience, but.

40:

I've a sneaking suspicion that doing it all yourself would reduce your productivity even more than these calculations suggest. Because I'll bet that lots of those things the publishers do need sudden and instant total attention that would wrench you away from writing: just when you'd worked out what you wanted to say you'd be up to your neck in licensing problems with a picture used in your cover art or something equally important and trivial.

And if you were sick, or on holiday, it wouldn't just that you would not be writing. Nothing would be happening: no review copies being sent out, no art work commissioned, no printers being chased. Nothing.

If you're lucky enough to be both good at something, enjoy doing it and can get paid for it, why on earth voluntarily take on all sorts of other stuff?

41:

Because I'll bet that lots of those things the publishers do need sudden and instant total attention that would wrench you away from writing:

Like trying to write code (a very creative process in many cases) while operating as a sole proprietor out of your house. Sometimes you can't just lock the door and turn off the phone for 8 hours no matter how much you want to do so.

42:
The author makes around 10% of the net sale price as profit. So does the publisher. Another 10% goes on production overheads, and the other 70% ... goes to middle-men like Barnes and Noble, Amazon, or Waterstones.

So

1) Disintermediate publishers. 2) ... 3) Virtually free ebooks!!!

is wrong. But,

1) Break Amazon's monopoly... 2) up to 70% off ebooks!

might work.

43:

Amazon pays out something like 35% to 70% royalties on ebooks. Charlie must be talking about the % on physical books?

44:

Charlie,

I feel like we've been round these houses too much before - and I quite understand your personal reasons for not wanting to self-publish (don't screw up a good thing).

However, as I think you realise, life is change and with the changes everyone can see coming around the corner, we can see that the status quo isn't going to hold. In particular, even though you say 10% goes to the publisher, I'm betting that much more than 10% eventually goes to them on eBooks, Amazon or not. 10% to the author isn't sustainable in an eBook world - the buyers won't wear it.

I'd also point out a particularly key reality. People are happy to give the author money for a book. They are unhappy to see publishers, distributors, retailers, Amazon eating into that substantially. As such, I try NOT to give Amazon/Apple etc. money for eBooks (plus I hate DRM). I'm much more likely to pay the author, on the author's site, £10 for an ebook. That characteristic of the marketplace has to get set against your 'discovery' aspect - and I'm betting it will grow over time.

Whilst I think you, and any author I want to read, should be putting 100% of their effort into doing what they are good at, I do think the edifice of publishing that has grown up over the centuries has created a model where the creators of value are disempowered and exploited (hello Marx). I think the scope for something new is there, even required for survival, and hope someone does it before the big vested interests get a vice like grip on the new marketplace, warping laws/contracts (even more) to their own ends.

Maybe I'm saying you need an article less on "why I don't self-publish" and more on "how we can fix the problems/break the contradictions with self-publishing".

45:

The problem with your analysis is that you know how publishing companies do things. Recreating their workflow is a horrible idea. You are correct about that. But doing things their way isn't the only option. If you imagine a digital first workflow, things will look very different. Post-production can take just minutes* when you plan properly, but expect it to take a few weeks the first time. If you liked the control of making all the decisions, doing it yourself would be appealing. Because you don't, you should look at working with a digital first modern publisher, like Bob Mayer and Jen Talty's Cool Gus Publishing. Not to replace your current publisher, but to cover your bets. Publishing is changing. Don't get left behind.

*To do that, you have to write like you are writing a serial and have the cover and book design done up front.

46:
Amazon pays out something like 35% to 70% royalties on ebooks. Charlie must be talking about the % on physical books?

Amazon pays that when they "publish" the book direct. So they're taking the publishing slice and the distribution slice of the profit - and can afford to give a different percentage to the author.

(... and the higher percentages only apply to some countries and have interesting riders if you dig into the details... and everybody forgets the "delivery" charge that Amazon throw onto the top each delivery which can take a slice out of the profits depending on the kind of book.)

OGH won't be getting 35% from any of his book sales on Amazon, because Amazon will take the normal retailer slice of the profit when they sell the ebook from Orbit or whoever.

In fact he'll probably be getting less actual money since ebooks tend to be discounted more - because the inaccurate folk-theory is that they cost substantially less to make.

47:
In particular, even though you say 10% goes to the publisher, I'm betting that much more than 10% eventually goes to them on eBooks, Amazon or not

Why?

Genuine question ;-) I'm curious why you think publishers get more money from ebooks.

48:

"Post-production can take minutes" -- well, post production doesn't include editing, either line editing, or copy editing, or proofreading. I don't think any of those steps are optional, and they actually account for about 80% of the labour-intensive side of what happens to the book after I finish writing the manuscript.

(I have looked at self-publishing. I have editors who I pay to work for me from time to time. I'm married to a typesetter/proofreader/graphic designer. I'm rusty but I've built e-commerce websites. What does it take to convince you that I'm not a newbie at this whole thing, and that I know what I'm talking about?)

49:

That's a fair general point. I reckon readers would be happy if authors get more money. But they don't seem to know who else has to work on the book they eventually read. Are they hypocrites, or is it just that they don't know about Amazon's minimum-wage warehouse workers?

50:

As I see it, if I went with IanS's suggestion, I'd get more money ... then spend it all on editorial and production services, and have the added headache of managing the people who do the work for me.

I. Do. Not. Want. To. Be. A. Manager.

I. Want. To. Write.

What part of this is so hard for people to understand?

Maybe it would help if I add that, like the vast majority of authors, I'm an introvert. I do not handle employer/employee relationships gracefully, and I resent being forced to interact with people. (The phone ringing just once while I'm working can throw me out of my workflow for half an hour.) The two hats -- production management, and novelist -- cannot be worn by this head at the same time.

51:

Eliz Bear @ 14 Point You are paid whist the book is being written. I don’t think Charlie emphasised enough the actual amount of MONEY as well as time actually saved by going through a publisher (provided publisher/agent etc is honest, of course – Georgette Heyer had this problem IIRC)

Graham @ 16 Yes Proof-reading, even a simple short scientific paper can be difficult – been there, done that. Also Charlie @ 25 Ditto zhochaka @ 38

@27 Was isst “Wurt” ??

52:

"Vurt" (not "Wurt") is a novel by Jeff Noon.

53:

Was isst “Wurt” ??

In this context, who knows?

But were you to type 'vurt' into Wikipedia, you will be taken to a description of the 1993 novel by Jeff Noon.

54:

I'd also point out a particularly key reality. People are happy to give the author money for a book. They are unhappy to see publishers, distributors, retailers, Amazon eating into that substantially. As such, I try NOT to give Amazon/Apple etc. money for eBooks (plus I hate DRM). I'm much more likely to pay the author, on the author's site, £10 for an ebook. That characteristic of the marketplace has to get set against your 'discovery' aspect - and I'm betting it will grow over time.

I strongly disagree. Apple and Google are proving you wrong. And my interactions with people at large indicate you are wrong. People like buying music/books/whatever from Apple and Amazon because it's easy. They have no interest in going to individual web sites to buy a book direct from an author. There are exceptions. The crowd around here being a big one. But we should not mistake the crowd we're in constantly with the universe at large.

Sears, then KMart, then Walmart destroyed the previous retail models by figuring out what people needed/wanted to buy and putting more of it in one place with lower prices in many cases. A&P did it for groceries overcoming fierce opposition along the way. Now there are very few small grocers or butcher shops or whatever outside of some very dense urban areas relative to the past.

55:

In the spirit of enquiry, I typed "wurt%20" into Wikipedia search, and discovered a sort of artificial dwelling hill (large enough to support a small village) from the Netherlands and Danish/German border regions.

56:

What does it take to convince you that I'm not a newbie at this whole thing, and that I know what I'm talking about?

Well there's the issue of facts having little sway over firmly held beliefs. (Say's he dealing with this concept while trying to take care of elderly relatives with lots of other relatives wanting to say what to do "because their way is better" without getting their hands dirty.)

57:

I agree with David L @54. I'd wager that most people who buy books never or very rarely think of the % the author makes on a sale. Let alone feel like they should do something about it by shopping differently.

Perhaps it's different for readers who are aware of the figures, have an author who they really like AND have a sense of community with but for the most part that's not the case. If Charlie chose to sell books from his site for this reason I'd do it but I doubt I'd go to much effort for all the authors I read.

58:

It is precisely the fact that you are not a newbie that is the issue. Everything that you have said is a description of recreating your current work process without a publisher. I agree with that would be stupid. I am not suggesting that you do that. I am suggesting that you adopt a new way of doing your job.

For example, you could get the editing done as you write. This will dramatically lower the cost of managing the editing (not the cost of editing). Editing need not take away any time from your writing because it would be incorporated into your writing process. It will slow you down the first time, but you are capable of learning and improving.

Everything about your current work process is designed for the benefit of a traditional publisher. Traditional publishers are high-cost producers in a digital first world because their work processes are ill-suited to it. The comparison that you make will always come out in favor of sticking with your publisher. It is a waste of time.

59:

For example, you could get the editing done as you write. This will dramatically lower the cost of managing the editing

Your conception of how books are written is rather peculiar; your idea won't work because I don't write a book from start to finish, A-Z. I work in bursts, extending what I've got, then go back and re-write earlier bits before I continue. Very often, somewhere around 70-90% of the way through the first draft I finally work out how the story starts and re-write the beginning.

The book has to be finished before it can go to an editor.

(I know some authors who claim to do everything by outlining and don't start writing until they've got a perfect outline. They're a minority -- 10-30% -- and I'm not one of them. For most of us, the process is iterative creative chaos with embedded feedback loops.)

Note that editing chapters as they're delivered is standard editorial practice in non-fiction, especially in current affairs related books and technical publishing. It just doesn't work for fiction.

Other tasks can be parallelized; for example, cover artwork. Unfortunately these are (a) peripheral tasks rather than critical path elements on the project time-line, and (b) traditional publishers, not being entirely stupid, already outsource and parallelize everything that can be outsourced and parallelized.

More emphasis needed: current book production workflow is arranged by publishers for their own convenience (which means efficiency and cost-effectiveness in producing large volumes of books on a conveyor belt). Production workflow for an author doing it for themselves doesn't need to be optimized for volume, so there's room for some fine-tuning around the edges. But unless you want a badly-edited typo-ridden product with crap cover design, you shouldn't cut corners. And there's less room for fine-tuning than you might think, because publishers have been optimizing their workflow for centuries (literal meaning intended: this is not an exaggeration).

60:

Yeah, the nearest I can see anyone coming to "editing in parallel with writing the first draft" is doing proof reading, and that's a largely pointless task until the author types "The End" ;-) and says "Finished". That doesn't even just apply to fiction: I sometimes find myself goign back and re-writing the earlier parts of technical documents just because I could have put them better.

61:

This comment isn't directed at anyone in particular but I find it interesting that in this discussion and others there are so many who absolutely insist that there must be a faster and more profitable way of writing books, even though they have no expertise in the field.

Is it because there are a few examples of authors who have done it? Or that publishers in other industries have a less favourable reputation? Or that people have experience with word processors and email and think that this is adequate replacement for production and distribution?

It's genuinely interesting that this is a topic for which people become so insistent and stubborn about. I wonder why.

62:

Charlie already parallelises first-pass proofreading and copyediting; he punts chunks of his work-in-progress to a group of pre-readers and gets feedback from them about technical details and infelicities, part of his "go back and fix stuff" writing process. Works for him.

63:

There are so many who absolutely insist that there must be a faster and more profitable way of writing books, even though they have no expertise in the field.

I noticed that too. It seems to be a variant on the "you're a writer, eh? I'm going to write a novel one of these days, too," fallacy -- which is to say, assuming that because the tools are easy to come by, the job must be easy to do.

(See also the lyrics to Dire Straits' "Money for Nothing". Any damn' fool can go out and buy a guitar. It takes a little something extra to play like Mark Knopfler.)

I also periodically -- every few months -- get email enquiries from dot-com start-up folks, who have decided that the next! exciting! way! to get picked up by YCombinator and then to IPO is to DISRUPT PUBLISHING FOREVER, and that their new social-media make-it-easy-for-authors startup is of course going to blow Traditional Publishing™ out of the water by making it easier to publish.

They don't generally appreciate it when I tell them that they're trying to solve the wrong problem. Their assumption is that because they are NEW and DISRUPTIVE then of course all resistance is futile and will crumble. Even though they haven't actually worked out who the customers are, or what the real problem is (competition from recreational activities other than reading), because they're not actually writers or booksellers. Or that there's a guy who got there first and has spent two decades trying to do what they're talking about ... and he's called Jeff Bezos.

64:

"you're a writer, eh? I'm going to write a novel one of these days, too,"

You can probably add to that fallacies of "you're an editor/proof reader/marketer, eh? I can format a word doc, do a spell check and post on forums too"

I notice this type of thinking a lot, mistaking having information for knowledge and tools for skills. I can't count the number of times I've heard people opine that their doctor (for example) is wrong because of what they've read on google. It's classic Dunning-Kruger, probably made worse as you say by the tools (especially computers and the Internet) being so readily available today. I really wonder what the dangers of this type of thinking are, what professions are being neglected because people don't appreciate just how skilled they are.

This topic is probably derailing though.

65:

That's the same sort of stuff I referred to doing myself with tech papers in the post you answered. I didn't know that about Charlie's process, but I'm not surprised.

Anyway, the point was that until the author's finished writing, proff-reiding ;-) and copy editting as such are a waste of effort, because they're as likely as not to introduce new trypos when they do their first re-write.

66:

Let's say that OGH is spot on that producing books to the same level of quality needs pretty much the same investment of time and money as current publishers put into the system.

(Personally I'd say he's right. About the only places I can see for things to have noticeable time improvements were if: 1) print was removed from the equation completely - which makes the back-end supply chain around the printers/printing/material/storage/distro radically simpler; 2) everybody adopts a common sane system for editing and change control rather than the sending Word 1.3 docs by carrier pigeon or whatever that folk [ab]use now. But I'd personally be very surprised if both of those together produced much more than a 10-20% reduction in time/money).

Here's how you make it faster.

You turn down the quality dial.

I've a nasty feeling that it doesn't matter as much as some of us would like to think.

I'm a voracious reader, I write a lot, but I'm a lousy copy editor. I will quite happily read first drafts and be able to give useful critique on the content and structure - but miss typos, name changes, etc. The stuff that pulls up any half way decent copy editor (and probably a fair chunk of the folk here) to a screaming halt.

Wearing my designer hat I'll wave a some comps in front of somebody ranting about the rivers and ladders making it almost unreadable - and the other person will go "huh"?

When most of your purchases happen via the "might also like" mini-icon do lovely covers matter that much?

So... wearing my cynical hat... what's going to happen is this.

Delivery time will get faster and cheaper. It will get faster and cheaper by folk doing less editing, less layout work, less design, etc. And - except for a few very talented/lucky folk, the product coming out of the other end will be worse for it.

And the majority of people will barely notice.

Suddenly I'm all sad :-(

67:

Here's how you make it faster. You turn down the quality dial.

Given that this blog article highlights that post-writing work takes up 33% of the total time is it really worth it to give this all up? You might be able to squeeze out three books in 18 months rather than two but will that result in more profit?

Consider that reading is a big investment relative to other entertainment media. You can't read in the background like you can watch films/TV/YouTube or listen to music. Reading requires near total concentration for several hours. If a book is riddled with typos, bad formatting and, worst of all, plot holes then that enjoyable time becomes a chore. On top of this a bad film won't necessarily put someone off the director/writer (few people take note of that anyway) and watching another episode or listening to another song isn't much effort even after be examples. For authors however its quite unlikely someone will pick up another book if the one they've read was bad.

It seems to me that this idea of writing faster and faster is trying to address a nonexistent problem. What's best for an author is that they regularly put out quality work and attract a healthy fan base. If their work is quality then new readers are likely to invest in the rest of the authors books. I don't understands why fans are so eager to sacrifice the very quality they like just to get the Next Book Now! I'd much rather get a quality book when it's ready.

68:

"The only sane way to do it would be to hire someone else to do all the boring crap on my behalf. And do you know what we call people who do that? We call them publishers."

There are actually two answers to that question. One is "publishers", and the other is "freelancers".

An author can also outsource all of the work that a publisher does to freelancers, if they so desire. This allows them to maintain ultimate control over all details and it allows them to make the final decision on covers, etc. And it also brings in professionals to help them do the work that they're not experts in.

69:

I know authors who have done that.

The general conclusion is that it's simply not worth it: the publishers can do it more cheaply, more effectively, and they can market. That's not even taking into consideration the fact that they have people -- managers -- who have experience keeping track of the subtasks.

ebooks aren't, themselves, going to be a game changer for books. The game changer is the death of the bookstore.

70:

Yes, but if you use free-lancers, then you have to manage them.

Which is something that Charlie would quite like to offload. Or rather not on-load.

71:
Given that this blog article highlights that post-writing work takes up 33% of the total time is it really worth it to give this all up? You might be able to squeeze out three books in 18 months rather than two but will that result in more profit?

In case I wasn't clear - I do not think in any way that what I was suggesting is a good thing ;-)

I'm pretty sure many authors would hate this world. I'm darn sure I'd hate it as a reader.

What I am less certain off is whether it would be an uneconomic world from the publishing / money-making end.

Let's say I automate the text -> ebook pipeline, do minimal lowest-bidder copy editing, and use stock cover designs. This lets me get 2 "bad" books out into the world for the price of 1 "good" book.

If those two books earn more than the one good book - a certain kind of person just rubs there hands and starts scaling the business.

McDonald's doesn't make their money by supplying the best food. It makes it's money supplying good enough food.

I've a nasty feeling that "good enough" books are going to become more profitable. The newer technologies help "good enough" books more than they help "good books", and that's going to do nasty things to the marketplace.

72:

I've a nasty feeling that "good enough" books are going to become more profitable. The newer technologies help "good enough" books more than they help "good books", and that's going to do nasty things to the marketplace.

I thankfully don't think this will happen. Reason being that just because word processors and the Internet have made it easier to copy/paste and distribute stories doesn't mean that the ability to reach potential customers is much easier.

How will anyone find out who you are or what your stories are like? You could try to create a site of your own but as 99% of blogs can tell you it's not that easy to develop a readership. The alternative is to go through an established site but then you're competing against all the other people who have written low quality high quantity stories. What will distinguish yours against the others? I can't see much of an answer beyond quality and if you write quality books then its better for you to get in with a publisher than throw your digital message in a digital bottle into the digital sea.

73:

Gee I'm glad I didn't post my little bit about, "why don't authors just do what music artists are doing and cut out the middle men?". I worked out that the only thing the "middle men" could do that couldn't be easily off-loaded was marketing. I guess I sort of ignored the little bit about needing a manager to coordinate all the various parts.

Anyway, speaking of Baen. Charlie, please talk to your people at Tor and get them to actually sell your books via Baen eBooks. They've been sitting in the list, and on my wishlist, for months (or more). And Baen is the only place I'll buy e-titles, and so until Baen start selling your titles, I can't buy them. (Baen just make it so easy; and I get HTML, which frankly is perfect for me. Web browsers are much better than "ebook reading software". I could easily go into a rant here about epub being a bastardised standard, but I won't.)

74:

"Rule 34" took 18 months to write, for example.

18 months of sweat and toil, yes...but it was one damn fine read, it I may say so.

75:

Baen's webscription storefront is a niche market; if anything, the current is running the other way (with Baen now selling their shelfload of wares via Amazon and other ebook stores).

In any event, it's not my editor at Tor who gets to decide to sell my books via a given channel; it's folks at Macmillan, a level or two up the corporate totem pole. (I can ask, but whether I get an answer ...)

76:

One might think that there isn't a very good market where authors can exert their power as authors to get, at the least, answers...

77:

Err, well you might consider the phrase "Wishing Will Make It So” linked to " When You Wish Upon a Star ... Your Dreams Come True "

Lots of people of my acquaintance have wanted to be PUBLISHED AND RECOGNISED authors...and more than a few people from outside of my circle of People I Know who have striven at the Art of Letters.

Longish in Time and far away and at a reading by Lindsey Davis - the writer who specialises Crime in the Roman Empire at the time of Vespasian - in a long since vanished Bookshop in Newcastle Upon Tyne, I was accosted by a Woman in the front row of the audience who interrogated me on my tastes in...Well she had a Questionire and she was working Out How It Worked, this writing for Fun and Profit thing. And so she asked, as she had others in the audience...why did I read what I read, and so forth. When I reached the end of Ms Davis's signing Q, Davis -who had noticed the Questionnaire had Halted at me - sighed and said " She WANTS to be a Writer " as indeed the Woman did for she had said ..well, I have the Dreadful 'People Will Tell Me Anything, Even Things they Wouldn’t tell their Priest or Medical advisor ' talent that SHE didn't see what all the fuss was about for SHE could DO this!She had worked for many years as a Librarian and SHE ..and so on and so forth.

Not necessarily a Loads of MONEY thing but rather a wish to be recognised. How did that Song go...?

“ Fame I'm gonna live forever !”

http://www.youtube.com/watch?v=poNxrtz_p40

78:

" In any event, it's not my editor at Tor who gets to decide to sell my books via a given channel; it's folks at Macmillan, a level or two up the corporate totem pole. (I can ask, but whether I get an answer ...)

" ...GLENDOWER

I can call spirits from the vasty deep.

HOTSPUR

Why, so can I, or so can any man; But will they come when you do call for them?

GLENDOWER

Why, I can teach you, cousin, to command The devil.

HOTSPUR

And I can teach thee, coz, to shame the devil By telling truth: tell truth and shame the devil. If thou have power to raise him, bring him hither, And I'll be sworn I have power to shame him hence. O, while you live, tell truth and shame the devil!

MORTIMER

Come, come, no more of this unprofitable chat.

"

79:

Our first book (I write with my brother) took nine years to make it to shelf. Our second, third and fourth took on average, one year apiece. I'll stick with publisher contract as impetus, thank you very much.

Dani

80:

The kicker that keeps getting missed here: 1) Being a working, full-time, self-supporting novelist is really, really tough. Not everyone can do it. 2) Charlie is succeeding in the traditional system, has found a way that works. 3) He's not currently a cubicle-dweller contemplating writing novels and discovering what works for him. 4) As he's said multiple times, experimentation is harmful to his cashflow here.

If he were making superstar money, he could afford to take a two year vacation. He could also gamble on something experimental and eat the cost if it failed.

So, I'm willing to keep buying the books conventionally. A friend of mine just self-published his second book. It's niche hard SF, plausible mid-future space politics and warfare. (The Human Reach, if anyone's interested.) It's still a hobby at this point, not a job. He's not even making Stross money yet. But he's having fun. He might have to reconsider his approach if he has a chance to do this full-time, even whether he wants to do it full-time.

81:

Hi, Dani! It gets easier with practice, doesn't it?

The novel that surfaced under the name "Singularity Sky" was written the first time from 1994-96. Then re-written 1996-98. And redrafted in 2000. The version that was published ran to 117,000 words; there are another 130,000 words of junk that got left on the cutting room floor in one stage or another. (If I wrote that inefficiently these days, I'd starve.)

Publishers' contracts: a phenomenon deeply underrated by those who've never had one.

82:

Despite the many and various suggestions to OGH, there is no evidence that doing any of it will books bearing the name Charles Stross more numerous in quantity, more frequent in appearance, or better [whatever "better" may mean] - and there is quite considerable evidence the reverse may be true.

I have read many novels, but have never written one, and wouldn't presume to tell a established author the best way to do it, when the best way to has evidently been found.

83:

Actually the position of the tax authorities in the order of priority of creditors in insolvency was moved in September 2003. Prior to that they were a preferential creditor, they then became an unsecured creditor.

According to this

http://www.finance7.co.uk/creditor_order_of_priority.html

Creditor Order of Priority

During any insolvency process, when company assets are realised there is a specific order in which creditors must be paid by law once the Insolvency Practitioners fees are taken. The Insolvency Practitioners terms for taking fees are predetermined when they are initially appointed.

The first creditor that will receive payment is the secured creditors (fixed charge). Secured creditors have the money that they are owed secured against an asset of the company, usually the asset that they have sold the company. For examples, if a company bought a piece of machinery that they would pay for over a 12 month period, the seller could secure the debt against the machinery which would give them the right to take back the machinery if repayments weren't upheld. Another example of a secured creditor would be a mortgage lender who has security against a property

Once these are paid, preferential creditors would be next on the agenda to be paid. Preferential creditors are usually internal and consist of things such as staff wages and holiday pay claims. Before September 2003, the Inland Revenue (now HMRC) were also classed as a Preferential Creditor, they are now considered an Unsecured Creditor.

Next to receive payment would by any creditors who hold a floating charge. These occur when a debt is secured against general rather than specific assets of the company. A common example would be an overdraft where the bank has a floating charge over assets such as debtors, stock or office furniture. Floating charge assets are assets that the company would use day to day thus making it vary in value over time.

Unsecured creditors are the last external creditors to be paid. These are usually unpaid trade creditors along with any unpaid taxes (PAYE, VAT etc.) and any unsecured loans.

The shareholders are the final creditors that will be paid. Apart from in a Members Voluntary Liquidation, it is rare for the shareholders to receive anything as this would indicate that the Company was solvent on a balance sheet basis.

84:

This whole value chain will get disrupted by technology eventually, it's only a matter of time. Won't be too long before computers will be able to do most of the editing if nothing else.

In the meantime the very fact that anyone can self publish and possibly have a ht is a pretty big disruption. Kick starter as a funding source is another big disruption. Spreaders and tablets are s third. The whole industry has already changed significantly over the last five years, just not so significantly that the old ways are no longer

The only question is how much longer will the old chain endure? Five years? Ten? Twenty at the outside? Somewhere between five and twenty years tops, IMO.

85:

How do you feel about the reduced creative control you have when you work with a publisher? You talk as if the editors do nothing but good for you, but have you ever known them to force a change that you felt was hurtful to your work? I'd be willing to give up some profits in order to let somebody else take care of all the production, marketing, and accounting aspects of publishing a book. I'd be willing to get an outright rejection letter from a publisher and try again. But I hate the idea of somebody else monkeying with my work and imposing his own style on it.

86:

...But I hate the idea of somebody else monkeying with my work and imposing his own style on it.

Haven't worked with an editor (yet, fingers crossed), and just guessing here, but I'm pretty sure that... Real editors don't work that way. They make suggestions for cuts and changes, the author can argue against them or ignore them (though probably not a good idea), but generally I imagine that a good editor is trying to make for the best book possible. Any editor trying to impose their own style and actually making changes without discussing them with the author would/should be fired pretty quickly.

87:

This whole value chain will get disrupted by technology eventually, it's only a matter of time.

Your faith in this ideological touchstone of the revolution is a little bit narrow; I see plenty of fields in which technology, far from being disruptive, actually reinforces the status quo.

(Not saying that publishing is one of them: but your assumption that disruption is inevitable, and good, is very questionable.)

88:

How do you feel about the reduced creative control you have when you work with a publisher?

In the final analysis, everything is negotiable: if they try to change my work in a way I don't like, I have the right to repay the advance, take my ball, and walk. (That's because I retain ownership of the copyrighted work in question: I just license them to sell it.)

Put it another way. I retain complete creative control: and they want to produce the best possible result when they publish my work. There shouldn't be any conflict here, as opposed to negotiation over the right course of action to do best by the work in question. If there is conflict, then the author gets to make a choice between the money and their artistic integrity (although if the author is clearly in the right, and the publisher is well-run, the author probably gets to keep the money and switch to a new editor).

89:

For example, you could get the editing done as you write. This will dramatically lower the cost of managing the editing (not the cost of editing).

Actually, this will most likely increase the cost of managing the editing, because the coordination will be more complicated. It's going to cost more to coordinate the two tasks within one book than to manage the editing of one book while writing the next one.

With modern version control tools the additional cost wouldn't be large, but it would still be there and it definitely wouldn't be a saving, much less a dramatic one.

90:

That's about the way it works.

There are some angles on it, though. Experienced senior editors get to work with newbie authors who are then so overawed by them that they mistake everything the editor says for an instruction rather than a suggestion. And sometimes you do encounter grossly unprofessional behaviour on the part of an editor. The commonest situation is that a copy-editor (an external contractor, remember) decides to go hog-wild and "correct" the author's prose rather than just looking for typos/inconsistencies. (At which point the author is entitled to fume, over-ride the CE's changes, and complain about them to the commissioning editor. And a CE who does this more than once probably doesn't get hired by that publisher any more.)

But the publisher/author relationship is not the draconian big corporation/little disempowered artist power imbalance that many outsiders seem to believe.

91:

I think a lot of people adamantly pushing some kind of new publishing paradigm are not necessarily would-be writers, but are perhaps techno-fantasists. And unlike regular, garden-variety fantasists (who wish they could live in Middle Earth or Westeros, God knows why), these techno-fantasists believe their fantasies can become reality!

So anything that seems to stand in the way of progress/flying-cars/jedi-weddings must be eliminated. Only if the old ways can be destroyed can their glorious shiny future come to be. Since this conversation is about publishing methods, the techno-fetishist of course will advocate trying Kickstarter freelancing, e-whatever publishing, quantum-editing, etc.,etc., and be more than ready to pitch the old methods into the rubbish bin of history. And it goes without saying that they Must Obviously Be Right.

92:

Ryan @ 61 This comment isn't directed at anyone in particular but I find it interesting that in this discussion and others there are so many who absolutely insist that there must be a faster and more profitable way of writing books, even though they have no expertise in the field. What a classic statement! Like the "efficent" bus operators were going to clean up inefficient BR back @ Privatisation .... Oh dearie me ....

A Howard @ 71 I've a nasty feeling that "good enough" books are going to become more profitable. They have been that way for a long time. James Herbert Stephen King Dan Brown - need I go on?

Arnold @ 78 LONG time since I did King Henry IV ptI for "O" level! Note, as usual with Stratford Bill, even in that short piece, the now-everyday phrases in use, which originated there. Scary, isn't it?

93:

I would argue that Stephen King is actually a very good writer -- he is to the late 20th century American blue collar worker much what Charles Dickens was to the 19th century English working class. Don't confuse popularity with poor quality!

(Dan Brown ... I won't argue the toss. James Herbert ... was not in Stephen King's league, but wasn't in Dan Brown's either.)

The thing is, books that are merely "good enough" -- popular entertainment that goes no further than is necessary to keep the reader's attention through a single reading, in other words -- tend to fall by the wayside over time, and their authors are forgotten. King, I think, goes the extra mile: his work has legs.

I'm not in this vocation merely to earn money. If that was all I wanted, I'd have gone into banking. So being merely "good enough" for a quick read isn't good enough for me. (You may now accuse me of harbouring delusions of artistic merit, if you want. It's an odd kind of art, certainly, but it's mine.)

94:

I don't think it's quite as dramatic as that.

Most people simply underestimate how difficult "management" is. There's a popular UK TV programme called "Grand Designs" about people building, or converting to domestic use, unusual properties. Trying to keep the costs down, many of them take on the project management job: they use architects to design, and specialists to do all the different aspects of construction, but organise it themselves. This inevitably turns out to be around an order of magnitude more difficult than they'd expected.

There are a lot of bad managers in business (not unrelated to the fact that it is so hard), but I suspect a lot of the techie contempt for "management" and "suits" comes from a belief than getting a dozen or strands of activity, all done by fallible people with their own strengths and weaknesses, to come together anywhere near on time, or in budget and to do what was expected when put together is a piece of cake, and people only do it because they can't code. They are wrong.

Charlie doesn't want to do management, he wants to write. He is right.

95:

I suspect a lot of the techie contempt for "management" and "suits" comes from a belief than getting a dozen or strands of activity, all done by fallible people with their own strengths and weaknesses, to come together anywhere near on time, or in budget and to do what was expected when put together is a piece of cake, and people only do it because they can't code. They are wrong.

Those techies have either never worked for a company with actual working management, or have benefited from the presence of such good managers that they never noticed what was going on behind the curtain. Good management is like oxygen; you only notice it when it's missing.

I've done courses in project management, worked for companies where being able to plan was part of everybody's job, and have to juggle project management of my own multi-year projects (with different deliverables due to be shipped to different customers). Adding my customers' management workload to my own management workload is the last thing I want to do.

96:

Charlie I may have confused S King with someone else .... Wrote a re-take on Beowulf & also one on the 19th C with murders. The history in the second was so bad I vowed never to read his stuff again, ever. Ah, got it (memory returns - not going down quite with brain-rot YET) Michael Crichton - what a load of un-resarched wrong crap...

"Management" gets a bad name because of the rubbish we have had to put up with here, such as "Lord" Stokes of BL .... And how well Brit car-making companies are doing with non-British management, but the same workforce & nasty unions, err ... umm . you what? Or people like Railtrack (shudder) though part of that was that their remit was deliberatey set up to fail, and the horrible railways were going to get the blame, but the cunning plot didn't work quite as expected ....

And, of course, the MBA syndrome - let's not go there, right now, shall we?

97:

Too bad King's plots suck. But he is absolutely brilliant at describing a bunch of awesome characters on an awesome background slowly going from Nowhere to Nowhere.

OK, OK, off-topic, I'll shut-up...

98:

It might be interesting to look at someone who does self publish. In this case, a guy who does Space Opera, and who was at the Melbourne Worldcon in 2010 as a Hugo nominee.

Howard Tayler of Schlock Mercenary.

He was a manager before he took up doing his web comic full time. His early stuff (still viewable if you go far enough back) is fairly crudely done, as far as artwork is concerned, but his storytelling was always firmly embedded in the SF world (unlike a lot of graphic novel producers). But his ability to produce a strip every single day, over the years that the strip has been running, is a tribute to the fact that he and his wife Sandra manage SM as a small business. He's a publisher (a little more than a book a year). He produces merchandise. He's just run a Kickstarter that funded in under 60 seconds (yes, you read that right - and he'd reached ten times his target in just over two hours).

When it comes to producing those books, he runs a web store too. The books arrive at his house in a truck, and palettes are fork-lifted into his garage. He and Sandra (and probably the kids too) pack those books. They post them. They store the remainder of the inventory.

But the thing is, Howard does have the managerial chops to do it, and he does have the people skills. In terms of creative content per day .. well, it's a tricky comparison, but I reckon Charlie is ahead.

Should Charlie want to follow that route, not only would he be spending more of his time not writing, he'd have to move house too, as I can't see him lugging boxes and boxes and boxes of books up his stairs and down again to the post office.

I strongly suspect the Foglios (the couple behind Girl Genius, also Hugo nominees in Melbourne) are similarly organised.

99:

Something that doesn't get mentioned enough is your energy level.

If you're young and energetic you can put in stupid amounts of working hours, haul bundles of books up and down steep staircases, and generally carry on like a young energetic person.

If you're older and less energetic ... well, that can be a problem. I've done the dot com death march and I know damned well I don't have the energy to do it today; I'm at the age where afternoon naps start to become necessary, rather than a symptom of sloth, if I'm at all run-down or recovering from an illness.

Expecting a fifty-something to behave like a 20-something startup monkey is just plain unrealistic. And the exigencies of self-publication mean that I'd either have to work twice as hard/twice as many hours to keep the management side moving, or I'd end up about half as effective as I am today at the writing side of things.

100:

Reading the comments, I remembered of two experiences with "self-publishing" authors.

First, Lawrence Watt-Evans, has used a contributing system of serialization for the publication of his Ethshar novels since a few years. But he is a known author, with dozens of books behind him, his readers-contributors signal problem in his writing, and his books are, in my knowledge edited after serialization.

Second, CJ. Cherryh, has created with Jane Fancher and Lynn Abbey Closed Circle Gateway to publish some of their works. Cherryh has republished reverted works of SF and Fantasy that she edited and rewrited (For the fantasy Chernevog, Jane Fancher is credited for her contribution to the rewrite).

Jane Fancher, has republished reverted books AND original books, one the fourth in a series originally published by DAW books but badly commercialized.

But I note that Cherryh is 70 years old, with a long backlist, publishing with DAW books her Atevi/Foreigner saga.

All these authors have lengthy careers in sf and fantasy, and others means that their self published works.

Excuse the length and the disorder of that comment, I wished to respond to remarks in the hundred of previous comments (and I am not an english speaker).

101:

It looks to me like there are 2 separate reason to self publish and nether of them is getting books out faster.

Reason 1 is the aspiring author who just can't seem to find a publisher who will put there book into production, and i hope for there sake the reason for this is an already full production line.

Reason 2 is the published known author want to keep more of the net revenue from the book going to their pocket. Instead of the publisher making residuals on what I understand is an hourly job.

So I guess the question for OGH is how does he get all the benefits of what a publisher provides right now without having to keep paying them past the cost of the work they did. The sliding percentage of revenue does work to a slight effect on this but I wouldn't be surprised if they had recouped all of there costs before he started receiving the higher percentage.

102:

Howard Tayler of Schlock Mercenary.

Apropos of not much, Howard Tayler has also gotten quite a few of his Schlock Mercenary books into my local library system. He's not unique in this, Jennie Breeden of the oddly named Devil's Panties web comic has also done it, but no other web comic artist is as well represented on the shelves. That's not just books sold but advertising for his website.

There are lots of web comics and plenty of artists who try to put out dead-tree editions...but getting into public libraries and brick & mortar bookstores isn't easy for the self-published.

103:

Things change. Technology accelerates it. The only thing up for debate is the timing. It's possible that book publishing may rattle along as is for twenty years just seems unlikely to me.

104:

I have worked as management both at the line and executive level for both traditional companies and tech companies. I currently work as a middle manager at a very well know Silicon Valley tech company. Techies have a great respect for managers-as-coaches but generally expect them to be techies or ex techies . They have a good respect for project managers who understand the details. When they are talking about "suits" they generally mean either business people or non technical people. Suits are a love-hate relationship.

The other thing that complicates is that many (but not all) Silicon Valley tech companies are run with fundamentally different organizational philosophies then old school firms. They are much more bottom up then top down, self organizing and lack central command and control structures. In the environment these companies operate in its far far more nimble and innovative way of structuring. It's also very hard to explain how fundamentally different it is from the hierarchical systems

105:

Indeed? Said he, in a Bondish Villain sort of way.

I have become convinced that if I were to attempt your Businesses Publicity Travel Model I'd either be carried off the aircraft on a High Tech Para Medic Stretcher or in a Body Bag ....now there’s an idea! Consider the merits of a Para Medics Carry the Victims Thingy - I've suddenly come to realise that I don’t know the proper nomenclature for the modern casualty stretcher - wouldn’t it be possible to incorporate a body bag into the thing just in case the onsite stabilisation didn’t work?

“Expecting a fifty-something to behave like a 20-something startup monkey is just plain unrealistic..”

Agreed, but surely the 50 something would use his skills and experience to employ the 20 something to make up the energy deficit? Thus the Literary Agent becomes more important as you grow older, and more experienced and Richer in your Craft?

Oh ... and, the Arts?

The way that I look at it is that the writer starts by learning his Craft ...which is how to use the tools of the trade to develop the Idea that sprung from the imagination. The Artist takes both the tools of the trade and the experience of what he has done with that idea in the past and raises the composite to a higher level. Lots of imaginative people who read fiction were dissatisfied with the way that Peter O Donnell concluded his Modesty Blaise series, in its written novel /short story form, with “ Cobra Trap “ but very few people would have had either the craft skill or the experience – leave alone the imagination - to devise an offshoot/homage of the Modesty Blaise series in which – as I interpret it – O’ Donnel must have met Ms Hazard and her Henchman in DRAMATIC Circumstances and afterwards toned the continuing story down to something that The Public could accept in a continuing series that nevertheless did run out of escape and evasion/now get out of this ideas.

106:

Things change. Technology accelerates it. The only thing up for debate is the timing.

This is a statement of ideology, not of fact.

For most of the duration of the human species, change has not been an overriding influence on our lives. In fact, it's only since roughly 1800 that you couldn't live your entire life using only knowledge and practices known to your mother and father. We are undeniably living through the era of the Great Acceleration; but it's probably[*] a sigmoid curve, and we may already be past the steepest part of it.

An interesting point is that this ideology works very well as a match for the political ideology of revolutionary republicanism which emerged in the 18th 17th century, and which everyone reading this blog[**] agrees with -- the ideology that replaced the Divine Right of Kings and the Great Chain of Being as an organisational paradigm with "Liberte, Egalite, Fraternite" (or, in bastardized form, Freedom, Equality of Opportunity [to make money], and Patriotism).

The doctrine of continual change through technology is not value-neutral; it feeds into the continual disruption that enables the permanent revolution of the anti-monarchists to roll forward, and prevents the oligarchs who sit astride the juggernaut from becoming too comfortable. It is in principle possible to suppress change; the problem is that suppression is a sub-optimal strategy in a polycentric world with competing interests. But once the capital imbalances that are driving development in the developing world and immiseration of the proletariat in the post-democratic first world subside, stasis will become an increasingly attractive policy to the oligarchs. (When the Great Acceleration stops, my guess is that the oligarchy will ossify into a nobility, and eventually a monarchical-system-in-all-but-name, within a century at most. And there are already worrying signs that this is happening.)

Please don't deny that you are a believer in this revolutionary ideology -- and it is revolutionary; so much so that Republican Democracy, Fascism, and Communism are just minor doctrinal disputes within it. It's okay to admit it here; I'm a supporter of this ideology, too. None of us are supporters of feudal monarchism; we're all the inheritors of the early Jacobins. Which makes us revolutionaries.

But it's important to understand that virtually the entire mainstream of political and social discourse today is radical and revolutionary by historical standards. (Hell, the concept of sociology itself is a construct of the revolutionary philosophers.) This is not an historically normative set of touchstone ideas to run a society on. We're swimming in the tidal wave set running by an underwater earthquake two centuries ago -- and like fish that live their entire lives in water, we are unable to see our circumstances as the anomaly that they are.

And, as Oliver Cromwell put it, "I beseech you, in the bowels of Christ, think it possible you may be mistaken."

[*] I'm discounting singularities here.

[**] Except you, Moldbug.

107:
Given that this blog article highlights that post-writing work takes up 33% of the total time is it really worth it to give this all up? You might be able to squeeze out three books in 18 months rather than two but will that result in more profit?

Sorry - I've obviously not expressing myself clearly. Let me try again.

1) OGH points out that the stuff that's done in the current publishing pipeline is necessary to producing good work - and that it's not going to get orders of magnitude faster.

2) Others are saying that Magic Technology (tm) can rebuild that pipeline - faster, better, stronger.

What I'm suggestion is a third option that's the worst of both possible worlds. Because that's the mood I'm in at the moment ;-)

a) OGH is correct - the work done in the current publishing pipeline is necessary to produce good fiction.

b) OGH is also correct - that work in that pipeline cannot get radically faster or cheaper

c) Magic Technology (tm) folk are only kinda-sorta correct. You can automate some stuff, but it's the sort of stuff that optimises for "good enough" rather than "great".

The combination of a + b + c changes the economics of publishing enough so that "traditional" publishers get driven out of the market. Because all the real money is in spotting the social media spike around the next "37 Shades of Grey" that popped out of last month's nanowrimo, throwing a small editing team at it to polish it up, and heaving it out of the door and marketing the f**k out of it.

In this future OGH will have to take on the traditional publishing bits himself. Either hiring in an expert or herding the cats on his lonesome.

This will kill his productivity and cut his profits - but we'll hopefully get the rest of the Laundry series ;-)

The world for new writers is harder though.

The next generation won't have the investment and risk spreading strategies of the publishers advance to do their writing with. The OGH:TNG's Scratch Monkey equivalent will probably sell just fine - but the author won't get the ongoing support they need to make progress from the publisher any more.

Since the writers who aspire to "great" need to manage all the necessary work themselves (or afford one of the exceedingly niche experts in the field) this slows down their progress, and slows down the release of new books... which are less profitable anyway in this wonderful new world of McFiction.

TL;DR: Just because authors need the current publishing framework to do their best work, doesn't mean it will actually survive.

108:

This will kill his productivity and cut his profits - but we'll hopefully get the rest of the Laundry series ;-)

Hopefully you'll get them anyway. Sales took a dent when we moved from Golden Gryphon to Ace (because "The Fuller Memorandum" came out the month Borders went bust, and advance orders for the Ace H/C were based on orders for the trade paperback of "The Jennifer Morgue", which had been partially cannibalized by the Golden Gryphon hardcover). But "The Apocalypse Codex" is doing really well, and my publishers want to position "The Rhesus Chart" as a new entrypoint to the series to get new readers on board. Which means, if they're successful, that the prospects for book 6 or book 7 are golden (or at least lined with silver).

TL;DR: Just because authors need the current publishing framework to do their best work, doesn't mean it will actually survive.

Yes. A sobering possibility. I'm probably going to be okay -- if I walk a very careful line between boring you by writing same-old same-old all the time, and scaring my publishers by being dangerously innovative -- but it could be really hard for a ten-years-younger version of me to break in today.

109:
Won't be too long before computers will be able to do most of the editing if nothing else.

What makes you think this?

I can think of some things in the editing process that might be usefully automated (the change control processes for many publishers would make any developer used to vaguely decent source control shudder).

However the editing itself... If you can automate that you can probably automate the author too. Even basic spelling, grammar & continuity checking pretty much require human level intelligence.

110:

So ..oo..Given these entirely exceptional circumstances, and what may be a reasoned wish on the part of the publishers to meet a High Point .well I think upon it as being a High Point...in the Book Buyer/prsent giver market at Newton Mass Time the " "The Rhesus Chart" as a new entrypoint to the series to get new readers on board." will be published in December this Year???

Yea, HAPPY, HAPPY Readership. Let Joy be un-confined and so forth.

111:

Any statement about the future has by it's nature some ideology in it.

However if you talk about just the present and say "things have changed alot in the last X hundred years, that up until now the rate of change has been accelerating" that is a statement of fact.

The planet has never gone through such a period of rapid change. Even before the 18th century that statement was true, just that the time frame was still, as you point out, not fast enough to impact an individual human life

It is possible that Peter Thiel and others are right, we have reached a technology or societal inflection point and are running into a brick wall.

Even that scenario is one of massive cultural change though, since how we are living now is not sustainable without constant technological advancements.

Change is inevitable under either scenario, we either go forward or fall back. Peter's view of the world is actually pretty apocalyptical.

The only future scenario that doesn't really have much possiblity is things staying the same as they are now.

112:

Won't be too long before computers will be able to do most of the editing if nothing else.

I think this grossly underestimates what editors do.

Even the most minimal part, quality control on the text, is not easily automated. We've automated spelling correction and look how well that's worked - there are websites dedicated to nothing but humorous autocorrect errors. Note that checking for typos is the easiest thing. We can expect that other things can be automated, and mediocre grammar checkers are common already; don't hold your breath for a program that will notice when the author forgets a character's ethnicity or sex.

There are also other things that get done, like asking the author what he really meant to say, or hinting that he should rephrase a particular clunky passage, or asking why the publishing house hasn't heard from him in a while.

A computer program can tell you that you've misspelled Mississippi again. It will not point out that you wrote Chapter Seven after a long night at the pub and should really do it all over again. We're going to need human editors for a long time to come.

113:

IBM Watson makes me think this. If you can win a game of jeopardy against the best humanity has to offer, probably editing a book is not far off.

Machine learning is getting pretty good, if we had a feedback loop so we can actually measure people reading a book and where they are dropping out and getting bored, throw in some A/B testing we could likely do something better then editing. Editing is still just some guys opinion even though he may be a very senior guy, he makes mistakes.

And yes, probably we will automate the author too, I think that is a bit further off though. Once you hit that point, you've basically automated all of the value creation homo sapiens are capable of.

114:

Chralie @ 105 Past the steepest part of the tech-sigmoid curve? Really? I assue you are suggesting that Q-computing is not going anywhere,then? If Q-c is going somewhere then we haven't even started... Ditto we are very close to efficient, repeatable (i.e. reliable) direct solar energy conversion (aka artificial photosynthesis) never mind the staedy improvement in conventional PV - both these technologies will have almost no effect (as at pesent) until they reach a tipping point, & then everyone is affected. [ Use my standard historical example here: Railways existed 1812-30, but the real revolution occureed 1830-50, After that, they conquered the world, & there was no going back ] Except .. I disagree as to the cause of the revolutionary underwater politico-social earthquake you mention. Which country won, conclusively & totally in the revolutionary wars of 1792-1815?? WE DID A "constitutional" (by the standards of the time) monarchy, with a strong middle & lower "upper" class, devoted to industrialisation & making money from it. The real revolution came from Boulton, Watt & Stephenson, & Brit politics was elastic enough (just - it was close) to accomodate those changes & run with them. The French revolution was actually a failure both politically & morally, as was the subsequent soviet set of revolutions.

Contrariwise, the USSA (as noted indirectly by Charlie) are heading backwards into a strong-oligarchy setup, to the extent, at times of trying to supress some technologies, unless, of course THEY control the supplies of it, which means even more moolah for them & none for us.

115:

I think you vastly overestimate what IBM's Watson did.

116:
IBM Watson makes me think this. If you can win a game of jeopardy against the best humanity has to offer, probably editing a book is not far off.

They're very, very different kinds of tasks I'm afraid. One is a highly constrained Q&A scenario. The other needs deep text comprehension and a complex model of how humans read and respond to fiction as a minimum. Success in one bears no relation to success in the other.

It's a neat trick - but it's like saying Deep Blue's success in chess means that we're going to get computer's editing books soon (when we don't even have good computer Go players yet ;-)

Machine learning is getting pretty good,

It's getting pretty good at some kinds of tasks in some kinds of contexts. The ones, unfortunately, a long way away from the general text comprehension - let alone being able to spot issues and suggest improvements.

if we had a feedback loop so we can actually measure people reading a book and where they are dropping out and getting bored, throw in some A/B testing we could likely do something better then editing. Editing is still just some guys opinion even though he may be a very senior guy, he makes mistakes.

1) Editing isn't just one guy's opinion. There is a lot of spotting stuff that is wrong. Subtly wrong. Like the character in Ch 2 being "Christie" and being "Christine" in Ch 11 for example. Or the accent of another character being inconsistent. Or the plot point you left dangling in Ch 5 never being resolved.

2) To do A/B testing you need alternatives to test and a way of judging success. At the very best you could use it to test the whether a particular set of editing decisions were good ones - it doesn't replace editing.

Even doing that much with A/B testing is hard since the numbers reading books and the number of changes that editors make, and the weak metrics we have available for judging the success of editing changes, mean that the likelihood of finding things of statistical significance is very low.

And yes, probably we will automate the author too, I think that is a bit further off though. Once you hit that point, you've basically automated all of the value creation homo sapiens are capable of.

Well - I think we're just meat too. So at some point we'll be making artificial editors. Unfortunately that'll be a fair way into the future judging by our current rate of progress, and we'll have to deal with the tedious ethical issues of creating conscious beings just to dig out our egregious writing errors ;-)

117:

agree that Watson is easier then Editor however I think it's a bit more then a neat trick, gonna be your doctor pretty soon too from what I am reading.

However there are also things like Siri, lots and lots of work going into NLP right now, only a matter of time.

My suspicion however is what will replace editing is not just automating what an editor does but achieving the same result with some other workflow.

You don't need to complete replace the human in the loop, making their job ten times easier/faster is probably just as good. Given that most of the editing is still happening with magic markers on hard copy that is getting fedex'd back and forth the bar is pretty low there.

As far as A/B testing goes, yeah you need an algorithm that is smart enough to make changes, that is true. However what it does do is eliminate the "deep text comprehension and a complex model of how humans read and respond to fiction" You don't need to know why they respond just what the response is. just like an ad targeting engine doesn't care why you like what you like. As far as weak signal, may I remind you that when you are reading from a tablet you have a camera pointed at your face....

118:

interesting article from Mohammed AlQuraishi on the future of ML, hope it gets past the spam filter

http://moalquraishi.wordpress.com/2013/03/17/what-hintons-google-move-says-about-the-future-of-machine-learning/

119:

Editing isn't just one guy's opinion. There is a lot of spotting stuff that is wrong. Subtly wrong. Like the character in Ch 2 being "Christie" and being "Christine" in Ch 11 for example. Or the accent of another character being inconsistent. Or the plot point you left dangling in Ch 5 never being resolved.

Yes, this.

I'm willing to believe we might see editor assisting software, such as something that plods through a text and generates a list of characters and some data about them - which might reveal the Christie/Christine problem. James Nicoll's Livejournal recently had a post in which folks speculated about an automated cliche detector; that would certainly be useful, and a sophisticated one might take into account the genre of the work. (Its rule 34 database doesn't bear thinking about.) More subtle things? I don't know.

A computer might be able to notice that a character named Piotr is identified as Romanian in Chapter Six and Ukranian in Chapter Ten. But for the foreseeable future it's going to take a human to diagnose this; is it an author's error, a character's lie, or two different characters named Piotr?

120:

needs deep text comprehension and a complex model of how humans read and respond to fiction as a minimum.

I'm going to go further and state, unequivocally, that writing convincing fiction (as opposed to running a Markov chain algorithm to string random prose fragments together without overall plot) requires theory of mind: the fiction generator needs to be able to model human behaviour, including the behaviour of humans who know they're being modelled by other people. It's an AI-complete problem.

You'll know it's been cracked at the point where you can read a software-generated narrative that relies on an unreliable narrator. If we ever get to that point, congratulations: we've reached the Vingean singularity, and all bets are off.

I'd also add the point that dealing with unreliable narrators is one thing that I don't believe grammar or prose quality checking software can handle, short of full general artificial intelligence. Is the gender-swap in pronoun use referencing a given name an authorial error, or indicative that the character is transitioning between genders? Or ironic, belittling usage in dialog? Context is everything.

121:

You don't need to complete replace the human in the loop, making their job ten times easier/faster is probably just as good.

You can't make their job ten times as fast. Can you improve their reading with comprehension speed by a factor of ten? Without missing important details?

Editing is one of a category of occupations that is dominated by Baumol's cost disease. In fact, so is writing anything above the level of a computer-generated sports report in a newspaper column (where the basic team/score details give you a skeleton you can feed to an algorithmic text generator).

122:

And, err ... which class of unreliable narrator does "Bob" [Laundry] fall into, then? I'm guessing a cross betwen "innocent innaccurate" & "through the eyes of madness" ??

123:
agree that Watson is easier then Editor however I think it's a bit more then a neat trick, gonna be your doctor pretty soon too from what I am reading.

Then you need to be reading stuff that's a little less pop-science ;-)

Computers have been out performing some aspects of doctor's decision making since waaaaay back in the 70's with projects like MYCIN ( http://en.wikipedia.org/wiki/Mycin ).

We've got some great stuff now that's going to, potentially, help a lot more. But these systems aren't going to replace doctor's for a good long while yet.

However there are also things like Siri, lots and lots of work going into NLP right now, only a matter of time.

And I've been hearing phrases like that since I first go into AI back in the 80's.

General text comprehension at the level to enable computers to make editing decisions - or even help editors make editing decisions - is a freaking hard problem. One that we've made surprisingly little progress on in the last twenty or thirty years. In fact most research has been focused away from this area to the more statistically rooted methods that Moore's law has made stupidly more effective.

Siri is superb - but the problem it addresses is speech recognition - not NLP. The NLP side of Siri could have been done in the 80's. If not earlier.

Siri is, in fact, a great example of where machine learning stuff is making progress in leaps an bounds. It's based on big data analysis and pattern recognition work. The kind of stuff that Geoff Hinton does.

It doesn't help at all in the areas where human editors work.

(If you'd like to learn more about machine learning Stanford's Andrew Ng has a great undergrad level course up on on Coursera https://www.coursera.org/course/ml ).

You don't need to complete replace the human in the loop, making their job ten times easier/faster is probably just as good. Given that most of the editing is still happening with magic markers on hard copy that is getting fedex'd back and forth the bar is pretty low there.

Well - I'd say it's already advanced a bit from that. All the publishers I've worked with over the last ten years have had an electronic workflow. It could be better - but it isn't going to be x 10 better. Most of the time is in the reading, the comprehension and the decision making. Not the markup and the change control.

As far as A/B testing goes, yeah you need an algorithm that is smart enough to make changes, that is true. However what it does do is eliminate the "deep text comprehension and a complex model of how humans read and respond to fiction"

Erm. How on earth are you going to have an "algorithm that is smart enough to make changes" in any meaningful way without deep text comprehension and a complex model of how humans read and respond to fiction?

You don't need to know why they respond just what the response is. just like an ad targeting engine doesn't care why you like what you like. As far as weak signal, may I remind you that when you are reading from a tablet you have a camera pointed at your face....

As somebody who does a lot of work in the usability and user testing field - including stuff like eye tracking - can I remind you that the information that you can get is pretty much useless in the vast majority of situations.

With an eye tracking rig I can tell now with 100% certainty that somebody had an N second fixation on a piece of text. To know whether they actually read or comprehended it - I have to ask them.

The signal from ad targeting engines is stupidly simple to determine. Somebody clicked on it - or not. A/B testing things like the checkout process of an e-commerce site is simple. You look at conversions (well - actually that isn't even that simple... but that's a whole separate conversation).

Knowing whether somebody enjoyed a novel - completely different class of problem. Different ballpark.

A ballpark in a different city.

On the moon.

Even if we ignore that problem - it doesn't work. Because the numbers don't add up. Currently a typical print run of a first novel is what - 3000-10,000 books?

You just don't have the raw numbers to get statistical significance on the number of editing changes that editors make. When you're running this number of test in parallel you run into multiple-testing correction issues and this sort of issue http://xkcd.com/882 .

... and that's with the numbers you get on a book that's been edited to a certain level of quality. How do you get people to read the bad books that you're hoping to A/B test to success?

124:

AH @ 123 Your link's broken Try this one instead ?

125:

Bah. Sorry about that. Could The Friendly Moderators fix please (darn full stop).

Also - I've just realised that I've ambled quite a long way towards the strong-AI / singularity strange attractor. If OGH/TFM feel this is derailing the conversation let me know and I'll hush.

126:

I have fixed all three links in your comment. All three cases had missing white space between the link and surrounding punctuation.

(I would suggest using the preview function and checking in those cases where you embed a link.)

127:

D'oh. Thanks.

(if only there was some way to get a computer to do an editing pass on my.... oh... wait... ;-)

128:

Since they're supposed to be a classified record of events to preserve knowledge that would be otherwise lost if Bob cops a R'lyeh one, we could assume they're accurate, but there are different levels of security clearance, and that could explain some lacunae. Looking at how Bob is promoted, and the tools the Laundry has to maintain security, it is plausible that the reader can't read all that Bob has written, and cannot notice the gaps.

And we humans seem terribly prone to constructing explanations for the gaps. It's not just the narrator who is unreliable, but the reader, especially within the conceit of the Laundry Series.

129:

It's remarkable how good speech recognition can be when backed up by Google's statistical analysis of language use. It makes good bets, and sometimes they are wildly wrong. It's closely related to their translation engine.

One example is how that translation engine makes the assumption that bishops are male. It is still mostly true, but there are enough exceptions to the rule to trip up machine translation.

Church of Sweden confuses machine translation algorithms. Some heads may have exploded.

If there were any high-velocity cranial excursions over that news, I could care less.

130:

It's an AI-complete problem.

Human level AI, surely? Otherwise human authors couldn't do it. (Unless you are all secretly weakly-godlike).

131:

They are much more bottom up then top down, self organizing and lack central command and control structures. In the environment these companies operate in its far far more nimble and innovative way of structuring. It's also very hard to explain how fundamentally different it is from the hierarchical systems

Yes, that's what the investors and the owners really want people to believe. But somewhere, you might not know them, come across then nor meet them, there's somebody doing a bunch of really dull, boring, traditional management and reporting crap. ESPECIALLY if you have investors.

Sure, you don't design whole products and then build them and sell them, Steve Blank thankfully has helped kill that model. But the topics to work on in sprints don't spontaneously appear, something, somewhere has a vision and direction and has their balls on the line when it comes to making bad decisions about the overall business direction.

It might not be as anal as the MS structure, or as single vision as Apple or Facebook - but if you don't think it's there you're being seriously conned.

132:

Oh good grief yes.

There's a lot of talk, with ideas like the Sprint, about getting feedback from the customers but just who is the customer. Who can say, "this is crap"?

It doesn't absolutely have to be somebody such as thee and me. In some senses, when Charlie was working at the programming coalface, the customer who had control was the banking industry. If they didn't sign off on what he was working on, it could not be used. Other companies, I suspect it might be the venture capitalists who have funded the company. I could name one or two where the paying customers don't get a look in. Yes, there are problems finding out what people want, but I have seen signs of a programmer-led rush for the shiny, with poor judgement of whether it works, or how it affects the overall system.

133:
>> It's an AI-complete problem. Human level AI, surely? Otherwise human authors couldn't do it. (Unless you are all secretly weakly-godlike).

That's what AI-complete means - human level intelligence and consciousness ( http://en.wikipedia.org/wiki/AI-complete )

It's a term that's been around a lot longer than all this singularity weakly-godlike entity nonsense ;-)

Strong AI is another one that was around long before the singularity stuff. The way I originally encountered it in the 80's was among cognitive philosophy folk who were separated into Weak AI (computers can exhibit intelligent behaviour - but not true human level consciousness) and Strong AI (computers can exhibit human levels of intelligence and consciousness).

For most people in the AI field that's still the way it's used.

134:

I've actually read Turing's original paper on 'The Imitation Game' and it's subtler and cleverer than the bastardised versions that are usually discussed. The point is to fool the interlocutor by being sophisticated enough to imitate, to act a role, that the AI is not. It sets the bar higher than just making convincing smalltalk. The AI, even though it is not a human, and does not have human experiences or emotions, should be able to imagine convincing responses. No doubt Turing's experience as a gay man in a homophobic society enabled him to frame the test in this way.

135:

I wonder if there's a viable business model in tacking crowd-funding onto the existing publisher set up? I.e. if the author has an idea that the publisher isn't sure will be a commercial success, they can post some rough-draft sample chapters and ask for up-front subs to make it a reality. If Charlie's publishers ever did this, and he punted his diesel-punk not-Fascist-honest-guv British Empire novel through it, I, for one, would put in as much as I could get away with without being divorced.

136:

I sent this blog post to my indie publisher and that resulted in an excellent long chat about what he termed the 'Franken-publishing' of my indie novel out May 3rd.

I hired my own professional developmental & copy editors, graphic designers & photographers, video & music professionals (book trailer), web designers, PR consultants, and drew upon skilled volunteers for proofreaders. My publisher is providing quality control from years of bookstore ownership experience and giving me guidance with printing requirements, online formats & selling, and promotion. Everything else is up to me to make happen.

It's expensive and I'm constantly aware of my level of privilege to even be able to do this. I'm also aware of the level of privilege to have had the time to write the novel in the first place. We joke about my position in the center of a small economy, but I'm extremely proud of being able to pay my team what they are worth. It makes my dayjob (SysAdmin on call 24x7) worthwhile.

I would like to be writing more than I am while staying in the production loop, but the creative freedom to 'keep it local' is more than making up for it. The grey area between self publishing and indie publishing seems to be about recreating all of the steps of quality as the initial blog post lists. I want my readers to have the best experience of the text and not be distracted by mistakes or low quality speed bumps. Who knows if it is going to go beyond my network, but the joy in the scheduled book release party is shared by a community. That's pretty fantastic.

137:

Not to mention details obscured or plain lied about for the sake of security. I'm thinking of details of location of various Laundry facilities which Bob may be deliberately lying about, just in case the memoirs fall into the wrong hands.

138:

However if you talk about just the present and say "things have changed alot in the last X hundred years, that up until now the rate of change has been accelerating" that is a statement of fact.

Actually I'd challenge this "statement of fact". A discussion of another thread perhaps, but I am remminded of an editorial in Analog at the turn of the century. From memory the question pose by the editor was:

"Would would have seen the most change in their day to day life: a person born in 1925 living to 2000; or somebody born in 1850 living to 1925?"

To that I'd add the case of somebody born in 1775 living to 1850.

Those of us born in the back end of the 20th century may have a slightly biased view and would be unaware of or ignoring some massive innovation and change that happening in the 18th & 19th centuries.

139:

Unless you include the "ubiquitous internet", I'd say anyone born after about 1965 was born after the last game-changer in the form of "reasonably cheap trans-oceanic air travel".

140:

Would would have seen the most change in their day to day life: a person born in 1925 living to 2000; or somebody born in 1850 living to 1925?

I think my grandfather saw about as much change as anyone. He was born in 1885 on a rural farm. One room house. Water from the spring. You chopped wood for cooking and heat. That was the year the phone was invented. He was active till he was 92 in 1977 and died at 97 in 1982. More than a decade after they landed on the moon. Life certainly changed for him.

141:

I'll take OGH's word for it about self publishing. Just keep 'em coming. Email them to me as attachments if you like. I'll send you the cover price and let's cut out the middle man. I'll proofread it myself.

Regarding the recent thread diversion toward rate of change. You could make the argument that it's subjective. We consider recent changes the important kind because they are in the foreground, and thus loom larger. The question assumes change is a single quantity. There are different kinds, and in different places. Certainly any given modern span of time in most of the developed world has seen a lot of new technologies, but someone born in France in the 1700s or Germany in 1600 would have considered a lot of changes to be happening.

But I think the modern era really has seen changes in the rate of change, things that are objective, not subjective. Probably an S curve like was discussed in a previous thread. It bootstraps. Jet travel means economic development means more patents, means teleport booths, means more economic development...

142:

Two months? Two months?

I can get a book out, once it has been to beta readers, in a week or two. That includes formatting the ebook, formatting the print book, designing the cover and loading to Amazon's KDP. I do my own proofreading (and before you scoff, check out my samples). If I felt unable to tackle these tasks, I could hire others to do them quickly for a reasonable fee.

Publishers, I have gradually realized, aren't much good at their job. Being used to the monopoly of distribution they enjoyed until very recently, they have become slack and inefficient. They are only just waking up to the possibilities of digital, and blame Amazon for missed opportunities and their own deficiencies.

How fortunate they are that plenty of writers still believe they need them.

Lexi Revellian

143:

You didn't actually read a word I wrote, did you?

144:

Oh, I think s/he read it. I suspect it's total comprehension failure that we see here.

Specials

Merchandise

About this Entry

This page contains a single entry by Charlie Stross published on March 21, 2013 12:35 PM.

Things publishers can't do (yet) was the previous entry in this blog.

The permanent revolution is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Search this blog

Propaganda