Home » Author Earnings, PG's Thoughts » Digital Arachnid: What Does Author Earnings Say to the Industry?

Digital Arachnid: What Does Author Earnings Say to the Industry?

9 February 2016

From Publishing Perspectives:

In the new February Author Earnings report, released Monday (February 8), things continue to look rosy for self-publishing authors and dire for the trade. But there are also announcements of changes in the approach—not entirely clear changes, mind you but in some ways promising. We’ll look at a bit of that later in this article.

To grasp the industry-political context here, we must remember that many who are skeptical of the efficacy of the Author Earnings effort point to the fact that it is an agenda-driven exercise.

Frustrated with what they said was a skewed and pro-industry picture presented at the Digital Book World in 2014’s “What Authors Want” survey, Mssrs. Howey and Guy set out to find a way to demonstrate that self-publishing is a viable route to earnings potential for authors. The result is the Author Earnings assessments, which many in the self-publishing community have defended as proof that their pathway to publication can be as good or better, financially, as the standard trade publishing route. Those fans, again, will be chuffed.

. . . .

Here’s the news: Author Earnings asserts that on Amazon’s bestseller lists, indie self-published titles account for more than twice the number of Big Five titles.

“What has changed,” the report tells us, “is the degree to which Amazon’s overall Top 20 bestsellers, and even the overall Top 10, have come to be dominated by self-published titles from indie authors—nearly half of which were not priced at $0.99 but rather ‘full-priced’ sales at prices between $2.99 and $5.99.”

. . . .

The most interesting question for us at this juncture is just what the trade publishing management attending DBW will make of this. Can it be that the “legacy” industry is being outclassed so substantially by “indie published” authors—the self-published sector?

. . . .

This is the language of self-publishing as what some of its champions call the “shadow industry,” a creative corps that cares nothing for the customs and concerns of the industry, and yet seems never to tire of carping at the establishment. It’s always worth noting that even some of the most-honored self-publishing bestsellers have taken contracts when offered.

And as anyone familiar with negotiating basics knows, by framing its results in ways that call out “the other side”—in this case, traditional publishing—Author Earnings repeatedly has hobbled its own efforts to widen the discussion. Rather than simply present an interpretation of the market and let that interpretation speak for itself, the material is served on a bed of right and wrong. Eyes glaze over, chips remain on shoulders, collegial exchange seems hard to come by.

. . . .

On the way out the door, Data Guy stops to pop publishing with a towel for agency pricing, of course, which gives us those out-sized prices on trade ebooks and Amazon’s “This price set by the publisher” notes on the pertinent sales pages. It may well be true that the publishers are shooting themselves in the feet with agency pricing on ebooks, perhaps contributing to the slowing of growth in digital reading on some level.

. . . .

Some of the most highly placed operatives in the trade will privately tell you (they’ve told me) that they’ve spent two years scratching their heads over Author Earnings’ digital derring-do. It’s reputed to have several parts:

  1. Sales reported by authors are compared to rankings of various titles on Amazon.com;
  2. A “crawling” of Amazonian sales pages is accomplished (our spider man or woman at work) to “scrape data” on a single day for each report (January 10 in the newest one)—spider goes out, spider comes back with the scrapings;
  3. An extrapolation of the results is made from that date; and
  4. Inevitably, a web of upbeat news for fans of self-publishing is spun from the results.

It might all be searingly accurate, spot on, perfectly right. But we can’t tell that.

And, as I mentioned in some commentary on the last report of 2015 at The FutureBook, what’s probably needed is a full analysis by a completely independent, reputable, capable firm, a unit of Nielsen or Forrester or PWC, KPMG, Deloitte, somebody, anybody. Please!

. . . .

What I’m being told is that a group of more than a dozen authors now makes available to Author Earnings, in some form, its actual royalty statements. The approach thus takes out the second-hand nature of reported sales. This takes us, Data Guy says, past the realm of “volunteered data points” collected and on to more precise observations made “during the precise same time period.” Significantly, Data Guy says that rather than using single-day sales, “this model incorporates sales history and matches Amazon’s true algorithm far better.”

And that’s the crux of the change: it involves the formula Data Guy uses to infer what a sales ranking means, based on what he tells us is now better-sourced actual data on sales provided by authors.

Link to the rest at Publishing Perspectives and thanks to Dana for the tip.

So, we need an audit of Author Earnings by KPMG or some other extremely expensive auditing firm.

PG wonders how often KPMG audits Nielsen Bookscan. Nielsen claims to record 75% of all retail sales of books in the US and is the bible of Big Publishing, controlling the careers of 98% of all traditionally-published authors. How do we know that Nielsen’s numbers are accurate? How do we know that all the retailers are providing Nielsen with accurate numbers?

Author Earnings makes its data publicly available in downloadable form with each new report. Here’s a link to all the data backing up the latest report. It’s a spreadsheet that runs almost 200,000 lines.

All the quote marks surrounding “crawling,” “scraping data” and “spiders” in the OP are breath-taking admissions of technical ignorance. Crawling and scraping data is what Google and Yahoo and Bing and a zillion other web search engines have been doing approximately forever.

To spell it out for the English majors in Manhattan, millions of Google spiders crawl the web 24/7, scraping data about the content of websites around the world. Google started doing this in 1998. But, of course, Big Publishing takes the long view and 1998 won’t happen there for a few more years.

As far as the methodology that Author Earnings uses, in PG’s view, it’s brilliant. He immediately understood it when it was described in the the first Author Earnings report. It’s a version of what Google does applied to book rankings on Amazon. Spidering, crawling and scraping are involved in case you had any doubts.

If Big Publishing employed anyone who knew more about computers than Spell-Check, it could have done the same thing that Data Guy and Hugh Howey did with Author Earnings.

Since Big Publishing has much more detailed information about Amazon sales of the ebooks of many, many tradpub authors, it could have extrapolated a lot more information about ebook sales than Author Earnings did.

But that would have taken more effort than lobbying the Department of Justice to make Amazon stop doing bad.

Unsolicited advice to Data Guy for his presentation at Digital Book World:

  1. Don’t forget a trigger warning for Arithmophobes before you begin.
  2. Include a map on your first slide showing the location of the closest safe space with no numbers. You may want to show the map periodically during your presentation.
  3. No more than three numbers on any Powerpoint slide.
  4. No decimal points.
  5. Don’t forget to mention Excel for Dummies.

.

a1

Click to Tweet/Email/Share This Post

Author Earnings, PG's Thoughts

152 Comments to “Digital Arachnid: What Does Author Earnings Say to the Industry?”

  1. Somebody save us from those uncomfortable numbers!
    Somebody…?
    Anybody…?
    Bueller?

    Love the “negotiation tactics” line.
    Dunno, but I never read the Reports as part of any negotiation. Manifestos ala COMMON SENSE maybe, but not a negotiation ploy.

    Like, seriously, what could they be negotiating? A “gentlemen’s agreement” on ebook pricing?

    Oh, my own unsolicited advice to DataGuy: wear kevlar. Preferably under a Darth Vader costume.

    • Oh, my own unsolicited advice to DataGuy: wear kevlar. Preferably under a Darth Vader costume.

      I think he should wear a black eye-hole mask cut from the silk of his dead brother’s vest. That way, when he leaves, the audience members can turn to each other and ask, “Who was that masked man?” and we can segue into Rossini’s William Tell Overture. 🙂

    • @ Felix

      “Oh, my own unsolicited advice to DataGuy: wear kevlar. Preferably under a Darth Vader costume.”

      LOL. Or a Guy Fawkes–V For Vendetta mask. 🙂

  2. “…yet seems never to tire of carping at the establishment.”

    Maybe because the establishment never tires of cherry picking, looking down its nose at self-publishing, or alternatively, trying to pretend self-publishing is of no importance.

  3. Probably the best way to handle this entire issue is just to ignore the Big Five — in a few years they will have adapted to the new paradigm or be out of business. Either way, reality has a tendency to win in the long run.

    • That would be a viable strategy…
      …if they weren’t trying to enlist the political establishment into a Microsoft-style lynching.

      Edmund Burke said it best.

  4. All the quote marks surrounding “crawling,” “scraping data” and “spiders” in the OP are breath-taking admissions of technical ignorance. Crawling and scraping data is what Google and Yahoo and Bing and a zillion other web search engines have been doing approximately forever.

    This is precisely where they lost credibility with me. Hello Publishing Perspectives, welcome to the 21st century.

  5. Your snark is high and much appreciated. 🙂

    I’d love to see Big Publishing put up the money to have an expensive data firm do an independent study and find out just how much it would cost that Hugh Howey and Data Guy are doing for free.

    The head-in-the-sand syndrome is getting worse in BP.

  6. Smart Debut Author

    I think what Porter finds hardest to grasp is that Author Earnings isn’t talking to or “negotiating” with “The Industry”…

    They clearly don’t give a f*** about “The Industry.”

    They are talking to authors…

    • ^this

    • Publishers: How dare you talk to us like that!

      Author Earnings: We weren’t talking to you.

      Publishers: How dare you talk about us like that!

      Author Earnings: We weren’t talking about you.

      Publishers: How dare you talk to authors about things that don’t concern us!

      Author Earnings: …

      • Yeah, it’s a total paradigm shift.

        The modern publishing industry has become like a shiny new maker faire — so many brand new excited readers and writers mingling, sharing their joy in finally being able to read, and write, all this amazing and cool new stuff.

        And then we’ve got the traditional industry middlemen, who keep trying to get the spotlight back onto themselves, like some once-lionized but now doddering old uncle insisting that all this newfangled stuff is nonsense — after all, his bosses at the widget factory once made someone “Employee of the Month” and now they drive a big, shiny Cadillac.

        It’s funny, but a little sad, too, eh? I find the psychology of it absolutely fascinating.

  7. “But, of course, Big Publishing takes the long view and 1998 won’t happen there for a few more years.”

    you made me snort my beer up my nose!

    • You are very brave if you attempted to read TPV with a beer in your hand, and were actually drinking as you read.

      It is a food-and-drink free zone. Our gracious host calls them as he sees them.

      Nourish and hydrate away from the screen; then again after you have been stuck here for a while. Never during.

      • @ Alicia

        But it’s a good way to clean the keyboard and monitor. Black coffee is my cleaner of choice. Anything with sugar in it will leave the computer sticky or gooey.

        Water works, too. But it’s a very boring drink.

        Of course, a paper towel is also needed to dry everything.

    • This one totally made me laugh out loud. Not just Lol inside my head. 🙂

  8. To grasp the industry-political context here, we must remember that many who are skeptical of the efficacy of the Author Earnings effort point to the fact that it is an agenda-driven exercise.

    Nothing wrong with agendas. AE conveniently provides all the raw data to drive anyone’s chosen agenda.

    In some circles, it’s a badge of honor to be have a poor understanding of math. These folks encourage and support of each other. The Author Earnings report may be the first time they realize how deficient they are compared to high school kids in Topeka, Kansas..

  9. AE’s fatal flaw is so obvious and egregious that industry people conclude either DG was out sick the day they taught thinking, or it’s feel-good propaganda for sad folks. Consensus is the latter. Then they forget all about it. Life is too short.

    On a personal level, I’m truly intrigued … don’t you see it?

    • I too must have been out sick when they taught logic as well. Will you do us the honor of pointing out the flaw(s) so we might all see? (And possibly ‘fix/correct for’ it?)

    • Oh, please share, fine sir. I don’t see it. Oddly enough, I have been making my living doing this sort of analysis for 20+ years. What is the fatal flaw?

      • Please confirm – you really don’t see it? Or is this just partisan argument? If the first, we’ll discuss. If the second, I don’t want to waste PG’s bandwidth.

        • I really have no idea what you are talking about. I have run a number of cross-validation experiments against the older AE figures. DG has a made an impressive improvement with this latest estimates. What is the flaw you see?

      • @ William

        There you go, flaunting mathematical/statistical expertise. Don’t you realize the OP is ignoring math? Probably even WhaleMath™. 🙂

    • He’s using real math to explain his results, not publishing industry accounting math?

    • I’ll admit to being confused why the major eBook sellers(Amazon, Apple, Kobo, and Google) are saying that eBook sales are going up while publishers are reporting sales going down or flat. Or maybe all 4 are lying.

    • Can you point out the obvious and egregious flaw? Tell us what industry people know.

  10. PG, you continue to contribute the best commentary.

  11. But there are also announcements of changes in the approach—not entirely clear changes, mind you but in some ways promising.

    You know, I get that it’s complex. It’s not like this is easy to understand, or even dive into. There’s a lot here, and it can be easily intimidating.

    But if there’s one thing you can’t ding Hugh and DG on, and even only one, you can’t fault them for lack of transparency. It’s all there. It’s all open.

    If anything about it is “not entirely clear,” that’s most likely because the user doesn’t understand it. Not because it’s not in the data.

  12. OK, briefly:

    Sales of BPH digital product through the Amazon channel is now hugely, massively, overwhelmingly focused on the pre-order model. Each week, one or two big brand authors release on a Tuesday. These authors have huge fanbases eagerly awaiting the release, goosed by heavy promotion over the previous three or four months.

    At midnight on Tuesday, Amazon books the sales, charges the credit cards, and downloads the product. Those author’s e-books are immediately #1, because of the immediate sky-high spike. The spike is a truly immense dynamic pulse through the system.

    AE avoids scraping on Tuesdays (either slyly or unwittingly) – and even if it did, it would interpret the #1 position as a sales figure of 7,000 for the day (under the old system) or 8000 – 9000 for the day under the recent revision. Whereas, 60 or 70 times a year, that Tuesday sales figure can be anywhere between 200,000 and 700,000 units for the day, and on occasion approaches or exceeds a million.

    Thus AE undercounts BPH tentpole releases by one or two orders of magnitude. Why? Possibly because the self-pub sales model it crowdsources its ranking-to-numbers conversion from exhibits steady-state behavior rather than large dynamic pulses.

    • All of which has absolutely no relevance to the earnings, actual or potential, of anyone except that small number of tentpole authors.

      • Tom, relax.

        • Mr. Child, I am relaxed. You are the one making hysterical claims against the accuracy and veracity of the Author Earnings reports.

          • I’m sure AE is well-intentioned, so questions about veracity would be loaded. It is, however, obviously inaccurate. Put it this way: I challenge it to use its so-called methodology to tell me how many e-books of my latest frontlist title I sold. If the number is right, I’ll buy Hugh Howey another boat.

            • This isn’t about you.

              • It’s about all of us, and reality serves us all equally well.

                • You aren’t, and probably never will be, in a position where you have to consider making a choice between Big Five publishing and going it alone. How much money you are currently making by being a tentpole author for a Big Five imprint is radically irrelevant to those writers who do have to make such a decision.

                • The data being used for these has been available since the first report, and if there was something truly wrong with the methodology and data, not only would it have been found by now, traditional publishing would be saying so loudly, and often, and would have put out their own reports saying so. Publishing has some very smart and bright people working for it, if these reports were wrong, we would have heard something by now.
                  But we haven’t. Why?
                  Is it not worth their time, effort, or money to do so? I get the impression that Traditional Publishing thinks that if it ignores the reports, they’ll just go away…and seeing as how well that attitude has worked for them in the past….

                  Your assuming that everyone occupies the same reality, when it can be shown that several groups, Authors United, Traditional Publishing, the Authors Guild, Indie writers, all have their ideas on what the “Reality” of the current publishing world is. Can they all be right?

                • if there was something truly wrong with the methodology and data, not only would it have been found by now, traditional publishing would be saying so loudly, and often

                  Phoenix Sullivan did a beautiful job of pointing out several problems with the methodology. Her article was excerpted on TPV last May:

                  http://www.thepassivevoice.com/2015/05/digging-deeper-into-author-earnings/

                  I think the changes introduced in this new report are improvements based on the points raised in that article.

                  I didn’t know about this Tuesday spike. I hope Data Guy finds a way to take it into account.

                  Publishing has some very smart and bright people working for it, if these reports were wrong, we would have heard something by now. But we haven’t. Why?

                  Because the people who blog about this and understand the math are outnumbered by the people who blog about this without understanding the math. So most of the arguments against the methodology are irrelevant.

                  This Tuesday spike seems relevant to me.

                • This Tuesday spike seems relevant to me.

                  Since others have reported that preorders count towards the sales rankings at the time of the order and not at the time of fulfilment, it seems to me that the existence of a Tuesday spike has not been demonstrated.

            • I haven’t read through all the replies yet, so maybe this has already been addressed. I don’t know what day AE drew their analysis from, but if it was a Wednesday, it’s probably the most accurate. Amazon’s rankings lag by up to 12 hours from what I’ve been seeing lately.

              Also, any book that has a Bookbub ad will see a significant spike as well, so an indie title with a Bookbub ad on a Tuesday could also jump well up in the rankings–especially if they stacked promos on various sites at the same time, such as FreeBooksy, Robin Reads, etc.

    • Hmm, if it’s that big a ‘pulse’ that Data Guy is missing, Shouldn’t the publishers be doing better than they’re claiming for ebooks? Or is it despite these ‘pulses’ all the publishers’ other ebooks are doing so poorly that even the Tuesday ‘pulse’ can’t bring their numbers back to life?

      52 Tuesdays in a year, a couple ‘boomers’ from each of the big boys each and every Tuesday, a few hundred titles not seen at their tricked out Tuesday ‘pulse’. “With data on hundreds of thousands of titles” dug up by Data Guy, those few hundred become a rounding error in the big picture, ‘pulse’ or no pulse …

      Something just doesn’t seem to be adding up …

      • Circular argument – it’s AE that says publishers aren’t doing well with e-books. In-house, publishers have re-balanced e-books, higher prices, and print sales very nicely. Everyone’s happy. I made out like a bandit in 2015.

        • No, it’s the industry itself that is claiming ebook sales are down. But if you want to spin that decline as ‘rebalancing’, well, that’s your privilege.

        • Actually it’s publishers that said eBooks are shrinking or flat. http://publishers.org/news/aap-statshot-publisher-net-revenue-book-sales-declines-20-through-third-quarter-2015

          •eBooks were down 11.1%, with most of the decline coming from Children/YA books (44.8%).

          There are other statements out there with a quick search.

          • Sure, but you make it sound like a bad thing.

            • All my book reading is done on a Kindle. If I see something from a favorite author, I will buy the ebook at a highway-robbery price. I can justify doing so because I’m such a slow reader, I get many hours of entertainment from that purchase. However, I would guess that there are others who, like me, read only ebooks — but who, unlike me, are unwilling to pay the high prices set by big publishers.

              I wouldn’t even take a free copy of a print book by an author whose work I like. I left paper books behind in 2009 and have never looked back. There are probably others who feel likewise about paper books — but who, unlike me, can’t justify purchasing high-priced ebooks from big publishers. If my theory is correct, authors are losing sales that could have been made if the price were lower. Thus, I don’t agree with you that “everyone” is happy. Those on a budget who read only ebooks certainly aren’t.

              Publishers and their authors could be far happier, economically, if publishers stopped keeping prices artificially high rather than opting for high volume sales at a reasonable price. I know of one author (international best seller) whose ebook sales are down significantly under the current pricing structure, but her publisher is blaming her. Put her in the unhappy column, too.

              • Like you, I read exclusively on my kindle. But I do read a lot, and I refuse to buy at highway-robbery prices. Those favorite authors of my past have become exactly that – of my past. I’ve found new authors to replace them, with prices I can afford. I don’t even bother looking at books from authors I used to consider “must buy.” For a while that hurt, but I don’t miss them now as I’ve found their replacements.

                • And that right there is the number one way trad-pub is killing themselves and the writers that signed up with them.

                  Even if(when) they finally drop their ebook prices, how many readers will have forgotten all about that writer they used to follow before they got priced out of the market?

        • That’s not what I said. The publishers themselves are reporting ebook sales are down. Not Data Guy, the publishers.

          I like the “Whereas, 60 or 70 times a year, that Tuesday sales figure can be anywhere between 200,000 and 700,000 units for the day, and on occasion approaches or exceeds a million.”

          60 or 70 Tuesdays in a year? there’s that funny math showing up early.

          But let’s use it to your advantage. So 60-70 times a year Data guy misses ‘pulses’ of “between 200,000 and 700,000 units for the day” So that’s 12,000,000 (60*200k) to 49 million (70*700k) missed ‘pulsed’ ebook sales — have I got that right?

          This out of the five big boys that together reported only selling 47.9 million ebooks in all of last year?

          Sorry, but maybe you’d best go confirm with your publisher how many ‘pulses’ they actually get — and please? Ask one of the accountants and not one of those jokers in sales?

          Still wishing for real numbers to play with …

          • AE ain’t the place for real numbers. Even they admit they “extrapolate” them.

          • He didn’t say there were that many Tuesdays in a year. He said 1-2 new releases by Big Pubbed authors on Tuesdays, which could result in 60-70 “pulses” during a year.

            • Either way, his ‘pulses’ supposedly account for a quarter to ‘all’ of the big5 ebook sales — I’d think Data guy would have noticed his numbers being that far off because the rest of it wouldn’t have added up for him.

              So, we need better numbers — or tartar sauce …

              • I’m far better with the numbers I personally crunch, but based on linked report about ebook sales being down for publishers the first 3 quarters of 2015, and Mr. Child’s statements…

                Wouldn’t that mean the Big 5’s numbers would be lower on the days AE spiders run, thus show all others as having larger percentages?

                (I’m really tired and dealing with kind of major back pain right now, so if that’s a stupid question, sorry.)

        • Lee, you totally miss the point of AE. Which is fine, because you aren’t the kind of writer it is trying to provide information to.

          First, if you’re correct, why do AE’s numbers fill in the blank between ebook sales for the big publishers going down (as has been widely reported) and Amazon saying ebook sales are up. AE has the answer, indies make the difference. So even if you’re on to something, your theory doesn’t explain what is really going in on publishing, particularly in indy publishing. Indy writers would like to know if it’s true ebooks sales are slowing or not. And guess what, it’s not true.

          Second, you’re not providing any actual data. You’re simply saying there’s a lot of stuff AE isn’t including that somehow proves big publishers are selling lots more ebooks (which even they don’t claim). So your theory is useless to anyone trying to get real answers unless you back it up with facts. Why don’t you provide us with some real data and information (like your own Amazon sales by date) and, like AE, show us exactly how you got it? Otherwise, we have to assume, at best, you’re guessing or exaggerating.

          Third, no one really cares about your theory because we don’t really care how big publishing is doing. Yes, its fun to talk about how big publishers shot themselves in the foot, but the real data indy writers are interested in is how are indy writers doing. AE answers that as best as it can with real data, rather than theories and mysterious data that you won’t reveal.

          • Mackay – AE is partly about how indies are doing, but not purely. It makes gleeful and inaccurate comparisons to trad publishing, which doesn’t help anyone make an informed decision. In fact it tempts them to make an uninformed, or misinformed, decision. That can’t be good.

            Bottom line: for most, indie is the way to go. But if you want to aim high, I think it doesn’t help for the height of the potential ceiling to be obscured by agenda-driven misinformation.

            • Our comparisons to trad publishing aren’t dire enough, in my opinion. We are comparing all of self-published authors to the 0.1% of authors who queried an agent and managed to get their book published. 99.9% of manuscripts never get turned into books along the trad-pub path, which means they earn 0 dollars and probably give up on the craft.

              The real injustice that leads to true harm is when trad-pub proponents hold people like you up as the standard for trad publishing. “If you work hard enough, and do everything right, you could be the next JK Rowling!” You insinuate the same thing in this very thread by saying that self-pub is right for most authors, but they won’t have the chance to top out where you’ve landed. Selling that dream, and leading authors down a miserable path where it’ll be 2-3 years before their art hits the market, and they’ll be paid a miserable rate every six months, and they’ll be limited to one novel a year, is just horrible professional advice. Cruel, even.

              I don’t think you mean to be cruel. You’re just giving advice about a system that has changed since you went through it. The system is changing so fast that when I give accounts of my own success, I caution authors that the landscape is different. Back in my day (7 years ago!), you could stand out with a $2.99 novel. There was less competition. Shoddy cover art didn’t hurt you as much. The KOLL freebie days were supercharged, and perma-free was pure magic. All of that has changed.

              Most of the advice I give these days is from what I’m learning from the new bestsellers and those grinding out a living. The Liliana Hart techniques, the Eamon Ambrose and Samuel Peralta stories. It’s constantly changing, man.

              And look at the last sentence in your comment above. “agenda-driven misinformation.” The irony, man. That’s exactly what you brought to this conversation. AE’s data is BETTER than the Big 5’s data, even if they were to combine it all. It’s BETTER than any other retailer’s data. It’s ALMOST as good as Amazon’s data. Combined with DG’s runs on B&N and some overseas retailers, and his work with some trad-pub experts on what their reporting has to say, I know for a fact that Data Guy has the best vantage point on this industry as anyone in the industry. The AE data is accurate, especially the pie chart percentages. It’s just not the truth you want to hear, and I get that. Please just be open-minded not only to being wrong about this report, but being wrong in general. Because you’re in a sector of the industry that’s making a habit of getting things wrong. Not a great place to learn. And you seem like someone who enjoys learning and knowing things.

      • Allen – further to your point: agreed, data on hundreds of thousands of titles is an impressive achievement, but as DG would agree, those sales are dwarfed by the uncounted 30m-or-so tentpole sales his system is designed to ignore, in order to make y’all feel good.

        • 30 million or more? So all other ebooks by the big boys accounted for less than 17 million ebook sales? Wow, just wow for all those other guys/gals that gave their books to those publishers …

          • That’s the nature of exponential curves. Self-pub has them too. Amazon just reported that fewer than forty self-pubbed authors – across all their titles, across the last five long years – have put together a million sales.

            • I’m at over 61k in just over 2 years (2 years, 1 month, 8 days to be exact).

              Should be closing in on 1 million by the end of this year, if I see the same increase this year as there was between 2014 and 2015.

            • Gosh, five long long years. A new market for self-published ebooks went from about zero to $500 million a year and it’s been so long. What, only forty people have sold a million books so far? What’s taking so long?

              And only one self-published book has been produced into an Oscar nominated film for best picture. What’s taking so long? It’s been five years.

            • I can almost bet you haven’t ever heard of me. I haven’t sold one million books, but I’ve sold half a million in 1.5 years. I can guarantee there are more than 40 authors who’ve sold a million books. I know more than that personally.

              • Lee is talking about 1 million+ bestsellers on Amazon. There are authors who have sold very well on other outlets whose sales aren’t counted. And Amazon is typically just reporting on Kindle sales. The authors selling units in the six figures are also selling a lot of print, audio, and foreign translations. So the number of 1 million sellers is undoubtedly in the hundreds already. Which is insane.

                Even the 40 – 50 Amazon-only million club is crazy when you think about it. The market is only five or so years old. That’s ten or so authors a year hitting the million club (averaged out over time). Trad-publishing would love that kind of run of successes.

                More craziness: Amazon paid out over $140,000,000 to KU participants last year. That’s insane money. All going to artists instead of paying NY and London rent.

        • Lee: I left a long reply to your comment (and to DG’s) below.

        • Why do I feel like I’ve just been condescended to by Lee Child just because I’m an Indie? I feel all dirty now.

          “In order to make y’all feel good.”

          Nice.

    • Preorders are actually kind of interesting — well worth digging into. Their impact on Amazon rankings doesn’t quite work the way you think it does, though.

      Preorders are counted in the ranking at time of order, rather than all affecting the ranking at launch. Observe the rankings of any BPH megaseller during it’s pre-order period, and you’ll see what I mean: they slowly rise through the rankings, weeks or even months in advance.

      Ranking changes correspond to when the author actually “earns” that money from a customer… 😉 … rather than when Amazon, or the publisher or author themself, gets paid.

      But it’s kind of immaterial, because the launch-day sales spike alone on BPH’s biggest hits will usually be enough to hit #1.

      See also: books where the movie version is opening in theaters… same effect. They definitely sell a lot more than 7,000 on those days.

      For the sake of argument, let’s go with your high-side estimates of 70 times a year x 700,000 = 49M Amazon units our model misses.

      If you feel strongly that’s an important consideration to an author choosing which way to publish, feel free to add 15% to the BPH total in our pie charts and call it a day.

      Lee, if you’re game, let’s do an experiment. 🙂

      Shoot me an email and we’ll coordinate a time period. I’ll scrape Amazon rankings for your books and run them through our algorithm. And you gather your Amazon US sales info, and we’ll privately compare notes.

      I think we’ll both learn something. Win-win.

      The offer’s open.

      And hey, if you’re at DBW, stop by and say hello.

      • If I’m there, I will. Amazon disagrees with you about how pre-orders work, though. We’ll compare notes on that too.

        • Actually, you have it completely backwards on how pre-orders work. They count every day, as pre-orders roll in, and there is no launch-day boost in rankings. Not on Amazon.com.

          Apple’s iBookstore does both: It gives the author a ranking boost for every sale during the pre-order period, and then ALSO counts all those pre-orders a second time, in one big lump, on the day of release.

          Amazon does not do this, even though I have urged them to consider it. I think new releases should get a big bump on the day the book comes out, to increase visibility. Right now, Amazon only counts those pre-orders once, and only as they are rolling in.

          I know, it’s confusing, because each retailer does it a different way. But I promise you that you have this wrong. And if this is really the big problem you have with AE, it’s a very strange one. Not only is it wrong, it wouldn’t matter if you were right. The Big 5 are not only reporting declining ebook sales, but we predicted the amount of decline last year right before publishers began announcing them. Our reports are that accurate: They have predictive ability, not just post hoc ability.

          Publishers have been celebrating declining ebook growth. We know from Amazon that ebook sales overall are on the rise. AE shows why. (There was an interesting exchange at LBF last year where a Big 5 CEO said, smugly, “our ebook sales are down” to an Amazon VP, who replied, just as smugly, “ours aren’t.”).

          Also: Making this about you (or self-pub authors like myself) is the exact opposite of what AE is meant for. Traditional publishing is death to midlist authors. It’s a lottery, and if you don’t win, you can’t even pay your bills. You are fired. Self-publishing has made it possible for the non-lottery winners to earn a steady living. This is the story that was never being told for years, as media outlets focused on the outliers. The real story is what’s happening outside the top 100. And for me, the best part of this story is that more authors are now sticking with it, not giving up, growing their craft, and producing more art. The old system discouraged artists. This one encourages them while also supporting them.

        • Here’s a KDP author thread about this issue, pretty clearly demonstrating that Lee is wrong about how preorders hit the rankings. https://kdp.amazon.com/community/thread.jspa?threadID=215111

          Which is pretty funny after all the “you REALLY don’t see what AE is missing?” back and forth above.

          • His confidence was inversely proportional to his correctness.

            But like I said elsewhere, it’s not his fault. He’s got people that he trusts who don’t know the first thing about their own business, because it has changed quicker than their minds will. And they are telling him stuff that he assumes to be true, because in the past these people knew what they were talking about.

            The great thing is that Lee is here in the trenches from time to time, so he’s exposed to glimpses of the new reality in the publishing landscape. I can see one of his writing pals saying something like, “This self-publishing thing is dumb, right?” And Lee saying, “Actually, for most authors, it’s probably a good idea. It just limits their upper end earning power.”

            With enough time, Lee will learn that even this is wrong. Andy Weir made bank last year, and he started in self-pub. Same with the 50 Shades of Grey phenomenon. That’s just two crazy examples in five years. Stay tuned. Let’s see how many more Lee Childs come out of the query-go-round, and let’s see how many get started with self-pub.

            My wager to Lee is this: In 20 years, if you look at the top 1% of authors who debuted in that timespan, 2/3rds of them will have gotten their start in self-publishing. Only 1/3 will have come from the query pile. And I think this is a conservative estimate. It might be much higher than this. I’d bet Lee my boat (it’ll be worth about $1,000 in 20 years).

            Lee is still holding up as examples of what traditional publishing can do, the authors who came along before they had to compete with self-publishing. In my genre, there hasn’t been a new author in the last seven years who holds a candle to the top 10 self-pub sci-fi authors. One of the top earners, Scalzi, makes a lot less than those 10 self-pub authors. And he got his start back in the day.

            What crime/thriller authors have taken off in the last seven years to reach Child or Patterson levels? That era is over. The ability to create blockbusters with marketing muscle, or by investing in authors for year after year, is over. Now it’s coloring books, picking up self-pub hits, political memoirs, movie tie-ins, and taking advantage of old ladies as they near death.

            Doesn’t bode well.

            • Agreed. And I particularly take your point that while Lee doesn’t necessarily understand this landscape, it is likely not his fault, and in any case, it’s laudable that he wades into the debate; and that he’s willing to acknowledge that for many or most authors, indie is the way to go. Being wrong is no crime, and being willing to engage is always a good thing.

      • Correct. Pre-orders are counted toward ranking at the time of pre-order, not on release day.

        • I’ve seen people running pre-orders surprised to realize their books are already being ranked before release day, so I’ll third that that’s the case.

        • Well, I suppose if you don’t know this, that makes it easy for a trad publisher to lie to you. Even if you’re a big name like Lee Child.

          • Usually it is safer to bet ignorance than malice. BigPub has a lot of tendency to stay ignorant.

            • “Never attribute to malice what you can reasonably attribute to stupidity.”–Raymond E. Feist, in a series on publishing contracts written for the SFFWA Bulletin.

    • I’m pretty sure I remember AE mentioning they couldn’t report directly on pre-orders(or advances which aren’t expected to earn out) right from the start. That problem isn’t as important as it was though. Why?

      As you mention, new big author releases are ongoing for years now, then while it throws off calculations it’s less a factor as time goes by. Because?

      Either they are making a splash and dropping down the charts fast, OR, there are less big authors/less releases hitting those charts if the number of traditional books in the top 100 is dropping.

      Because, otherwise those authors would still be dominating Amazon’s lists for a long time. Amazon’s lists don’t just consider same day sales, they factor in sales over time and a big initial spike plus decent follow up sales should keep an author at the top for a while.

      Also, publishing sources and Neilson have reported eBook sales by traditional publishers are flat or declining.

      The question then is who is taking the brunt of that? If big authors like you (you mention below 2015 was a great year) aren’t, then it’s the midlist authors.

      Considering midlist authors get a better percent on eBooks, then that hurts them.

    • Whereas, 60 or 70 times a year, that Tuesday sales figure can be anywhere between 200,000 and 700,000 units for the day, and on occasion approaches or exceeds a million.

      Interesting! Now I’m hoping that Data Guy will run the AE spider on a few Tuesdays and incorporate that data with the non-Tuesday data to give us a more complete picture.

      Thank you for sharing your insider information, Lee. I believe you alluded to it in your comment to a post a few days ago. You piqued my curiosity then. Glad to have that curiosity satisfied now.

    • Ok, I am game. Let’s see if we can agree on a way to test your assertion. First of all, this massive boost wouldn’t show up whether or not DG spidered on Tuesday. Your scenario is one the few real issues that could be raised with the AE methodology. If the top end of the sales ranks is regularly represents substantially more sales than normal, the calculations could be very off. Your informants are clever. The question I have is: Are they playing you?

      So, how do we test this? I can think of a number of tests. This is the best:

      The AAP regularly post ebooks receipts from their members. We have a very good idea of what percentage of those sales are represented by the Big 5. We could estimate the monthly sales based on AE’s numbers and compare to the official reports. If your theory is correct, AE will be substantially underestimating the sales.

      Does this seem reasonable?

      • I tried to give the Tuesday argument the benefit of the doubt. But there is another problem with it, even if the Mega Seller rankings are adjusted on Tuesdays.

        The single digit rankings for one day won’t automatically disappear the next day. They are slowly amortized (for lack of a better word.) So if the spider crawls on Wednesday or Thursday, the effect of Tuesday’s (purported) spike would still show up in the rankings.

        I don’t believe the AE report tracks actual sales figures as that information wouldn’t be available on Amazon’s public web site.

        I believe the Tuesday argument has almost no merit.

        • Precisely. If we ran the report on a Thursday, any book that was given credit for 100,000 sales on Tuesday would be firmly lodged in that top spot. It wouldn’t come down for days.

          What really disappoints me about Lee’s theory is that he posits this as deliberate dishonesty on our part. As if we avoid Tuesdays on purpose. DG runs the reports on days he gets an itch in his crotch and has time to scratch it. There are no shenanigans here. In every way that we can tilt the odds in the favor of the Big 5, we do. We don’t mask our glee at the results (who doesn’t love seeing the people win out over mega corporations?), but we are diligent about controlling for our bias.

          We give Lee the benefit of the doubt when he is wrong. It saddens me that he doesn’t extend the same courtesy when he (mistakenly) believes that we are wrong. If we are, it’s not on purpose to deceive. Our goal from the beginning has been to shine light on data that Amazon hides in order to help authors of all stripes. I’m constantly fighting for better terms for traditionally published authors, and I blog about the results. I’ve gotten time-limited print-only deals, and I’ve told authors which publishers are willing to go this route and what sort of negotiations took place. I have yet to see a mega blockbuster trad-pub author do the same. Instead, they get these most-favored-nation clauses in their contracts that screw the little guy by not allowing publishers to offer more than 25% of net on ebooks. I’d wager anything that Lee has this clause in his contracts and doesn’t even know about it. I’d bet he doesn’t even understand how this hurts new authors. Not many people do. They just trust their publisher and their agent. I wish more cared. Some real good could be done for other authors.

          • You are way more gracious than I would be after being on the receiving end of such… an attitude.

            Reflects properly on both.

            • I’m sitting on my boat in the Bahamas, drinking a whiskey, and watching a great film on a big screen TV. How is someone on the internet going to put me in a bad mood? It’s just not possible.

          • It saddens me that he doesn’t extend the same courtesy when he (mistakenly) believes that we are wrong.

            Honestly, it’s been my impression that Lee does a little “poking of the anthill” here at TPV to amuse himself.

            I believe the first time he ever commented on TPV, he was hanging out at the airport, bored, waiting for his flight to board passengers. His first comment was quite rude to PG, and it seemed the furious defense by TPV commenters quite amused him.

            I can’t help wonder if a similar desire to “poke the anthill” colors some of his comments now. All pure speculation my part, of course, but these thoughts do cross my mind.

            That said, I respect Lee for continuing to engage here and for showing an interest in learning about the indie world. It is very cool that he comes straight out and says that the indie path is undoubtedly the best one for most writers. That’s awesome!

            • You also need to understand that Lee is part of the establishment. I doubt his paycheck has been effected by this as publishers won’t admit there’s a problem. Once it affects his personal bottom line (a smaller advance) you can be certain he’ll demand answers.

              Until that happens, he’ll still get millions per book with no hope of earning out based on royalties. If his paycheck goes from $12 Million to $9 Million due to shrinking sales, he’ll sit up and take notice. It’s just a matter of how long BP can deny reality and still keep their 1% authors happy.

              • Oh, I do understand that Lee inhabits the tradpub world and fully to his benefit.

                But if anyone else has noticed his “poke the anthill” tendency, they haven’t mentioned it, and I was tired of the silence on the subject. So I spoke up.

  13. Rolling up pre-orders to the launch window is what the NYT does but not Amazon.

    I seem to remember that the Amazon sales rankings are a rolling average of several days/weeks precisely to avoid gaming by single day reporting spikes.

    • All I know is that it takes at least 24 hours before a newly released (not pre-order) title that has sales receives a ranking, and that the ranking does fluctuate throughout each day.

    • Yes. That “pulse” exists thanks to marketing specifically for the launch day.

      But – Amazon sees that spike, and averages it into the ranking that already existed from pre-orders, and continues to average it into post-launch sales. (And, actually, it is not just a simple “window” average – they decay the effect of a spike, for whatever reason it occurred, over several weeks.)

      The thing to remember is that Amazon goes through all of the gyrations they do not just as a service to the authors – it is a service to their customers, and drives many of their algorithms for marketing to those same customers. They are as accurate as they can possibly get for the long-term appeal of authors – which translates to sales (what the author and Amazon are interested in, period).

      • If so, then the publishers would get a bigger/better ‘pulse’ by ‘not’ offering a pre-order button and having all the sales roll in once the book is actually out there.

        But then they risk ‘something special’ coming up and distracting the readers just as their ‘book’ is rolling out.

        Darn, so many choices on what to try to game out of the system.

  14. A key part of Lee Child’s scenario seems to be incorrect. I checked a few big name authors with upcoming releases available for preorder on Amazon. Nora Roberts has a Penguin published book coming out on Tuesday, April 12. Her book is already ranked 791 in the Kindle store. Here is the link:
    http://www.amazon.com/Obsession-Nora-Roberts-ebook/dp/B0152FFZYI/ref=sr_1_3?s=digital-text&ie=UTF8&qid=1455077221&sr=1-3&keywords=nora+roberts

    I also checked Rick Riordan’s newest and a couple of others. I really think that Lee Child’s inside information is simply incorrect. Amazon is counting the preorders towards these books sales rank at the time of the preorder, not at release. But I still want to do the comparison to the AAP numbers.

    • Then Data Guy’s numbers are closer than Lee Child and his insiders think. What’d that old war guy say about not disturbing the other team while they are making a mistake?

    • But I still want to do the comparison to the AAP numbers.

      The AAP numbers are a great calibration check — in fact, with some insider help we’ve already tagged pretty much all of the significant AAP-reporting publishers and imprints in our database, one by one. (800 or so of them, IIRC).

      The part that’s frustrating is having to wait 3-4 months until the AAP gets their own monthly StatShot numbers collated, so that we can compare them.

      But we’ll be able to do and share an AE-to-AAP comparison for these January numbers, sometime in May.

    • Yes, Lee is incorrect on this. It’s not his fault, though. He’s probably chatting with publishing peeps, and they know a lot less about how Amazon works than the average PG reader.

      I’ve got a hilarious email chain with my editors at S&S on why they should categorize WOOL in a SF subcategory. They couldn’t grasp that subcategories inherit parent categories, doubling the shelf exposure of the book. Simply couldn’t grasp the concept. And I like to think I’m a decent explainer. It took a dozen emails from me for them to get it, and another month or two for them to make the change.

      I’ve spent a lot of time working with major publishers, and they are frankly clueless on how this stuff works. I don’t say this to be mean. I want them to understand. A lot of authors got harmed in the move to agency because they don’t get what’s happening out there. And genre authors have been harmed for years by having their debuts priced like new non-fiction. It’s painful to watch how slowly they are figuring out things that were crowdsourced and understood six years ago on KBoards. You’ve got a startup mentality on one side, with tens of thousands of bright, energetic people working together and sharing their data and info — and on the other side, you’ve got bean counters with entrenched thinking. It’s no contest. To Lee’s credit, he’s here, and he’s engaging, so maybe he’ll learn some of these things and take the info back to his peeps and get them up to speed.

  15. Responses to Lee Child and Data Guy…

    Lee Child:

    Some of my favorite posts on TPV are from you, Data Guy, and Hugh Howey. When you posted here some time ago, while most people carried on an interesting conversation, I thought one or two people got out of hand and acted like total jerks toward you (so, if it happens again, just remember it’s only one or two out of 245 million worldwide TPV readers…also I’m just clowning around with you below, not intentionally trying to be a jerk). So, please keep the conversations going.

    In respect to pre-orders, I’m seeing that pre-orders are tracked daily, as ordered. Example: Your nemesis, James Patterson, has “15th Affair” set for release on May 2nd 2016. It’s sales rank today is #849. Also, I’d bet a Hugh Howey catamaran (well, not really) that JP’s gonna come out with a line of adult coloring books, so you might want to up your game, pal. By the way, a Jack Reacher coloring book is the only adult coloring book I’d ever buy.

    However, when you or JP or the next surprise-Harry Potter novel (or, as DG mentions, a movie tie-in) are sticking at #1, the AE model will use the same fixed number it always uses (7k or whatnot) and that will skew results. You can see this as a fault or as room for improvement/model tweaking. For one thing, AE could keep taking a monthly snapshot of ranks 101 and below and track ranks 1-100 on a daily, ongoing basis (without scraping much additional data), and then factor up for the mega-selling brand authors when they are in the 1-3 spots.

    While we can shoot arrows all we want at the AE model, it might be the best model out there. Whereas the other models probably won’t evolve, the AE model can get better and better. Outlier indie and trad authors can provide Data Guy with their personal data. And Data Guy can “adjust for outliers” which would be easy (assuming outlier participation), since, by def, there are only a few of these beasts in the wild. You probably aren’t going to give Data Guy your full sales reports, but you might want to throw him some useful crumbs.

    Below, I discuss how KU and Amazon-imprints further muddy the waters for AE and for all other industry reporting agencies.

    The best information is the private data only accessible by Amazon itself. So, ya work with what you can access–which goes for AE and all others. If you want to throw away the AE report, then you need to throw away all the other reports in the industry (except for the confidential internal available only to Amazon execs).

    Most importantly, I want you and Data Guy to stay friends forever. You guys have the coolest pen names of anyone I know (although neither of you schmucks made the list of famous pen names on Wikipedia; oh well, better luck next time).

    https://en.wikipedia.org/wiki/List_of_pen_names

    Data Guy:

    I see the biggest problem areas with the AE model being 1) Kindle Unlimited and 2) Amazon-imprints (of course, this is an even bigger problem with all other publishing reporting entities as they choose to simply ignore these rapid-growth areas). In respect to KU, I believe you might have addressed this somewhere previously, but how are you currently addressing KU in respect to conversion to $ earnings and unit sales?

    In respect to Amazon-imprints (which is THE rocket ship on your current charts), how are you addressing the “Kindle First” month-1 releases? (I’ve never seen you address this, but it seems to be the elephant on the ceiling of the room.) The 4 to 6 Kindle First books hog the top 3 to 6 spots on the best seller list for the better part of every month. However, it would be incorrect to count these as either unit sales or $ sales. They are “free” with Amazon Prime memberships. So, they should either be counted as free books ($0 sales, 0 unit sales, and purged from the AE best seller lists) or they should be counted as sub-KU books (because you are not buying them, if you are in Prime, but paying a monthly/annual membership, first for 2-day shipping, second, for free streaming movies, third for free streaming music, and fourth for one free newbie author book per month). In respect to the earnings of Kindle First authors, the month-1 earnings may be approximately $0 as per one KF author had revealed in a podcast interview. Also, the pages-read-per-Kindle-First book (where Prime members must choose from a mere 6 books offered) would be far less than the pages read per KU book (where you have millions of books to choose from). To make the waters even muddier, with the KF books controlling the top spots on the charts from “free-ish” downloads, some non-Prime readers see these and pre-order them (for true $ and unit sales). Of course, Kindle First Amazon-imprint sales after month-1 become actual unit and $ sales (but maybe half the standard kdp royalty going to the author).

    Also, since I hopefully have your ear here, are KU page reads factored into Amazon’s sales rank algos?

    People can shoot arrows at AE all they want, but AE is the most transparent reporting model out there and the only one that seems to be constantly improving and accounting for rapidly-changing market conditions. As more mega authors share their personal reports with AE, the AE model can better track the secret Amazon internal data.

    And one last thing, DG: What are you wearing when you give your speech at DBW? Just a custom spidey mask, or the whole bodysuit?

    😛

    • Good questions, BD.

      RE: how AE handles KU:

      http://authorearnings.com/calculating-indie-author-earnings-under-ku-2-0/

      Probably needs a slight update now.

      RE: Amazon Imprints & KindleFirst:

      Phoenix pointed this out a while back. I have it on good authority that those authors do get paid during their KindleFirst preorder periods. So from an AuthorEarnings perspective those aren’t “free” downloads for that author… but the payment rate isn’t necessarily 35% of Units x Price during the KF period.

      Delete rows 1, 2, 4, 19, 24, and 30 from our spreadsheet and you can measure the effect of not counting them at all: It reduces the Amazon-pub share from 11% of units to 9%, A-pub gross sales from 9% to 6%, and A-pub author earnings from 10% to 7%. Depending on the terms in each of those A-pub author contracts, the real numbers lie somewhere in between.

      RE: What I’m wearing onstage at DBW?

      Wait, what? I’m supposed to wear clothes?

      • Rather than clothes, you could consider going nude but with body paint in a minecrafty/pixelated style to maintain the secret identity.

  16. I’d like to chime in here, in support of the crucial aspect of what AE means to authors. I’ve been with Simon & Schuster for five years and received truly huge advances for five books for which I will be eternally grateful, and successfully independent also for three of those years, so I’ve seen both sides of the fence in great detail.

    Arguing the toss over potential inaccuracies in the AE report is essential – all information must be challenged in order to find truth, but it doesn’t make a damn of difference to authors out there as long as the general AE report picture is itself accurate. That picture, of independent authors making a future for themselves more RELIABLY via independent publishing, as Lee himself admits, is the best bet in the current publishing climate. The overwhelming evidence now across many AE reports is that writing, as a means of making a living, is now within the grasp of authors who previously would have been, as Hugh reminds us, mid-listers as trads and essentially out of a job.

    Lee, you’re a big hitter and I know you rightly earned every bit of it, but your criticism really does miss the point and as Hugh said, even if you were correct it doesn’t matter one little bit when I see so many previously full-time trad’ published authors here in the UK being forced back into day jobs while at the same time I see more and more indies going full-time. The effect of big-hitting first-day sales don’t concern them – long-term trends do, because that’s how their careers are being built.

    The AE report, when viewed from here on the ground in terms of the real-life, day to day effect on authors, is spot-on. Personally, I hope that the major publishing houses don’t figure it out, but that struggling mid-listers signed to them do and have the courage to break out on their own.

    Hell, if a noob like me can do it and make it work…. 🙂

    • It’s also critical to remember the climate before and after AE. This is where I think Data Guy deserves a freakin’ medal. Prior to the first AE pie charts, there was a story every week about how awful self-publishing was. The “Third rate cattle” piece. The “tsunami of crap” piece. The “average income is $50” piece.

      I remember this clearly, because it was happening while DG and I were working on the first report. We knew what the data said and the impact it was going to have. We knew the people who needed the information would appreciate it, and everyone else would attack it. All we cared about was the former.

      Hundreds of authors have reached out to me or told me in person that AE was what changed their minds to go indie and that they are glad they made this decision. The attacks on self-publishing seemed to moderate almost immediately. The arguments became, “self-publishing is NO BETTER than publishing traditionally.” The tsunami of crap crowd began saying, instead, that people should take whatever path is right for them.

      Overnight, we won equality. The stigmas shattered. The reporting hasn’t been the same since.

      All we did was pull back the curtain. Hundreds of us knew this was a big deal from being in the trenches and talking with thousands of authors a year. I was at a different conference every week, giving talks, attending talks, being on panels, sitting in on panels, talking to Big 5 CEOs and Amazon VPs, picking everyone’s brain. It was at least three years ago that I was urging more coverage on what was happening in the midlist. No one wanted to touch the story, so we gathered information on KBoards. Then Data Guy happened. The rest is history.

      We know how many views the reports are getting. And I know industry peeps are terrified of this information. It reveals how powerful Amazon has become and how much market share indies have taken. It’s a big deal. Our data has been available for two years, and I have yet to see a refutation of the core finding, which is that indie authors are out-earning Big 5 authors on the Kindle store.

      That’s not only liberating for authors, but it puts pressure on publishing contracts and digital royalty rates. What I don’t get is the trad-pubbed authors who aren’t using this to their advantage. They see it as an attack on their decisions, which were made in a clouded past. Instead, they should experiment with their works or pressure their agents and publishers for better terms.

      • For all you do, and DG, and all the other Indies who’ve been pushing authors to think before they sign, I can only say thank you.

        You’ve made my life better by simply opening my eyes.

  17. Reading these comments has been enlightening on so many levels that someone needs to write a blog post or a news article to explain it all.

    But I’m an indie author, what do I know?

  18. There are so many generous people in indie publishing. They help others freely, without reward beyond thanks and the promise to pay it forward. It might be about formatting, or designing a better book cover. Between two writers not in competition but in goodwill.

    Then there are those who reach many people. Hugh Howey and Mark Dawson are two who come to mind this morning. Thank you, gentlemen, for your generosity, time and always kind words.

  19. FYI: Publishing Perspectives is owned by the Frankfurt Book Fair – which itself is owned by the German Publishers and Booksellers Association – so it is, quite literally, a mouthpiece of the industry. Porter Anderson hires himself out as a consultant (part of his commercial offering is “exposure in columns and posts”) and his clients are a wide variety of companies in publishing. So… yeah.

    tl:dr ignore

  20. What impresses me here is that Data Guy has accepted feedback and makes significant progress in improving his reporting every cycle.

    Personally, I’d love a third view of the market. Bring on even more Data Guys. The higher the bar goes, the better the data that we get.

  21. Great response, PG — I had to cover my mouth when reading your comments whilst drinking my morning cuppa. 😉

    Here is the story of self-publishing that AE is painting:

    I submitted to agents in 2011 – 2012 with a paranormal romance novel and was turned down by the biggest agency to rep my kind of book.

    I self published my book in 2012, along with the rest in the series. I self published another series and then another so that now, I have a backlist of 12 full-length novels, 3 boxed sets, and several novellas.

    My income:

    2012: $9700 (June – Dec)
    2013: $97,500
    2014: $154,000
    2015: $334,000
    2016: $40,653 (Jan)

    I am not a big name in indie romance publishing. I am solidly mid-list. My eBooks release in the top 100, 1000, 5000 depending on the series. I have made over $600K in 3.5 years.

    That agency that rejected my debut novel? They contacted me in 2013 when I had a hit novel and offered representation.

    I wouldn’t accept a traditional deal unless it was seven figures because of the terms. I can make more as an indie mid-list author than I could as a traditionally published mid-list author. I keep total control and have all the data on a daily and hourly basis. I get paid monthly. All the business stuff I do as an indie, that traditional publishers do for their authors? It ain’t rocket science.

    I may not be one of the 40 indies who have sold a million eBooks, but I don’t really care. Those indies are the black swans of publishing just as authors like Mr. Child are the black swans of traditional publishing.

    Most successful indies are like me. We can sell 60,000 – 100,000 books a year and earn a very nice living at 70% share of the price of our books.

    • THIS is the story of self-publishing. It’s authors who are not household names making serious money and getting no attention or press.

      Do most self-published authors earn very little and fizzle out? Of course. Just as most traditionally aspirant authors never get a book to market.

      But the authors who publish a dozen works, in a genre that sells, and treat their works professionally, have a great chance of earning a living that bookstores and publishers can’t take away from them.

      I don’t see how anyone can be against this. It’s an awesome development. Except for those whose bottom lines are being affected, which puts traditionally published megastars and NY middlemen in the position of being anti-free speech, anti-entrepreneurial, anti-tech, etc. Because of their greed and selfishness, they end up becoming complete cranks.

  22. History shows the company or segment that is gaining market share usually wins over the company or segment that is losing market share.

    The losers often object, citing how much money they are making. Some tell us they are making more than last year. Some individuals will scoff, citing their personal income. But, as time moves on, and market share continues to shift, all that stops.

    Unit market share often moves before dollar market share. Then they both move. That’s what we are seeing here. Market share has been shifting from the big publishers to the independents.

  23. Lee’s contention piqued my curiosity about Big Five preorders. So I took a look.

    Preorder sales were already included in our data. If you want, you can tally ’em up for yourself in the downloadable AE spreadsheet (see columns N & O).

    On January 10, 2016, those upcoming preorders comprised 6.3% of all Big Five Kindle unit sales and 8.7% of their Kindle dollar sales. So somewhat smaller than advertised, it seems.

    Lee’s definitely right about one thing… those books are mostly scheduled for Tuesday releases. But their preorder sales aren’t all going to pile up and skew the rankings on that future Tuesday — they are getting counted in those rankings right now.

    This is why it’s valuable for us all to share data and discuss it. We can all learn about our industry from each other.

    All of us are smarter than any of us.

    Perhaps William Ockham, or another one of you eagle-eyes, can spot something else interesting in the below (sorry for the formatting.)

    mysql> select round(bigfive_preorder.salesperday/all_bigfive.salesperday*100.,1) percent_of_daily_bigfive_units_from_preorders, round(bigfive_preorder.dailyrevenue/all_bigfive.dailyrevenue*100.,1) percent_of_daily_bigfive_revenue_from_preorders
    -> from
    -> (select sum(salesperday) salesperday, sum(dailyrevenue) dailyrevenue from kindlebooks where bigfive=1 and preorder=1) bigfive_preorder
    -> left join
    -> (select sum(salesperday) salesperday, sum(dailyrevenue) dailyrevenue from kindlebooks where bigfive=1) all_bigfive
    -> on 1=1;
    +———————————————–+————————————————-+
    | percent_of_daily_bigfive_units_from_preorders | percent_of_daily_bigfive_revenue_from_preorders |
    +———————————————–+————————————————-+
    | 6.3 | 8.7 |
    +———————————————–+————————————————-+
    1 row in set (0.98 sec)

    mysql> select dayofweek(datepublished) day_of_week, sum(salesperday) daily_bigfive_preorder_unit_sales, round(sum(dailyrevenue),2) daily_bigfive_preorder_revenue
    -> from kindlebooks
    -> where bigfive=1 and preorder=1
    -> group by dayofweek(datepublished);
    +————-+———————————-+——————————–+
    | day_of_week | daily_bigfive_preorder_unit_sales | daily_bigfive_preorder_revenue |
    +————-+———————————-+——————————–+
    | 2 | 419 | 5374.83 |
    | 3 | 8820 | 109833.33 |
    | 4 | 2 | 19.98 |
    | 5 | 12 | 74.96 |
    | 6 | 1 | 4.49 |
    | 7 | 0 | 0.00 |
    +————-+———————————-+——————————–+
    6 rows in set (0.51 sec)

    mysql> select FLOOR(DATEDIFF(datepublished, ‘2016-01-10’)/30) months_before_launch, count(*) preorder_title_count, sum(salesperday) unit_sales_per_day, round(sum(dailyrevenue),2) gross_consumer_dollars_per_day
    -> from kindlebooks where preorder=1 and bigfive=1
    -> group by FLOOR(DATEDIFF(datepublished, ‘2016-01-10’)/30);
    +———————-+———————-+——————–+——————————–+
    | months_before_launch | preorder_title_count | unit_sales_per_day | gross_consumer_dollars_per_day |
    +———————-+———————-+——————–+——————————–+
    | 0 | 202 | 4569 | 59197.82 |
    | 1 | 143 | 964 | 10313.13 |
    | 2 | 84 | 972 | 11601.15 |
    | 3 | 69 | 850 | 11489.88 |
    | 4 | 57 | 583 | 6910.27 |
    | 5 | 34 | 453 | 5373.32 |
    | 6 | 20 | 398 | 4463.08 |
    | 7 | 15 | 211 | 2743.92 |
    | 8 | 25 | 246 | 3148.58 |
    | 9 | 2 | 2 | 23.98 |
    | 11 | 1 | 2 | 20.99 |
    | 14 | 1 | 1 | 2.99 |
    | 18 | 1 | 1 | 6.49 |
    | 182 | 3 | 0 | 0.00 |
    +———————-+———————-+——————–+——————————–+
    13 rows in set (0.50 sec)

    mysql> select case when bigfive = 1 then “Big Five Published” when multiauthor = 1 then “Small/Medium Publishers” when amazonpub = 1 then “Amazon-Imprint Published” when indie = 1 then “Indie Self-Published” else “Uncategorized Single-Author Publisher” end publishertype, count(*) titlecount, sum(salesperday), sum(dailyrevenue)
    -> from
    -> (select indie, multiauthor, amazonpub, bigfive, salesrank, soldby, publisher, author, title,
    -> datepublished, FLOOR(DATEDIFF(datepublished, ‘2016-01-10’)/30) monthsbeforelaunch, salesperday, dailyrevenue from kindlebooks where preorder=1 order by salesrank) a
    -> group by indie, multiauthor, amazonpub, bigfive;

    +—————————————+————+——————+——————–+
    | publishertype | titlecount | sum(salesperday) | sum(dailyrevenue) |
    +—————————————+————+——————+——————–+
    | Uncategorized Single-Author Publisher | 220 | 3266 | 9745.615043997765 |
    | Big Five Published | 659 | 9254 | 115307.59956109524 |
    | Amazon-Imprint Published | 63 | 16124 | 95862.43979775906 |
    | Small/Medium Publishers | 557 | 1940 | 11330.409903287888 |
    | Indie Self-Published | 87 | 3345 | 12035.090093970299 |
    +—————————————+————+——————+——————–+
    5 rows in set (0.58 sec)

    • 1. Thank you for what you do DG
      2. Are you really going to appear in public and let everyone know your secret identity?
      3. I’m not sure what the above post means exactly; I did get an “OMG HUGE BRAIN STUFF HAPPENING” thrill when I read it 🙂
      4. I love this line: All of us are smarter than any of us.

    • What I see is that readers don’t buy ebooks very far in advance. Looks like they wait until almost the week of release to buy an ebook.

      An interesting note: The Big 5 live and die by the Amazon pre-orders. It’s all they talk about prior to a launch. “How are the Amazon numbers? What are they like this week?” is the constant chatter in the editorial offices. They base their print runs off these numbers. It’s a really big deal, as Amazon’s data is the best predictor of demand.

      This is the reason Hachette was so angry at losing their pre-orders. It had nothing to do with sales and everything to do with data. What Amazon was doing was removing one of the many side benefits they provide, at no extra cost, to show Hachette how much they relied on Amazon without paying them for the services.

      • That’s a really fascinating insight — I hadn’t thought about the implications of Amazon preorder visibility in steering the Big Five’s marketing-allocation and print-run decisions.

        The modern publishing world really revolves around online sales, nowadays — one way or another.

        • They wouldn’t want you or anyone else to know anything about it.

          Can you think of the madness it would cause if a group decided to ‘game’ the pre-order numbers and then they all cancelled the day before the book was to come out?

          (not that I would suggest that anyone should mess with what’s left of their feeble minds …)

        • Yup. An advantage of working with a few of the Big 5. They have to call Amazon (or email) to get the number, and it doesn’t come quickly, so they really hang on that stuff. It’s an obsession, those pre-order numbers. Probably one of the reasons Lee thinks it’s a bigger deal with regards to AE than it really is. Not only are they nuts about those numbers, they don’t even get how they work.

          It really is bizarro world inside the Big 5.

          • Shame on you HH, shame …

            Though strange by our standards, the bizarro world has rules that it follows — how dare you compare it to trad-pub and the qig5! 😛

        • When I pre-order a book Amazon doesn’t charge me for it until day of publication, e or print. Does that help or hinder the data?

          • Rank is affected immediately. Pay isn’t logged until the release date.

            • now I really wonder what would happen to rank if a bunch of people were to cancel a preorder on the same day

              • The boost in Sales Rank is always credited immediately. It doesn’t matter if it is a pre-order, a normal sale, a KOLL borrow, or a KU borrow, you will get the appropriate boost in Sales Rank for one paid sale right away (usually in a few hours, a little longer if the system is slow/glitchy). And it’s always one-way, so your Sales Rank doesn’t get dinged if someone returns a book seconds after purchasing it, or cancels a pre-order, or returns a book one page after borrowing it. So, to answer your question, if a bunch of people simultaneously cancelled a pre-order, it would have zero affect on Sales Rank.

                And just to back up what was said already: Lee is wrong.

                Amazon most certainly does not roll-up pre-order sales for a launch-day ranking boost. That is demonstrably false. Some of the other retailers do (e.g. Apple and Kobo – I can’t remember re B&N), and that’s how the NYT calculates “Week 1” sales for the bestseller lists, but you absolutely do not get a pre-order related ranking boost on launch day. It simply doesn’t happen.

                Either Lee is confusing the Kindle Store with something like the iBookstore, or he is seeing his book rise in the rankings on launch day from organic orders, extra visibility, launch marketing, the effect of being on all those front tables in stores nationwide, and so forth.

                What *might* skew things in terms of what day AE grabs data is that, traditionally at least, the large publishers tended to release on a Tuesday for all sorts of reasons. And a lot of big indies are doing the same to maximize their chances of hitting a list.

                • B&N’s Nook bestseller list works the same as Amazon’s Kindle bestseller list: they count pre-orders as they come in, not all at once on the on-sale date.

                  Apple and Kobo are the only ones who give a generous double hit: they count pre-orders as they come in and all at once on the on-sale date.

                  Lee’s confusion may stem from the fact that the retailers do report all pre-orders on launch date to the USA Today and NYT bestseller lists. But for compiling their own ebook bestseller lists? No. Each retailer has their own quirky system. Amazon’s Kindle bestseller list works exactly as Hugh and Data Guy have described above.

      • Mr. Howey – “What I see is that readers don’t buy ebooks very far in advance. Looks like they wait until almost the week of release to buy an ebook.”

        The change in consumer behavior is going deeper: ebook shoppers are starting to really internalize the “ebooks are always in print” meme. Which means more and more are buying just before reading and instead of buying everything mildly interesting (or hyped) when they first see it, they are simply adding the title to their wishlist.

        This came up repeatedly over at Mobileread as a possible explanation for the early reports that “ebook sales are declining”.

        The reports aren’t real but the changed behavior is. As it spreads those preorders and launch day spikes will shrink as more and more readers stop stockpiling and building large TBR hoards and adopt just-in-time purchasing.

        And that is just one of the many ways consumers are adjusting their habits to better exploit ebook tech.

        • This all sounds right to me. It’s how my buying habits have changed. I used to hoard ebooks; now, I shop after finishing one book and buy the next.

          • Anoth behavior change is that Readers may also be hoarding fewer ebooks because, as the years go by and one’s devices die, storing books on the device risks losing those books.

            Amazon limits the number of copies of an ebook to five devices. When my first Kindle died an untimely and lamented death at my clumsy hands, all the books I had stored on it became trapped to that device as far as Amazon is concerned. Those books have only four “device lives” left. I no longer store books on my devices; only the current book I am reading.

            (Amazon may have a process for releasing the “imprisoned” copies from dead devices, I haven’t contacted them to ask. eventually they will need to.)

            If I’m not storing a book on my device that also removes an incentive to pre-order a book. It makes more sense to put it on hold with my public library . If I get lucky and get an early slot I won’t have to pay for it at all. The higher prices libraries pay for TradPub e-books might balance this out but it does reinforce individual consumer behavior to not pay for eBooks.

            Another disincentive to pre-ordering is the publisher may lower the price later.

            These are the changes in my behavior as a consumer with respect to the tradpubs. For Indies and Small Press I feel more connection to the publisher and more responsibility to the author, so I’m more likely to pre-order and/or pay full price for an ebook.

            • I don’t think that is Amazon’s doing.
              I think that is per-book dependent because right now I have 10 devices registered to my account:

              3 Kindles
              1 Fire tablet
              1 Phone
              3 PCs
              1 Sony ereader
              1 Android STB

            • through the kindle pages on the website, you can disconnect a device, which should restore this if it’s a matter of devices at one time.

              but if a publisher limits it to 5 devices over your lifetime, that could be a problem.

              I wondered what that ‘unlimited devices’ thing meant on the books I’m buying

    • Also: I’m assuming the “182” was an upper bound, and those three titles are set to release in a couple years, and not literally in fifteen years. Please tell me this is so.

    • The only question I have for now is why there only 6 rows in that day of the week query? That would mean that no row in the table had a day 1 publication (i.e. Nobody publishes on Sunday). Does Amazon simply not allow that? I thought we got rid of blue laws (that’s a joke, google/bing it)

      • This was filtered to include Big Five preorders only, so either they don’t schedule Sunday preorders… or they just don’t do it for the books that they plan to market and expect anyone to buy.

        • Oops, I misread the query (really should have pasted into an editor). What that really says is that there was at least one book published on a Saturday by the big five, but zero sales. Which is even weirder in a way.

  24. Yeah, no clue.

    It’s a Hachette preorder due to officially launch in a month or so… which has racked up a total of zero Kindle preorders over the last couple weeks or so. Guess it’s not getting much marketing from them…

    mysql> select * from kindlebooks where bigfive = 1 and preorder = 1 and dayofweek(datepublished) = 7\G
    *************************** 1. row ***************************
    id: 2112565
    asin: B0169ATLTU
    title: Walking The Himalayas
    author: Levison Wood
    format: Kindle Edition
    preorder: 1
    price: 1399
    topsoldby: Hachette Book Group
    publisher: Little, Brown and Company
    publisherline: Little, Brown and Company (March 5, 2016)
    soldby: Hachette Book Group
    datepublished: 2016-03-05
    salesrank: 596810
    averagerating: 0
    totalcount: 0
    fivestarcount: 0
    fourstarcount: 0
    threestarcount: 0
    twostarcount: 0
    onestarcount: 0
    categories: Books > Science & Math > Nature & Ecology ; Books > Travel > Asia > India > General ; Books > Travel > Specialty Travel > Adventure ; Kindle Store > Kindle eBooks > Science & Math > Nature & Ecology ; Kindle Store > Kindle eBooks > Sports & Outdoors > Outdoors & Nature ; Kindle Store > Kindle eBooks > Travel > Asia > India ; Kindle Store > Kindle eBooks > Travel > Specialty Travel > Adventure
    drm: 1
    titleKey: B014DUSMCA
    lendingenabled: 0
    kindleunlimited: 0
    kindlecountdown: 0
    kindlefirst: 0
    filesize: NULL
    printlength: 304 pages
    texttospeech: 1
    xray: 0
    pagenumberssourceisbn: NULL
    language: English
    whispersyncforvoice: 0
    pagecount: 304
    isbn13: 9780316352413
    batch: 20160111052829.553-28
    created: 2016-01-10 22:33:13
    updated: 2016-01-10 22:33:13
    1 row in set (0.80 sec)

  25. Hi, from Ireland.

    Watching the debate from here is pure 15 round entertainment.

    The lottery theory of trad publishing is right though. What agents and publishers sell is a dream ticket for a lottery 2 to 3 years in the future.

    And they expect you to accept that, because that’s the “way it works”.

    But the real Achilles Heel of trad publishing is the lack of diversity in storytelling, which the Tent Pole model produces as an intended consequence.

    More Jack Reacher novels and James Patterson cutter cutter paperbacks fill the book stores each year and meanwhile the kids are on YouTube and Facebook and Tumblr and getting their VR game systems pre-orders in.

    We argue about data, but the demand for stories is shifting. Extending stories online is the answer IMO.

  26. Lee: Here’s a question.

    Let’s assume that you’re right, and self-publishing is the right answer for most, but the traditional path is best for authors who want to “aim high”.

    Let’s ignore for the moment the Andy Weirs, E L Jameses, Hugh Howeys, A .G. Riddles, etc., who self-publish, out-earn their traditional peers like Scalzi, etc, and find themselves in a strong negotiating position when traditional publishers come knocking.

    Seems to me that if an author is “aiming high”, they shouldn’t settle for a mediocre advance … especially since, if they are good, they can make a lot more money than that advance by self-publishing.

    So here’s my question: given that most trad-pub advances come in below what the author could make on their own in the first 12 months, what dollar advance is the minimum that a debut author “aiming high” should accept?

    $100,000?
    $500,000?
    seven figures?

Sorry, the comment form is closed at this time.

%d bloggers like this: