New Author Earnings Report

113 thoughts on “New Author Earnings Report”

1. What the author community and PG’s above analysis misses is that it is not KDP Terms and Conditions that Data Guy falls foul of, but the License and Access section of Amazon.com’s Conditions of Use. So Data Guy’s information is only publicly available to anyone who accepts the license restrictions. Data Guy was never compliant, due to the exclusion of data mining or robots, but he is in further trouble because he presumably sought to “exploit an Amazon service for a commercial purpose without the express permission of Amazon.” Amazon is always looking for expenses to reduce its taxable profitability and Data Guy may be about to provide just such an opportunity. By breaching these terms Data Guy loses the license to use Amazon.com and therefore the data is neither public nor legally obtained.

   A bigger reason for Amazon to pull the plug is hinted at in the restrictions imposed on Book Report. Amazon will tolerate a certain amount of hits to its database, but stopped Book Report doing sales histories of more than 90 days. How long will Amazon tolerate a 24/7 data-mining activity on their servers.

   Older hands might recognise my name as the author of an article featured on PV (thanks again, PG) critical of Data Guy’s statistical methodology and his project’s destruction by the borrow rank boost from Kindle Unlimited, which helps Amazon imprints more than anyone else. Much of the statistical problems are ironed out by Author Earnings / Bookstat looking at the whole of Amazon and not just Top 100 of sub-sub-categories as he initially did. But the Kindle Unlimited problem has not gone away. Those selling less than $10 million per annum are being done a favour by being excluded from buying access to worthless statistics. And they will remain worthless until Amazon returns to rank boosts for sales only.

   • Good points, Mercia.

   • And they will remain worthless until Amazon returns to rank boosts for sales only.

     Worthless is a function of use. The lack of precision makes things worthless for some uses, but very valuable for others.

     For example, market analysis rarely has exact precision, yet it is very useful for some people.

     Lawyers engage jury consultants to help them pick juries. Their information lacks precision, but may be quite helpful for the lawyer’s purpose.

     Military intelligence is never complete, yet it is very helpful to the generals.

     Unemployment and labor participation rates are never precise, and surely lack some things, yet economists find them very useful tools.

     Few survey samples rarely conform exactly to the ideal sample standards for a target population, but often their findings are very useful to some people.

     The user determines what is useful, and what isn’t. I can’t make that determination for him. Nor can I choose his purpose for seeking any type of data or information.

2. Regardless of how I feel about this effort (personally, I am ambivalent), I can’t see how DataGuy’s effort is any different than Google crawling the web or the companies that gather directories of phone numbers and then sell access to them. (Naturally, just my opinion and supposition as to what DG is doing).

   It appears to me that all DG is doing is running programs that check sales ranks … just very powerful, robust programs that check millions of books very frequently … but still, ultimately only gathering publicly available information, just much more information gathered much faster than a single person could do.

   As for the report naming authors, I don’t see how this might be considered illegal. First off, all publicly available information that the author has authorized Amazon to share via the TOC for KDP. And really, not any different than BookScan or music sales charts or again, online directories. By consenting to sell their books on Amazon, they are consenting to Amazon gathering and sharing that sales data to any degree Amazon’s wishes. They have essentially become public figures in this capacity. DG is simply gathering what Amazon has decided to share publicly.

   The potential conflict is between Amazon and DG … they could block his programs, assuming they describe it as similar to a low-level Denial-of-Service attack (since DG’s services must be bombarding Amazon’s servers with information requests … not a real threat because Amazon services are so robust, but I suspect that would be the basis for a ban-hammer).

3. Here is another blog post on the topic exploring why Amazon might want to close down what data guy is doing
   http://aimwarddrift.blogspot.com/2018/01/ok-so-about-data-guyauthor-earnings.html
   I hadn’t thought of it from this angle before, if data guy has access to sellers data, does he also have access to the buying habits of Amazon customers and if so, how would Amazon react to this information being sold by a third party.

   • “1. This isn’t publically available data that Data Guy is using. This is data extracted from Amazon’s private servers by 3rd party robot web crawlers. ”

     That is false. The rest of the post is gibberish.

   • This isn’t publically available data that Data Guy is using. This is data extracted from Amazon’s private servers by 3rd party robot web crawlers.

     No. The crawler is a program that goes to each Amazon book page, just like anyone can. It reads the price and ranking like anyone can. One could do the same thing with a yellow pad and #2 pencil, but not nearly as fast.

     Nothing on the book pages contains data about customers.

     I suspect Amazon has extensive defenses against anyone trying to breach security on private servers.

   • At the newspaper where I worked, we had a programmer who wrote a crawler to scrape information from businesses on Facebook and Yelp in order to populate data fields for our own venture. The crawler replaces teams of humans wielding yellow pads and #2 pencils.

     Any programmer could write a crawler, and there’s no reason to think DG’s information hasn’t already been scraped by other people for whatever reason. If anything, this incident should wake up indies to to the realization that potential vendors or business partners may have done their homework on them exactly as DG demonstrated. Don’t go into negotiations thinking they haven’t.

     I looked at the original list of authors as the equivalent of lists of the “top paid Hollywood stars,” or “top paid rock stars.” The salaries movie stars make on movies or in general get reported all the time; just go to iMDB. I don’t know if that information includes the foreign commercials actors sometimes do, or whether or not they have their hand in other ventures. So I agree caveats and footnotes should be added — “This information does not include any licenses authors sold for foreign rights or movies, etc.,” which would cover the possibility of other income DWS mentioned.

     At the same time, I’ve routinely seen movie stars discussed in terms of whether they can “carry” a movie or “fill seats,” and how their salary for a movie directly relates to that factor. Having or not having that kind of leverage is not a new problem, just one indie authors weren’t expecting to have. Surely it’s useful to take into account whether you will be regarded as a “supporting actress” or a “box office draw”?

     From what I can tell, the biggest concern is that scammers will know who to target … but how do you know they weren’t already using crawlers (or just teams of humans) to troll for victims? They seem to know who is doing promotions, who has a certain sales rank, and so on. What makes you think they weren’t crawling BookBub and other promo sites, let alone Amazon? This is a different problem to be solved (and solved it should be).

4. DG responds somewhat (in his own comments):

   http://authorearnings.com/report/january-2018-report-us-online-book-sales-q2-q4-2017/#comment-297486

5. As I stated in the previous post on AE report:

   I was on the list. If anyone was surprised to see my name there, they haven’t been paying attention to the bestseller lists during the year. None of those names should surprise people who are paying attention to the business end of business. I don’t care that I was named. I believe knowledge is power and I have no issue sharing what I know. I was also one of those authors that told everything about my traditional publishing contracts.

   And for the record, I’m 100% introverted, so I don’t enjoy attention for attention’s sake. But business data is important and relevant. I have no issue with the data being public or how it was gathered.

   • But don’t you feel a little irked that the knowledge will only be accessible to Trad pub? I agree knowledge is power. I think if the info is going to be accessible, it should be accessible to anyone who wants to pay DG for access. It seems harsh that authors will be cut out of the loop.

     • Well, I guess complaining didn’t get the authors what they wanted, did it? That information is still out there and now they won’t know. Yes, I think it’s a shame. I’ve always been transparent about my business.

6. I started to write something when there were 64 comments, but shyte happened. When I came back, there were 88 comments and some of the new ones were looonnnggg. I note that Data Guy — no stranger to TPV — has not chimed in, and I think that was a wise choice.

   I can smell the fear through the internet. I read a lot of anger and hate and misinformation and incompetent legal opinions (not yours, PG), and I got images in my head of torch-wielding mobs descending on Data Guy for a right rosy application of street justice. Maybe that’s just the writer in me waxing poetic. Maybe not.

   What I wanted to write about this, I no longer will. Not today. Minds are made up, and no one wants to listen anymore. PG, I hope you revisit this issue in a week or so, after the anger has burned out.

7. just to add
   Miramax and the entire film industry including Sundance and other displayers and all distributors… one will never likely know the real numbers. Not ever. If you’ve ever had a movie deal, or even a movie option, you see from the inside, the evasive loops and underground stashes and calling /hiding/ sheltering various payouts and expenses by weasel names/terms. What is reported at boxoffice, in reports, royalty statements, and more is a sodden mess of paper trail, a tangle of false doors, empty storefronts, set asides for favs, dishonest reporting by certain distibs/brokers/ agents/producers, etc. In all that too, there are the honest people. Who tend to be the actors, screenwriters, authors…

   It’s a wonder a film ever gets made and anyone gets paid after the middlemen take their grosses and manipulate the accounting. You’d have to have ten crack auditors working round the clock for a year to de-falsify the ‘claims’/tangle of gross ticket sales and net of just one film.

   “Miramax doesn’t get to decide what to tell us about how much money their films made. The box office tells us that, through some clever wizard who compiles the data and reports it to us. “

8. I won’t comment on whether Amazon owns the use of the data it publishes on its website because I have no idea, and I’m certainly no expert in matters of privacy, but it seems to me people have misunderstood private vs. public information and what is fair (i.e., ethical) game for reporting.

   By Dean Wesley Smith’s own words, we can see that Data Guy has not exposed Dean’s personal business income. Dean makes money through Book Funnel, he sells on eBay (maybe things other than books!), has income from movie options, and so on (And God bless! Count me as one of the many who are filled with respect and admiration for all Dean’s and Kathryn Kristine Rusch’s success and their contributions to our industry.) Presumably, the top 50 authors have some similar diversification of income. So, DG has not hacked into these authors’ QuickBooks accounts for an exhaustive report on their personal finances. As many others have detailed, he’s largely looking at ONE selling platform, extrapolating publicly available data and analyzing the product, and the name associated with the product.

   I can sympathize with the alarm of suddenly realizing your name has been put on a list with a dollar figure next to it, and how that’s going to have the world beating a path to your door in ways good and bad, but go cry on J.K. Rowling’s shoulder and see if she doesn’t laugh out loud! By what special right does the #1 self-published author on Amazon, ranked only two slots below Dan Brown in ebook sales, get to remain a complete unknown, with her annual Amazon sales a closely held state secret?

   It is good to note, as Dean did, that independent authors are a business, whether they like it or not, and businesses don’t have absolute control over all the data associated with their product. Miramax doesn’t get to decide what to tell us about how much money their films made. The box office tells us that, through some clever wizard who compiles the data and reports it to us. Even Amazon doesn’t get to decide to tell us about certain aspects of their annual numbers. They have reporting requirements under federal law, and by federal law those reports are publicly accessible. I once worked for a non-profit and my annual salary was posted in a report on our website. No resulting target on my back for scammers of course, but it was still uncomfortable.

   If particular authors trusted DG and feel they were misinformed either by commission or omission about what they were helping him do, then I full-throatedly stand with them in their outrage because that IS unethical. But it seems what’s really being implied is that these authors pulling down sales of six- and seven-figures annually are not business-savvy and therefore not ready for their close-up. They are the most successful authors on Amazon but didn’t want it publicly known how successful they are, for reasons that are their own. Fair enough to feel that way; I just have less sympathy for it.

   We have been clamoring for years for self-publishing to be taken as seriously as the trade publishers. Well, it looks like we’re there. As someone else said, we may soon see self-published authors on the Forbes annual list of top-earning authors. Those authors will not be asked for their permission, and I’m sure Forbes will not be amenable to smudging out any names. So maybe the Author Earnings report has done all the top dogs a favor in advance of the really white-hot spotlight – because let’s face it, Data Guy is no Forbes!

9. FYI for those that haven’t recently looked at the Author Earnings report in question, the Indie list has been replaced with this:

   “ETA: We had initially shared a ranked Top 50 Indie Ebook Sellers list here (with units and dollars blurred out, of course). But then some of the authors on it started emailing us and asking for their names to be blurred out, too. As a courtesy to those authors, upon request we did so, but after the first few it became too much of a hassle. And besides, with a quarter of the list greyed out, it no longer effectively illustrated the point we were making, anyway. So we yanked the whole list, and will just simply state what we observed on it.”

10. Somewhere in the middle of getting my words done for today, it occurred to me that there seems to be a disconnect here. Data Guy is selling our individual-level data to competitor entities. That’s mostly being discussed as a privacy concern. That’s the least of my worries, folks. This isn’t just an interesting report Bookstat is providing – it’s actionable, real-time data that could do serious harm to author careers. [Which has always been true, but having it collected in one easy place radically changes the level of threat and the number of authors potentially at risk.]

    In case some of you don’t share my paranoia or creative imagination, here’s some of the ways I would use that data (in a parallel universe where I have significantly shadier ethics and way too much time on my hands.)

    • I would track trending preorders and put my ghostwriters to work generating near copycats that release before the preorder books of popular authors in my genre.
    • I would set an alert to watch for price changes and pulse ads to ride the coattails of authors about to have a Bookbub or other big promo blitz.
    • I would analyze words in titles that do particularly well for whatever metric I care about and use that to name my new series.
    • I would identify authors who seem to be fairly hands-off on marketing and mostly focused on writing and put them on my list of authors who probably won’t notice a scam aimed at them for a while.
    • I would test tactics like sending a few hundred random people to someone’s product page, or disliking reviews, or posting one-star reviews, or reporting content issues, and see how it impacts that book’s sales. I would develop a very precise list of just how I can subtly screw with someone’s books in the algos.
    • I would use that carefully developed list of tactics to clear books out of the way on the days I want to take a run up the bestseller list.
    • I would also carefully analyze my scammy tactics that are attempting to legitimize my scammy books. Which ones work best to trick the algos into thinking I’m real?
    • I would hire myself out. For a price, I can dramatically improve your career—or fuck with someone else’s. And I would be good at it, so I’d get lots of clients.
    • I would sell custom data digging to indies willing to pay. Want to know the release tactics, geographic and vendor sales distribution, and marketing strategies of your 10 closest competitors? Done. Want alerts when someone in your genre is trending? Done. Want to know who manages to combat the 21/30/60 day cliff? Done. Want to know who never has a book rank above 1000 on Amazon US but still makes $500K a year? Done.

    None of these are a stretch. Except for the last one, all of them are already happening and real-life authors have had their earnings significantly impacted because of them. What Bookstat is providing is a way for that to level up. Way, way, WAY level up.

    Okay, back to words. This is my last post on this subject. Either authors care about this or they don’t. I’ve said my piece. It won’t be me they’re aiming for. 😉

    • I agree with all your points. This is how the real marketplace works. It’s not easy, and players don’t care if the competition approves of what they do. Nobody is about to ask the competition for permission.

      • That may well be true, but I don’t have to be happy about one of our own enabling the competition to this degree.

        And I’m pretty sure Amazon isn’t going to ask for permission for whatever it decides to do either. I think there’s a good chance we’re going to lose a really valuable data source.

        • Would you prefer one ‘not’ of our own to do it instead? Say someone like that Mike S. guy that takes in data and then cherry picks it to say more towards what he thinks his audience wants to hear?

          • Yes. I’m all for feeding the big publishers bad data ;).

    • …it’s actionable, real-time data that could do serious harm to author careers.

      Agreed.

      One small example from the 1980s of how business information can be used by a competitor…

      I worked for a small company in the RPG game publishing industry. At that time, there was one big whale and a bunch of smaller fish in the lake. My company was one of the larger small fish, and the whale decided we might pose a threat to its market share.

      In those days, you needed to announce your new releases a good 4-6 weeks ahead.

      So whenever the biggish little fish announced a new release, the whale would deliberately schedule one of its releases to hit a day or two before. The distributors had only a limited amount of money in any given week with which to buy product for distribution. So they would reduce their orders from little-big in order to be able to buy enough from whale.

      The maneuver was perfectly legal and legit, but it hurt little-big significantly.

      The whole publishing world is so different now that the specifics in my example are probably irrelevant. But the principle of using business information to damage a competitor holds good, I believe.

11. Don’t like DG’s report? Wait until Amazon posts the average percent of a book that is read on Kindles.

    • I really wanna know that..as a READER.

      • Likewise. If 90% of readers stop reading with four chapters left, I’m probably not going to start the book. Unless it’s the Necronomicon, but by now we all should be familiar with the “no reading aloud” warning. 😉

12. Bookstat’s business rests entirely on Amazon allowing it to exist. There is no real reason why Amazon needs to update its bestselling list every hour based on sales. There is no real reason why it needs to list millions of titles in the bestselling list. It can curtail or pull that public data tomorrow. It can construct best selling lists to suit its own priorities. Or it can use all the sales data it as on each of its customer to construct suggested reading lists tailored to each customer based on past purchases, foregoing bestselling lists altogether. In short, if Amazon decides that it doesn’t want this data known,it can remove it from its website and replace it with some other discovery method that is less tied to sales. The question is, does Amazon care? We’ll see.

    • Or they could only list the top 25 in every category and no other rankings. That would make it tough to follow fluctuations.

      I actually find it fascinating to develop a non-sales related way to promote books to specific customers. There are a hella lot of top-selling books I don’t give a fig about. Bet they could figure it out. Would be more helpful to me than whoever sold the most. What is selling that is interesting TO ME…

      • Yes. I look forward to Spotify’s Release Radar and Discover Weekly play lists to discover new artists. Amazon could offer a similar “Personal Librarian Service” based on your purchases (and/or preferences) that could be edited with thumbs up or down (to the book or author) without having to buy the book on the list. And Amazon could also offer hundreds of “Play Lists” grouping similar books together based on also purchased,or their own criteria. In any event, any changes would be for their customer’s (and Amazon’s) benefit, rather than for their suppler’s benefit, though I’m sure many writers would benefit, and many would be hurt as well.

    • Or they can kill every spider that tries to crawl their systems.

      • Its just web traffic, man. On public pages. Pretty tough to stop. Its trivially easy to spoof everything from IP address to browser type in the http heading.

Comments are closed.