Confidential Business Information

This content has been archived. It may no longer be accurate or relevant.

From Kristine Kathryn Rusch:

When the big indie publishing movement started almost a decade ago, we were all startled by it. From the ease of publishing ebooks directly to readers to the way that fortunes could be made, seemingly overnight, it all felt too good to be true.

And that created a culture of Prove It To Me. Part art of that culture was absolutely necessary. Most people did not believe that self-publishing could make them money. Those who did had either done it before, or had such wild success that they were as startled as the rest of us.

Joe Konrath and others started posting their indie sales numbers pretty early on.  I’m linking randomly to Joe’s blog posts from 2010, just because I remember him being so open about the numbers.

I also liked the tone of his blogs—surprise that the numbers were working out the way that they were; pleased that the numbers were working out the way that they were; and a small bit of worry that the numbers might not continue working out that way.

I’m assuming that they have for Joe, although I don’t know. He stopped reporting his sales numbers a long time ago. But he’s still going at it, still publishing his books, still analyzing what’s going on (mostly in his own career), and occasionally—very occasionally—blogging for writers.

In those days, most self-published writers posted their numbers, sometimes from all the sources (all two or three of them), and usually did so monthly. It was part of what became the indie community—this openness about how well you were doing, mixed with isn’t this great? and a lot of cheerleading (If I can do it, you can too.)

The cheerleading is mostly gone. Now it’s more business focused, and many of the people who are posting their numbers are trying to sell their method for garnering those sales.

. . . .

There’s nothing sinister about keeping information private. (Although there can be, at times.) Most businesses just prefer to work that way. They want to control the reveal of information.

I’ve often been in conflict with that reveal, because I’ve been a reporter off and on throughout my life. There are rules and ethics behind how reputable reporters get information, many of those adjudicated in courts of law.

In doing this blog, I often know much more information than I can comfortably (or legally) reveal to you folks. It’s part of the business.

So let’s come back to publishing.

. . . .

The latest Author Earnings report came out on January 22, after nearly a year of silence.  Data Guy has been capturing information what he says are the million top selling titles every single day, and then using his algorithm to turn that information into actual sales numbers. He’s tweaking Amazon’s algorithms so that they work for him.

Data Guy has done this, keeping his name and identity private, for four years now. He has done a lot of work to gather this information. He says he only uses information that’s publically available…to someone with his skills. Then he uses his own secret sauce to scrape and interpret the data.

I’ve always had a slightly queasy reaction to Data Guy’s methodology, because I can’t verify it. At worst, I figured, he was a white hat hacker. At best, he was right about how he got the information. Early on, I was usually one of the last people to report his findings, expecting Amazon to shut him down.

After years, and his appearance at various conferences, I figured Amazon knew about him. If they didn’t like what he was doing, or saw it as a threat to their private algorithms, they would shut him down. Apparently, in this early stage, anyway, they saw him as another data capture service.

They might also have had no qualms about what he was doing because he was not making money from it.

Anyway, this report’s findings are a nice confirmation of what we’ve all been saying—that the market has matured, and ebooks continue to grow. That would have made a nice two-day story, something that would have helped the publishing community along.

But hand-in-glove with the new report was the announcement that Data Guy has started a new company. It was only a matter of time before Data Guy did this. Either he would quit offering the information for free and go on to doing other things, or he would monetize his work. (I had thought, early on, that Hugh Howey paid him. I’m not sure what their arrangement is.)

In response to a question I asked on the site, Data Guy said the cost of providing this information has become so extreme he either had to monetize it or quit. That makes sense to me.

The method he chose to monetize this product, however, angered a ton of indie writers. The anger is only growing as I write this on January 26.

. . . .

This is not what Data Guy promised indie writers when he started setting up this company. First, he spoke at the annual conferences for Romance Writers of America and Novelists, Inc, promising writers that if they shared their personal data with him, he would then create an algorithm that would allow them (for a fee) to see their own numbers as scraped by his algorithm.

A lot of writers—many big names—shared their data with him, so he could tweak his product. When he announced the new business, however, it was clear that his business model revealed all of this information…to companies that have revenues of ten million or more. I did a piece just a few weeks ago on traditional publishers. Ten million is the minimum threshold for what I was calling “small traditional publishers” —if you look at publically available financial data on those companies.

How Data Guy will be able to verify this information is beyond me. I now consider him untrustworthy when it comes to handling information. He gathered information from writers under false pretenses, and is now giving it to their competitors. (I write this on January 26, so this may have changed.)

The writers themselves cannot access their own personal information, to even see if it’s accurate. (I can guarantee that it does not check a writer’s complete sales or income. Writers earn from many, many more sources than Amazon, even online.)

. . . .

When the free report came out, there was a list of the top-selling indie writers—by name—with the promise that some big corporation could get their sales numbers if the big corporation paid for it. Just the list of the top 50 indie writers had some big reveals that the writers themselves did not want public.

To make matters worse, none of those writers gave him permission to use their names. I suspect (although I do not know) that some of those writers gave him information to help him tweak his algorithm, thinking they would be getting personal data from him, not data on other companies. He betrayed them.

Writer after writer wrote to him, demanding their names be removed from that list.

Link to the rest at Kristine Kathryn Rusch

Here’s a link to Kris Rusch’s books. If you like the thoughts Kris shares, you can show your appreciation by checking out her books.

10 thoughts on “Confidential Business Information”

  1. He’s tweaking Amazon’s algorithms so that they work for him.

    How did he access Amazon’s programming?

  2. I don’t think anyone will change their mind on this issue,the hysteria is too strong, the moral outrage too potent.
    I also don’t think anyone who deliberately violated their contract by giving out sales data to a third-party Really have a leg to stand on.

  3. Publicly accessible data, including names (or pen names) shouldn’t be a big deal.

    As far as sales numbers…it’s gonna be an guesstimate. Maybe a really good one since there were those who gave him exact sales numbers, but it’s still gonna be a guesstimate for all those who didn’t share their exact numbers with him.

  4. I would ask, where is this anger she says is growing? Granted, I quit the Kboards forums ages ago, but the indie author’s I’ve been around and talking two haven’t said boo about this. There was the initial few posts when the report came out (most of the ones I saw were here), but where is this hue and cry?

    • I’m not sure if you caught Dean Wesley Smiths post on the matter, but a lot of the out rage was there.
      There was also the authors emailing data guy to move their names from the 50 best selling author’s list, though many comments about that weredeleted.
      Much of the rest occurred on Facebook.

  5. So DG took information that was available to anyone, and with the willing participation of some authors, invested the time, energy and money to put together something that every indie out there liked.

    Until he figured out a way to monetize it, and then many of those authors, who talk about and share their sales and sales rank in various groups and forums, both private and public, were upset he shared the information?

    Are the names being redacted because authors don’t want people to know what their status in the industry is, or because their ranking is less than they thought it is?

  6. Just the list of the top 50 indie writers had some big reveals that the writers themselves did not want public.

    To make matters worse, none of those writers gave him permission to use their names.

    Those writers put their goods up for sale to the public in public marketplaces. Data Guy doesn’t need their permission to use their names and to use the data he has compiled about them from public sources. If it can be compiled from publicly available information then by definition it is not confidential business information, much as she or others might wish it would be.

    • I agree, DaveMich.

      Data Guy (as far as I know) developed his own algoritms/equations/magic to approximate the Amazon ones (reverse engineering). He was not using the amazon ones because they are proprietary and they didn’t want to share nicely.

      He is perfectly entitled to do what he wants with what he has developed.

      It might be polite if people who contributed to his project with their own data got something back but I suppose that is between DG and those contributors.

      In the interests of full disclosure, I haven’t read any of the source material but been following PG’s excellent summaries of the Author Earnings project in this forum.

  7. It’s funny how many loved his data when they could get it for free and when it showed them how well they were doing at going their own way (vs trad-pub). It’s funny how much those same people loved their high rankings on Amazon’s ‘best selling books’ lists.

    But they’re not so much in love when he ties all this publicly available data and sells it to those willing to pay for it.

    I can’t see them being ashamed that they write – or they’d be using pen names. Nor why it being known that their books are selling well. (Unless they’ve been fibbing on their income tax or something – but then again this is one data type guy’s data guess – not something the IRS can take them to court with … 😉 )

  8. I think Ruschs Point is that independent authors will be on the back foot when it comes to contractual negotiations between themselves and a traditional publisher or movie company if they don’t have access to the data that date a guy is providing to larger clients.
    Here is part of One of her comments on the original post:
    ‘However, you miss my point: DG’s statistical prowess and interpretive skills are considered excellent. So when he says that XXX Indie writer has sales of YYY and is the top of her game, then his analysis will have a lot more impact than, say, my stupid guesses.

    It is good business for XXX Indie writer to know what DG has reported to a publishing house, a Hollywood studio, and so on, so that XXX can counter with–um, no, that’s incorrect data and here’s why, or something. The problem is that if DG’s business is successful, a lot of indies will lose business without even knowing why. Some studio will buy a report on XXX Indie writer’s sales, decide they’re not high enough or that XXX Indie writer doesn’t appeal to the right demographic, and the chance of an offer will disappear.

    Like you, I doubt DG will have the client base he needs. Even he acknowledges that the big publishers have their own internal data guys now, so he’s going for the mid-level publishers and anyone else whose company valuation fits his criteria. I doubt this is the threat I just made it out to be in the previous paragraph.

    But writers need to know that there’s the possibility for another DG to arise, one with some actual street smarts, and that sharing information with someone like that will only bite you in the butt. If you discover that someone like DG exists and is selling an interpretation of your data, the best thing you can do is buy that report for yourself so that you know what information is out there that you’ll have to counter one day. My irritation with DG? He didn’t make this data available to anyone who can afford a flat fee (which could be ridiculously high). Instead, he set yet another nosy bar, gathering even more information he has no right to.”
    In another comment she talks about perception and how it is important for an independent author to understand or control how they are perceived.

Comments are closed.