Disruptive Innovation

The Rise of Robot Authors: Is the Writing on the Wall for Human Novelists?

26 March 2019

From The Guardian:

Will androids write novels about electric sheep? The dream, or nightmare, of totally machine-generated prose seemed to have come one step closer with the recent announcement of an artificial intelligence that could produce, all by itself, plausible news stories or fiction. It was the brainchild of OpenAI – a nonprofit lab backed by Elon Musk and other tech entrepreneurs – which slyly alarmed the literati by announcing that the AI (called GPT2) was too dangerous for them to release into the wild, because it could be employed to create “deepfakes for text”. “Due to our concerns about malicious applications of the technology,” they said, “we are not releasing the trained model.” Are machine-learning entities going to be the new weapons of information terrorism, or will they just put humble midlist novelists out of business?

. . . .

GPT2 is just using methods of statistical analysis, trained on huge amounts of human-written text – 40GB of web pages, in this case, that received recommendations from Reddit readers – to predict what ought to come next. This probabilistic approach is how Google Translate works, and also the method behind Gmail’s automatic replies (“OK.” “See you then.” “That’s fine!”) It can be eerily good, but it is not as intelligent as, say, a bee.

Right now, novelists don’t seem to have much to fear. Fed the opening line of George Orwell’s Nineteen Eighty-Four – “It was a bright cold day in April, and the clocks were striking thirteen” – the machine continued the narrative as follows: “I was in my car on my way to a new job in Seattle. I put the gas in, put the key in, and then I let it run. I just imagined what the day would be like. A hundred years from now. In 2045, I was a teacher in some school in a poor part of rural China. I started with Chinese history and history of science.”

. . . .

Did the AI do any better with Jane Austen? The opening phrase of Pride and Prejudice – “It is a truth universally acknowledged” – provoked the machine to gabble on: “that when a nation is in a condition of civilization, that it is in a great measure the business of its leaders to encourage the habits of virtue, and of industry, and of good order among its people.” This does sound rather like some 19th-century political bloviator, even if a slightly broken version. (The second “that” is redundant, and it should read “in great measure” without the indefinite article.)

. . . .

Is there greater cause to worry further down the literary food chain? There have for a while already been “AI bots” that can, we hear, “write” news stories. All these are, though, are giant automated plagiarism machines that mash together bits of news stories written by human beings. As so often, what is promoted as a magical technological advance depends on appropriating the labour of humans, rendered invisible by AI rhetoric. When a human writer commits plagiarism, that is a serious matter. But when humans get together and write a computer program that commits plagiarism, that is progress.

. . . .

The makers’ announcement that this program is too dangerous to be released is excellent PR, then, but hardly persuasive. Such code, OpenAI warns, could be used to “generate misleading news articles”, but there is no shortage of made-up news written by actual humans working for troll factories. The point of the term “deepfakes” is that they are fakes that go deeper than prose, which anyone can fake. Much more dangerous than disinformation clumsily written by a computer are the real “deepfakes” in visual media that respectable researchers are eagerly working on right now. When video of any kind can be generated that is indistinguishable from real documentary evidence – so that a public figure, for example, can be made to say words they never said – then we’ll be in a world of trouble.

. . . .

Perhaps a more realistic hope for a text-only program such as GPT2, meanwhile, is simply as a kind of automated amanuensis that can come up with a messy first draft of a tedious business report – or, why not, of an airport thriller about famous symbologists caught up in perilous global conspiracy theories alongside lissome young women half their age. There is, after all, a long history of desperate artists trying rule-based ruses to generate the elusive raw material that they can then edit and polish. The “musical dice game” attributed to Mozart enabled fragments to be combined to generate innumerable different waltzes, while the total serialism of mid-20th‑century music was an algorithmic approach that attempted as far as possible to offload aesthetic judgments by the composer on to a system of mathematical manipulations.

. . . .

But until robots have rich inner lives and understand the world around them, they won’t be able to tell their own stories. And if one day they could, would we even be able to follow them? As Wittgenstein observed: “If a lion could speak, we would not understand him”. Being a lion in the world is (presumably) so different from being a human in the world that there might be no points of mutual comprehension at all. It’s entirely possible, too, that if a conscious machine could speak, we wouldn’t understand it either.

Link to the rest at The Guardian

PG says, “We have a lot of rain in June. Is the buzz dead better than the couple? The maddening kill crawls into the wealthy box. When does the zesty liquid critique the representative?”

(PG’s comments are courtesy of Random Word Generator, TextFixer and Word Generator.

And also:

After leaving the crumpled planet Abydos, a group of girls fly toward a distant speck. The speck gradually resolves into a contented, space tower.

Civil war strikes the galaxy, which is ruled by Brad Willis, a derelict wizard capable of lust and even murder.

Terrified, an enchanted alien known as Michelle Thornton flees the Empire, with her protector, Chloe Noris.

They head for Philadelphia on the planet Saturn. When they finally arrive, a fight breaks out. Noris uses her giant knife to defend Michelle.

(Plot Generator)

Finally, a blurb for a romance novel:

In this story, a serene police chief ends up on the run with a realistic witch-hunter. What starts as professional courtesy unexpectedly turns into a passionate affair.

(Seventh Sanctum)

Click here, then click the Play button to listen to the blurb

(Natural Readers)

Our Software Is Biased like We Are. Can New Laws Change That?

24 March 2019

From The Wall Street Journal:

Lawyers for Eric Loomis stood before the Supreme Court of Wisconsin in April 2016, and argued that their client had experienced a uniquely 21st-century abridgment of his rights: Mr. Loomis had been discriminated against by a computer algorithm.

Three years prior, Mr. Loomis was found guilty of attempting to flee police and operating a vehicle without the owner’s consent. During sentencing, the judge consulted COMPAS (aka Correctional Offender Management Profiling for Alternative Sanctions), a popular software system from a company called Equivant. It considers factors including indications a person abuses drugs, whether or not they have family support, and age at first arrest, with the intent to determine how likely someone is to commit a crime again.

The sentencing guidelines didn’t require the judge to impose a prison sentence. But COMPAS said Mr. Loomis was likely to be a repeat offender, and the judge gave him six years.

An algorithm is just a set of instructions for how to accomplish a task. They range from simple computer programs, defined and implemented by humans, to far more complex artificial-intelligence systems, trained on terabytes of data. Either way, human bias is part of their programming. Facial recognition systems, for instance, are trained on millions of faces, but if those training databases aren’t sufficiently diverse, they are less accurate at identifying faces with skin colors they’ve seen less frequently. Experts fear that could lead to police forces disproportionately targeting innocent people who are already under suspicion solely by virtue of their appearance.

. . . .

No matter how much we know about the algorithms that control our lives, making them “fair” may be difficult or even impossible. Yet as biased as algorithms can be, at least they can be consistent. With humans, biases can vary widely from one person to the next.

As governments and businesses look to algorithms to increase consistency, save money or just manage complicated processes, our reliance on them is starting to worry politicians, activists and technology researchers. The aspects of society that computers are often used to facilitate have a history of abuse and bias: who gets the job, who benefits from government services, who is offered the best interest rates and, of course, who goes to jail.

“Some people talk about getting rid of bias from algorithms, but that’s not what we’d be doing even in an ideal state,” says Cathy O’Neil, a former Wall Street quant turned self-described algorithm auditor, who wrote the book “Weapons of Math Destruction.”

“There’s no such thing as a non-biased discriminating tool, determining who deserves this job, who deserves this treatment. The algorithm is inherently discriminating, so the question is what bias do you want it to have?” she adds.

. . . .

An increasingly common algorithm predicts whether parents will harm their children, basing the decision on whatever data is at hand. If a parent is low income and has used government mental-health services, that parent’s risk score goes up. But for another parent who can afford private health insurance, the data is simply unavailable. This creates an inherent (if unintended) bias against low-income parents, says Rashida Richardson, director of policy research at the nonprofit AI Now Institute, which provides feedback and relevant research to governments working on algorithmic transparency.

The irony is that, in adopting these modernized systems, communities are resurfacing debates from the past, when the biases and motivations of human decision makers were called into question. Ms. Richardson says panels that determine the bias of computers should include not only data scientists and technologists, but also legal experts familiar with the rich history of laws and cases dealing with identifying and remedying bias, as in employment and housing law.

Link to the rest at The Wall Street Journal

Boom Time for Used Booksellers?

19 February 2019

As PG was opening a couple of packages of hardcopy books for Mrs. PG (she does read a lot of ebooks, but, in some cases, used books are less expensive and some books she wants in hardcopy to share with family and/or friends), it occurred to him that Amazon has almost certainly given used booksellers an opportunity to reach a far wider group of prospective purchasers than were ever available to them in physical used bookstores.

Most of the hardcopy used books that arrive in the mail come well-packaged and most are clearly packed by more sophisticated equipment than a roll of stamps and a stack of envelopes.

So, is PG correct about Amazon and used booksellers?

Has the ability to sell to a much wider online audience affected the pricing of used books?

Has the used book business undergone consolidation with small used bookstores closing and selling their inventory to large, online-focused used booksellers?

Are there people who are paid by larger used booksellers to be scouts for large quantities of available used books?

Real-Time Continuous Transcription with Live Transcribe

5 February 2019

Not necessarily to do with books, but two of PG’s offspring are hearing-impaired, so he follows topics like this. He’s also interested in developments in artificial intelligence, so it’s a double win for him.

From The Google AI Blog:

The World Health Organization (WHO) estimates that there are 466 million people globally that are deaf and hard of hearing. A crucial technology in empowering communication and inclusive access to the world’s information to this population is automatic speech recognition (ASR), which enables computers to detect audible languages and transcribe them into text for reading. Google’s ASR is behind automated captions in Youtube, presentations in Slides and also phone calls. However, while ASR has seen multiple improvements in the past couple of years, the deaf and hard of hearing still mainly rely on manual-transcription services like CART in the US, Palantypist in the UK, or STTRin other countries. These services can be prohibitively expensive and often require to be scheduled far in advance, diminishing the opportunities for the deaf and hard of hearing to participate in impromptu conversations as well as social occasions. We believe that technology can bridge this gap and empower this community.

Today, we’re announcing Live Transcribe, a free Android service that makes real-world conversations more accessible by bringing the power of automatic captioning into everyday, conversational use. Powered by Google Cloud, Live Transcribe captions conversations in real-time, supporting over 70 languages and more than 80% of the world’s population. You can launch it with a single tap from within any app, directly from the accessibility icon on the system tray.

Link to the rest at The Google AI Blog


The Future of Music, Where Middlemen Have Met Their Match

4 January 2019

From OZY:

“Hey, Dad. I want to show you a song.”

The speaker was my 16-year-old daughter. Music for her? Primarily visual and to be enjoyed in video clips. Video clips that did not always feature videos. Sometimes it was just some clip art and the music. But no record store, no record album, no tape — reel-to-reel, eight track, cassette or otherwise — and finally no compact disc. And she’s not alone in how she’s digging on the music she digs on.

According to Nielsen’s music report, digital and physical album sales declined (again) last year — from about 205 million in 2016 to 169 million copies in 2017 — down 17 percent. Over the past five years, right up to Nielsen’s mid-year report, sales had fallen by roughly 75 percent. That decline is coinciding with a streaming juggernaut that continues to grow. How much so? Last year streaming skated, quite easily, beyond 400 billion streams. You include video streams and you have figures over $618 billion. You look back at the year before and you see a 58 percent increase in audio streams.

While this buoyed the damned-near-moribund music industry to the tune of 12.5 percent growth from 2016 to last year, the music business is now, as it has been, all about discovering the music that can generate all of those streams. And that’s where things get curious because record labels that are used to creating heat now have to go places where the heat is being created to stay viable and vibrant.

. . . .

With a number of presently high-profile artists — Odd Future, Lil Yachty, Post Malone, etc. — being “discovered” on places like SoundCloud over the past five years, entire communities of music fans can beat both the hype and the Spotify/Pandora/SiriusXM radio/Amazon algorithms that suggest if you liked this, you might also like that, by starting there, and branching out. First stop: Instagram.

“People come in all the time and play me stuff from their IG feeds,” says Mark Thompson, founder of Los Angeles-based Vacation Vinyl (that sells, yes, primarily vinyl). “So I’m hearing bands that it soon becomes pretty clear have no label, no representation, nothing but an IG feed and maybe some music recorded on their laptops.”

To put this in perspective, in July 2018, Instagram added the music mode in Stories, and just that quickly streaming started to feel … old. Because from the musicians’ mouths to our ears, unmediated music finds its way from the creator to the consumer. Spotify is trying to adapt too — it has over the past year begun to sign deals with independent musicians to give them access to the platform.

. . . .

“It’s free,” she says, having endured speeches about listening to unpaid/stolen music. Since she and her friends don’t ever listen to more than 60 seconds of any song, at least while I am around, this raises the question: Is it a business and is it sustainable in the same way that Apple Music, Tidal, Deezer or iHeartRadio have managed to be?

“Unknown,” says former promoter and music industry executive Mark Weiss. “But the business is where the ears are. And if the business is any damn good it’ll figure out how to stay in the conversation.”

. . . .

Flash-forward to record contracts from the mid-1990s that covered cassette tapes, vinyl, compact discs and “future technologies not yet known.” The digitization of analog music had already changed the landscape for everything from crime to interior design.

Whereas previously you’d have needed a turntable, an amplifier, maybe a preamp, a tape player, a receiver, speakers and a subwoofer to listen to the music that you’d be playing off of tapes, vinyl or CDs, after everything was digitized you just needed a phone and speakers.

Link to the rest at OZY

The Current State of Disruption (Planning for 2019 Part 1)

27 December 2018

From Kristine Kathryn Rusch:

 For years now, I’ve done a year-end review, examining what happened and where the industry stands.

. . . .

I wrote down lists and links and reviewed notes and thought long and hard about things…and still couldn’t figure out how to wrap my arms around what I wanted to talk about.

I initially thought about combining the different parts of the industry under topics, and examine the topic rather than that part of the industry. But the industry is diverging in some important ways, making that way of writing these blogs exceedingly difficult.

This afternoon, it struck me: I write the year-end reviews so that I can focus on what to expect from the year to come.

So rather than look in detail at what happened in 2018, I’ll be looking at what happened with an eye toward the future.

. . . .

A reminder: I write these weekly business blogs for other writers who want to make or already have a long-term career. If you’re just starting out, some of this stuff won’t apply to you. If you’re a hobbyist who never wants to quit your day job, again, some of this stuff won’t apply to you. Don’t ask me to bend the blog toward you. There are a number of sites that cater to the beginner or the writer who doesn’t really care if she makes a living.

. . . .

For the most part, however, dealing with beginner and hobbyist issues doesn’t interest me. I’m a long-term professional writer who has made money as a writer since I was 16, who has made a living at it since I was 25, and who started making a heck of a great living at it by the time I was 35. I started writing these weekly blogs to make some kind of sense out of the disruption in the publishing industry in 2009. I did it for me, because I think better when I am writing things down.

The disruption continues, albeit in a new phase (part of what I’ll discuss below), and so I am focusing on what I need to focus on for my long-term writing career. I hope that some of these insights will help the rest of you.

. . . .

The disruption in the publishing industry will continue for some time now. Years, most likely. I don’t have a good crystal ball for how long it will go on, but we are past the gold rush years in the indie publishing world and have moved into a more consistent business model. It’s at least predictable, now. We know some patterns and how they’re going to work.

. . . .

The disruption in traditional publishing has gone on for nearly two decades now. It began before the Kindle made self-publishing easy by giving writers an easily accessible audience. Traditional publishing became ripe for disruption in the 1990s when the old distribution model collapsed.

Many of you saw it from the outside—the decline of the small bookstore, the loss of bookstores in small towns, the rise of the bestseller only in chain bookstores. All of that came from a collapse in the distribution system, from hundreds of regional distributors down to about five. (I don’t off the top of my head recall the actual number.) That made publishers panic. They couldn’t figure out what kinds of books sold best in the Pacific Northwest as opposed to what sold well in the Southeast, and worse, they didn’t have time to figure it out.

(When I came into the business, a top sales person for a major book company would know that science fiction sold well in California and quest fantasy sold well in Georgia, that the Midwest really enjoyed regional books, while New Yorkers often didn’t.)

Bestsellers sold everywhere, so publishers ramped up the production of already-established authors and sent those books all over the nation. Then, when the crisis leveled out, the publishers did not return to the old ways, scared of what to do. They continued to push for huge sellers rather than grow newer books.

Writer after writer after writer got dumped by their publisher in this period, while some new writers made fortunes because they wrote books that were similar to existing bestsellers.

When the Kindle came around and disrupted publishing, both writers and readers were ready for something new. That combination of forces created the blockbuster indie sellers—which were not blockbuster to traditional publishers. (The writers were making significantly more money, but selling fewer units than trad pub bestsellers.)

Hold that thought for a moment while I remind you that another disruption—a different one—was hitting publishing at the same time. Audiobooks went digital, and exploded. It became easy to download an audiobook and listen to it on your iPod (remember those) or your favorite MP3 player. Some cars made it easy to hook up those players to the sound system of the car.

And thus, commuters wanted everything on audio, and the demand in audio grew exponentially. As so many industry analysts said five or six years ago, if the Kindle hadn’t come around, the big story in publishing would have been the audiobook.

And here’s another publisher problem: most publishers never secured audio rights to the books they published. That money went directly to the authors.

. . . .

For years now, those of us who watch business trends have predicted that book sales would plateau. In reality, “plateau” is the wrong word for overall book sales. Those continue to grow, sometimes in ways that aren’t entirely measurable. New markets are opening all the time, bringing in new readers.

The system for measuring both readers and sales is so inadequate that we can’t count the readers we have, let alone the new readers who are coming into the book industry sideways. However, there is a lot of evidence—scattered, of course—that new readers are coming in. (I’ll deal with this in future weeks.)

Readership is growing, but individual sales are mostly declining. Traditional publishing’s fiction sales are down 16% since 2013. Traditional publishing has a lot of theories about this, delineated out in the Publishers Weekly article I linked to.

Indie writers believe a lot of the trad pub sales migrated to them. Maybe.

But some of what happened here was the inevitable decline from the gold rush of a disruptive technology.

Let’s look at traditional publishing for a moment. Traditional publishing moved to the blockbuster model at the turn of the century, meaning that the books that were published had to have a guaranteed level of sales or the author’s contract wouldn’t be renewed. The sales rose, partly because traditional publishing was the only game in town.

In that period, if you went to bookstores all over the country, and followed that up with a visit to the grocery store, as well as a visit to a story like WalMart or Target, you’d find the same group of books on the shelves. A few more in Target than in the grocery store, and certainly more in the bookstore, but still, the same books. And the airport bookstores were the same way.

If a reader needed reading material, he only had a few hundred titles at any given time in the stores to choose from. So the reader read the best of what he found, not necessarily what he wanted to read.

Then the disruption happened. Kindles and ereaders proliferated. Readers found books they’d been searching for, often for years. The readers also found some genres and subgenres that they hadn’t seen in a decade or more, usually books by indie writers that oculdn’t sell to the big traditional companies.

The boom in ebooks grew and grew and grew. (And if traditional pubishing hadn’t dicked around with pricing, their book sales would have grown even more.) That’s why the S-curves on that graph grow precipitously in between Stages Two and Three. Adoption increases revenue for a very very very short period of time.

That kind of growth is not sustainable for years, though. That’s why I say it was an inevitable plateau. If you’ll look on that graph again, though, you’ll see that both curves end higher on the y-axis—the profit axis—than they were at the beginning.

But hitting that plateau after years of rapid growth and, in the case of traditional publishing, a near-monopoly on the market, is painful. And that’s what we’re experiencing.

Also, sales are spreading out. I’ll talk about this a bit more in the next couple of weeks. But think of it this way. Instead of a lot of readers reluctantly reading the latest blockbuster because they’re trapped in the airport and can’t find anything else to read, those readers are now downloading dozens of books on their phones, and reading a variety of things—some of which we don’t have measurements of. Those readers have left the blockbusters they barely liked behind and found books/authors they like better.

So the money that would have gone to five different authors at three different publishing companies is now going to twenty authors, and only two of those authors are with traditional publishing companies. The books the readers are reading, though, aren’t the latest blockbuster by that author, but an older book that came out a decade ago. The price is lower, and the companies aren’t interested in those sales. They want the newest book to sell the most copies.

The consumer spends the same amount of money, but spreads it out over a wider range. Many of these sales are untrackable. Not all of those twenty authors report their sales to anyone, and not all of those sales were made through traditional channels. A few of the authors sold on their own websites. Some of those books came out of bundles. And some came out of a subscription service like Amazon. The traditional publishing companies lost most of the revenue, because their book sales have legitimately declined.

But that doesn’t mean people are reading less or that fiction reading is declining.

I’m not the only one who sees this. Mark Williams of The New Publishing Standard had the same reaction to the traditional publishing fiction numbers that I did. He wrote on November 18:

The big problem we have is that the fiction market, much more so than the wider book market, is so fragmented now, thanks to digital (by which I mean not just ebooks and audiobooks but online POD and most of all social media democratising the promotion of fiction titles), such that it seems like fewer people are reading fiction, but the reality is likely just the opposite.

The fragmented market is but one thing we’ll talk about in the next few weeks. We’ll look at how writers can use that market to their own advantage.

Link to the rest at Kristine Kathryn Rusch

PG always appreciates the analysis Kris and Dean bring to the publishing world, traditional and indie. He was going to add a few of his thoughts to Kris’ excellent post, but, perhaps as a result of holiday hangover (not the alcoholic kind), his little gray cells are not as well-regimented as usual.

Here’s a link to Kris Rusch’s books. If you like the thoughts Kris shares, you can show your appreciation by checking out her books.

Here is the most recent Kris Rusch book selling on Amazon:

Why I Left My Big Fancy Tech Job and Wrote a Book

8 October 2018

From Medium:

Several years ago, I was sitting in the audience at a big tech conference, learning about a startup that made it easy for people to rent rooms in other people’s houses for short stays. In a world where people can now travel to any part of the world and share someone else’s home, could we hope, the CEO asked, for greater cross-cultural understanding? “Would nations have less war if the residents lived together?”

I closed my eyes, breathed deeply, and felt an immense sense of peace and hope for humanity wash over me.

Then I opened my eyes and thought, “Isn’t this basically a hotel in someone’s house — a cool, convenient, unregulated hotel?”

When it was my turn to take the stage, I too had a grandiose proclamation: Our startup, I declared, was helping people make meaningful connections in the real world.

What I really should have said was: We help people hook up.

On the plane ride home, I began to write what would eventually become The Big Disruption, a satirical novel based on my experience working at both a startup and one of the biggest tech companies in the world. I had no goal at the time other than to provide a bit of cathartic escape from the tech industry, where, on the surface, things seemed really important and exciting.

We were doing big things!

Bringing the internet to the developing world!

Singing songs to orphans!

But also, on some level, it all felt a bit off.

So, where to begin?

. . . .

To be sure, Silicon Valley has built some great products that have truly changed our lives for the better. And I do think that in many, many ways, it has taken noble stands during difficult times and helped redefine what people expect from companies, well beyond just the tech industry. It has also led me to some of my best friends and greatest opportunities, for which I am very grateful. There is so much I really do love about this world.

But there is also what drove me to leave the big tech company last fall and take a break. The issues that I got tired of defending at parties. The endless use of “scale” as an excuse for being unable to solve problems in a human way. The faux earnestness, the self-righteousness. All those cheery product ads set to ukulele music.

I wrote this book for two reasons. First, I wanted to explore what drives the insatiable expansion of the big tech companies. Despite how the industry is sometimes portrayed in the media, I don’t really think the management teams at Facebook, Google, Apple, Uber, or Amazon wake up each morning thinking about how to steal more user data or drive us all out of our jobs. Those are real consequences, but not the root cause. Rather, it’s the desperation to stay on top and avoid being relegated to a dusty corner of the Computer History Museum that pushes these companies into further and further reaches of our lives.

Second, I wrote this book because we should be able to love and celebrate the products that we build — but without ignoring the hard questions they raise. We need to end the self-delusion and either fess up to the reality we are creating or live up to the vision we market to the world. Because if you’re going to tell people you’re their savior, you better be ready to be held to a higher standard. This book is my small way of trying to push us all to be better. Meaning…

You can’t tell your advertisers that you can target users down to the tiniest pixel but then throw your hands up before the politicians and say your machines can’t figure out if bad actors are using your platform.

. . . .

You can’t buy up a big bookstore and then a big diaper store and a big pet supply store and, finally, a big grocery store, national newspaper, and rocket ship and then act surprised when people start wondering if maybe you’re a bit too powerful.

And you can’t really claim that you’re building for everyone in the world when your own workforce doesn’t remotely resemble the outside world.

Link to the rest at Medium

PG notes that this book was published on Medium. Click Here to read in a Medium App or on the web.


Data, Algorithms & Authorship in the 21st Century

5 October 2018

From SSRN (footnotes omitted, a few paragraph breaks added):

“Data is the new gold. It’s the new oil. It’s the new plastics.”
— Mark Cuban, 2017

Over the last decade the music, motion picture, and publishing industries have faced what many have characterized as a crisis. Online piracy and the digital technologies that enable it are said to have destroyed traditional models of content creation and distribution.

The music industry is most often offered as the leading example. In the nearly two decades since the digital file-sharing service Napster burst on the scene, recording company revenues have plunged by approximately 72% in the U.S., or almost 80% adjusted for inflation.

A great deal of that decline in revenue can be traced to the ability to distribute and share content digitally without either legal permission or much chance of consequence.

The story appears to be dire, and yet it is increasingly obvious that the crisis narrative obscures more than it reveals. To be sure, the shift to digital and the related upsurge in online piracy — a phenomenon we refer to here as the “first digital disruption” — dramatically re-organized power within the music industry and transformed the ways in which the industry does business and makes (or does not make) money. But the industry adjusted, and the disruption did not fundamentally change the way music is created.

The first digital disruption mainly undermined a particular set of music industry business models. Most of the impact fell on middlemen (record labels, publishing companies, and retailers) who saw their revenues sink. And even there, the story has been as much about creation as disruption. Record labels, formerly the dominant force in the industry, are much diminished today.

But streaming services, such as Spotify, Apple Music, and Tidal, once tiny, are now important players. Turning the destructive potential of digital distribution on its head, they have utilized the internet to pioneer new and lucrative modes of content dissemination. Indeed, the total revenue of digital distributors now exceeds the total revenues of recording companies.

The U.S. live music industry has also grown substantially, and is expected to continue to grow at about twice the rate of the overall economy.  And even as record company revenues have shrunk, the best evidence suggests that more music is being produced than ever before.

On the other side of the market, consumers pay less, and have more access to, that cornucopia of music than ever before.

The next digital disruption is going to reach deeper. It will re-order how creative work is produced, and not simply how it is promoted and sold. It will transform our notions of authorship. It will raise fundamental questions about the nature and value of human creativity. And, perhaps less consequentially for the world at large — but of central importance to lawyers — it may shift how we think about the the value and utility of, and even the moral justification for, intellectual property rules.

What is this second digital disruption? We can see its onset in the high-stakes merger between AT&T, which owns digital cable and satellite networks for distributing video programming, and Time Warner, which produces film and television content. The Department of Justice challenged the merger, arguing that it would harm competition in video programming and distribution markets. In its pre-trial brief, Time Warner argued for the merger by noting that, as a stand-alone content producer it faced a competitive disadvantage versus rivals, such as Netflix, Google, and Facebook, that produce content but also own a digital distribution platform. As Time Warner argued:

First, unlike Google and Facebook, Time Warner has no access to meaningful data about its customers and their needs, interests, and preferences. In most cases, Time Warner does not even know its viewers’ names. This data gap impedes its ability to compete with Google, Facebook, and other digital companies in advertising sales, which are critical to Turner [Broadcasting (the owner of Time Warner]’s viability, and which allow Turner to keep subscription fees much lower than they otherwise would be. Whereas digital companies have the data and the technology to deliver advertisements that are both specifically addressed (shown) to a particular viewer and tailored to that viewer’s specific needs and interests, Time Warner cannot target its television advertising in those ways, creating an increasing competitive disadvantage for the company. The data gap also gives online video programmers a competitive advantage in the production and aggregation of content based on extensive data about the content preferences of their viewers.

This spring Judge Richard Leon of the United States District Court for the District of Columbia agreed, holding that “traditional programmers and distributors are experiencing increased competition from innovative, over-the-top content services [i.e., companies that provide video programming over the Internet] …. Those web-based companies are harnessing the power of the internet and data to provide lower-cost, better-tailored programming content directly to consumers. The dramatic growth of the leading [Internet video providers] in particular, including Netflix, Hulu, and Amazon Prime, can be traced in part to the value conferred by vertical integration — that is, to having content creation and aggregation as well as content distribution under the same roof.”

Data is at the core of the second digital disruption. In Mark Cuban’s words, data is “the new gold”: the resource that will create, and likely destroy, fortunes in the content business.

The “data gap” Time Warner spoke of is not just a competitive disadvantage for firms that produce many different types of creative content. Access to data about consumer preferences is rapidly becoming a competitive necessity, and the inability to gather such data, on a massive scale, is a fundamental disability.

Increasingly, we will see the rise of firms that own large and even dominant digital distribution platforms but also produce content for those platforms. Indeed, this trend is visible already. Netflix, Amazon, and, not yet but perhaps soon, Spotify, use the data they collect on consumer preferences and usage to make decisions about advertisements. All now use this data to decide how to organize and recommend content to users.

And some use their data to produce content that is more effectively targeted to consumer preferences. It is this last twist — the use of data to shape content creation, which we refer to as “data-driven authorship” — that is ultimately the most interesting feature of this new model.

Link to the rest at SSRN

PG says indie authors are conducting a variation on the concept in the OP with increasingly sophisticated salting of key words within their promotional materials in order to attract the types of people who will want to purchase their books.

One example is the more frequent use of author or title comparisons in book descriptions, such as, “If you like Penelope Blunderbuss, you’ll love ________”

When Amazon’s algorithms are trying to present books a reader will want to purchase, if that reader has just finished a book by Penelope, the algorithms may bump a book that includes Penelope’s name up near the top of its suggestions for that reader.

This is the great, great, great grand-descendant of Search Engine Optimization, first used by PG about 15 years ago to push his company’s products higher in the Google search results when people searched for those products.

Search algorithms have become enormously more sophisticated during the intervening years, particularly at Amazon, where they know both what you’ve searched for and what you’ve purchased, but the first principle of a successful search engine – show the customer what the customer wants to see – hasn’t changed.

Next Page »