The Antitrust Case Against Facebook, Google, Amazon and Apple

16 January 2018

From The Wall Street Journal:

Standard Oil and Co. and American Telephone and Telegraph Co. were the technological titans of their day, commanding more than 80% of their markets.

Today’s tech giants are just as dominant: In the U.S., Alphabet Inc.’s Google drives 89% of internet search; 95% of young adults on the internet use a Facebook Inc. product; and Inc. now accounts for 75% of electronic book sales. Those firms that aren’t monopolists are duopolists: Google and Facebook absorbed 63% of online ad spending last year; Google and Apple Inc. provide 99% of mobile phone operating systems; while Apple and Microsoft Corp. supply 95% of desktop operating systems.

A growing number of critics think these tech giants need to be broken up or regulated as Standard Oil and AT&T once were. Their alleged sins run the gamut from disseminating fake news and fostering addiction to laying waste to small towns’ shopping districts. But antitrust regulators have a narrow test: Does their size leave consumers worse off?

By that standard, there isn’t a clear case for going after big tech—at least for now. They are driving down prices and rolling out new and often improved products and services every week.

That may not be true in the future: if market dominance means fewer competitors and less innovation, consumers will be worse off than if those companies had been restrained. “The impact on innovation can be the most important competitive effect” in an antitrust case, says Fiona Scott Morton, a Yale University economist who served in the Justice Department’s antitrust division under Barack Obama.

. . . .

“Forty percent of Google search is local,” says Luther Lowe, the company’s head of public policy. “There should be hundreds of Yelps. There’s not. No one is pitching investors to build a service that relies on discovery through Facebook or Google to grow, because venture capitalists think it’s a poor bet.”

There are key differences between today’s tech giants and monopolists of previous eras. Standard Oil and AT&T used trusts, regulations and patents to keep out or co-opt competitors. They were respected but unloved. By contrast, Google and Facebook give away their main product, while Amazon undercuts traditional retailers so aggressively it may be holding down inflation. None enjoys a government-sanctioned monopoly; all invest prodigiously in new products. Alphabet plows 16% of revenue back into research and development; for Facebook it’s 21%—ratios far higher than other companies. All are among the public’s most loved brands, according to polls by Morning Consult.

Yet there are also important parallels. The monopolies of old and of today were built on proprietary technology and physical networks that drove down costs while locking in customers, erecting formidable barriers to entry. Just as Standard Oil and AT&T were once critical to the nation’s economic infrastructure, today’s tech giants are gatekeepers to the internet economy. If they’re imposing a cost, it may not be what customers pay but the products they never see.

. . . .

The story of AT&T is similar. It owed its early growth and dominant market position to Alexander Graham Bell’s 1876 patent for the telephone. After the related patents expired in the 1890s, new exchanges sprung up in countless cities to compete.

Competition was a powerful prod to innovation: Independent companies, by installing twisted copper lines and automatic switching, forced AT&T to do the same. But AT&T, like today’s tech giants, had “network effects” on its side.

“Just like people joined Facebook because everyone else was on Facebook, the biggest competitive advantage AT&T had was that it was interconnected,” says Milton Mueller, a professor at the Georgia Institute of Technology who has studied the history of technology policy.

Early in the 20th century, AT&T began buying up local competitors and refusing to connect independent exchanges to its long-distance lines, arousing antitrust complaints. By the 1920s, it was allowed to become a monopoly in exchange for universal service in the communities it served. By 1939, the company carried more than 90% of calls.

Though AT&T’s research unit, Bell Labs, became synonymous with groundbreaking discoveries, in telephone innovation AT&T was a laggard. To protect its own lucrative equipment business it prohibited innovative devices such as the Hush-a-Phone, which kept others from overhearing calls, and the Carterphone, which patched calls over radio airwaves, from connecting to its network.

After AT&T was broken up into separate local and long-distance companies in 1982, telecommunication innovation blossomed, spreading to digital switching, fiber optics, cellphones—and the internet.

Link to the rest at The Wall Street Journal

Why Amazon is the new Microsoft

16 January 2018

From TechConnect:

A few years ago, chatbots were supposed to take over as a leading way to interact with the internet. They would live on our phones and in our messaging apps. Whenever we needed anything, all we had to do was type out a question.

Things are turning out … differently.

Chatbots, bots, virtual assistants and agents are all about the conversational UI — about interacting with a computer through natural-language words and sentences.

The conventional wisdom used to be that the chatbot revolution would be driven by pre-emption, interjection and agency, as exemplified by Facebook M and Google Now.

Instead, the killer features are hands-free voice interaction and ubiquity — the main strengths of the Amazon Alexa platform.

. . . .

Facebook M is dead.

Facebook plans to close it’s M chatbot service on Jan. 19.

Facebook M, which launched in August 2015, was experimental, available to only 10,000 people in Silicon Valley.

When M first emerged, it was widely assumed to represent the future of how chatbots should and would work.

. . . .

Google Now is dead, too — sort of.

A few years ago, the conventional wisdom in tech circles was that Google Now was the most sophisticated virtual assistant.

Google Now was introduced in Android in the summer of 2012.

The best thing about Google Now was pre-emption: Display cards would pop up to alert you to things (rather than waiting for you to ask). Google Now used your location, calendar and, above all, Gmail messages to figure out what kind of help you needed, and it would try to give you that help with suggestion cards. One of its best tricks was to see on your calendar where you were going, check your current location, check the traffic between those locations, and give you advice about when to leave.

Meanwhile, the coolest feature of Google Assistant is interjection, which means it will pay attention to conversations in Allo and make suggestions based on the conversation.

Unfortunately, hardly anyone uses Allo, and so the amazing interjection powers of the Google Assistant are largely unknown and generally unused by the larger public.

. . . .

A couple of years ago, Amazon Alexa was considered to be the weakest and least sophisticated chatbot or virtual assistant on the market. (Oddly, MS-DOS and, later, Microsoft Windows initially had similar reputations.)

While agency, including the ability to buy things, was once assumed to be an important feature of a virtual assistant, it’s clear even for Alexa that buying things is secondary.

According to an Experian study last year, fewer than one-third of surveyed Echo owners have ever bought something through Alexa.

The vast majority of tasks involve setting a timer, playing a song, reading the news, checking the time — really, the most basic functions of a smartphone made convenient by voice interaction.

And yet Amazon is clearly dominating the space. This week’s CES showed that the industry is following Amazon’s lead.

Alexa appeared at the show inside projectors, ceiling lights, cars, glasses, showers, washing machines, earbuds, speakers — and even Windows 10 PCs.

. . . .

Amazon CEO Jeff Bezos this week became the world’s richest person, according to the Forbes list. Over the past few decades, that spot was normally occupied by former Microsoft CEO Bill Gates.

The symbolism is timely; it was at CES this week that Amazon became the new Microsoft.

Microsoft rose to dominance by controlling the operating system that the majority of people and businesses used.

Amazon is now doing something similar with Alexa. While Alexa isn’t even close to becoming as important as Windows, it is becoming the operating system of the post-PC, post-smartphone future.

The reason is very simple, and perfectly described by Sam Dolnick, who oversees digital initiatives at The New York Times. He said: “We are living in a world where the mobile phone is dominant, and audio, which doesn’t require your eyes or your hands, is the ultimate mobile medium.”

Link to the rest at TechConnect

Evaluation of Speech for the Google Assistant

8 January 2018

From the Google Research Blog:

Voice interactions with technology are becoming a key part of our lives — from asking your phone for traffic conditions to work to using a smart device at home to turn on the lights or play music. The Google Assistant is designed to provide help and information across a variety of platforms, and is built to bring together a number of products — including Google Maps, Search, Google Photos, third party services, and more. For some of these products, we have released specific evaluation guidelines, like Search Quality Rating Guidelines. However, the Google Assistant needs its own guidelines in place, as many of its interactions utilize what is called “eyes-free technology,” when there is no screen as part of the experience.

In the past we have received requests to see our evaluation guidelines from academics who are researching improvements in voice interactions, question answering and voice-guided exploration. To facilitate their evaluations, we are publishing some of the first Google Assistant guidelines. It is our hope that making these guidelines public will help the research community build and evaluate their own systems.

Creating the Guidelines
For many queries, responses are presented on the display (like a phone) with a graph, a table, or an interactive element, like you’d see for [weather this weekend].

But spoken responses are very different from display results, as what’s on screen needs to be translated into useful speech. Furthermore, the contents of the voice response are sometimes sourced from the web, and in those cases it’s important to provide the user with a link to the original source. While users looking at their mobile device can click through to read the original web page, an eyes free solution presents unique challenges. In order to generate the optimal audio response, we use a combination of explicit linguistic knowledge and deep learning solutions that allow us to keep answers grammatical, fluent and concise.

. . . .

Formulation: it is much easier to understand a badly formulated written answer than an ungrammatical spoken answer, so more care has to be placed in ensuring grammatical correctness.

Link to the rest at the Google Research Blog


Google Offers Hand to News Publishers

2 October 2017

From The Wall Street Journal:

Google is rolling out a package of new policies and services to help news publishers increase subscriptions, a move likely to warm its icy relationship with some of the biggest critics of its power over the internet.

Google said it will end this week its decade-old “first click free” policy that required news websites to give readers free access to articles from Google’s search results. The policy upset publishers that require subscriptions, believing it undercut their efforts to get readers to pay for news.

Google, a unit of Alphabet Inc., said it also plans tools to help increase subscriptions, including enabling users to log in with their Google passwords to simplify the subscription process and sharing user data with news organizations to better target potential subscribers.

With billions of people using its search, YouTube and other web properties, Google has an outsize influence on a wealth of industries and modern society.

. . . .

The new publisher rules are good news for the print industry, which has largely struggled to convert its business model to the internet as print advertising sales have plummeted in the digital age. Google and Facebook dominate the internet ad industry, and news organizations are increasingly reliant on those two tech giants for web traffic. Google says it drives 10 billion clicks a month to publishers’ sites.

Some newspapers even asked Congress this year to exempt them from antitrust laws so they could negotiate collectively with the tech giants.

. . . .

“We really recognize the transition to digital for publishers hasn’t been easy,” Google Chief Business Officer Philipp Schindler said in an interview. He said a strong news industry boosts the utility of Google search and helps Google’s ad business, which sells ads on news sites. “The economics are pretty clear: If publishers aren’t successful, we can’t be successful.”

. . . .

Kinsey Wilson, the former executive editor of USA Today who now advises New York Times Co. , said publishers must be careful about letting Google be the middleman to its readers. “Google can remove some friction,” he said, “but publishers have to stay vigilant.”

Link to the rest at The Wall Street Journal

Amazon Seeks to Defend Alexa’s Lead as Competition Heats Up

1 September 2017

From The Wall Street Journal: Inc.  is pouring more resources into Alexa to maintain its edge as competition heats up among artificial-intelligence assistants, according to people familiar with the company’s thinking.

Amazon is adding hundreds of engineers to the Alexa program and giving it hiring preference over other divisions, the people said. It has also put Tom Taylor, a veteran Amazon executive known for scaling high-growth operations, in charge of the business, after the former Alexa chief retired.

Alexa powers Amazon’s Echo speaker device, which was the first of its kind when it was launched nearly three years ago. The Echo has about three-quarters of the U.S. market for smart speakers, with more than 11 million total devices sold through the end of last year, according to analyst estimates.

. . . .

On Wednesday, Amazon said it was teaming up with Microsoft to allow their voice-enabled digital assistants to work together, a move analysts said would help boost Alexa’s content. But the two won’t be sharing data.

Amazon draws its user data primarily from its retail website, which gives it a big advantage when it comes to shopping and other related machine learning, and its Echo devices.

Google gathers data from its widely used search engine in addition to its Android operating system for smartphones. It is directing its voice assistant at Amazon’s stronghold, e-commerce, with the partnership it announced with Wal-Mart Stores Inc. last week. Users of its Google Express shopping service will be able to order from the retail giant by voice via Google’s virtual assistant.

. . . .

Amazon also integrated Alexa with outside developers and products, adding more than 15,000 skills that allow consumers to ask Alexa to digitally shake a Magic 8-ball or turn on the lights. Alexa has also been added to devices ranging from Ford Motor Co. cars to Sears Holdings Corp.’s Kenmore refrigerators, which Google is also trying to do.

Still, Amazon needs Alexa to keep getting smarter. Josh Vickerson picked up a Dot late last year when it was on sale for $35. The 24-year-old originally tried out features like playing Jeopardy and integrating his Fitbit . Now he uses it to play videos, set timers and turn off the lights.

Link to the rest at The Wall Street Journal

Casa PG obtained its first Echo (which is currently marked down to $99) not long after it was first released. Since then, a flock of little Dots (also marked down) have shown up.

PG doesn’t claim to be a power user, but does have enough Dots that one is always within easy reach of his voice. They’re turning out to be very helpful and PG uses them for far fewer services than they can provide.

PG understands that his Echo and Dots will soon be able to control Casa PG’s Sonos speakers (very highly recommended). That will be wonderful. PG probably needs to buy a couple more Dots so whenever he whispers a command, Sonos starts playing.

Are the tech giants too big to be good partners for book publishing?

22 August 2017

From veteran publishing consultant Mike Shatzkin:

An online discussion forum that includes publishers and librarians and tech people usually sends me several emails a day. About 10 days ago, a conversation evolved about Google Book Search and the Google Library Project, two initiatives by the search giant that were initiated in the early part of the last decade.

Because both programs essentially gave Google a trove of book-published content for full text search, there was a wariness among the publishing community about them when they started. In time, publishers (through the AAP) sued Google and the course of the lawsuit ultimately led to a sharp curtailment of Google’s ability to just do the scanning. After a while, it appears the reservoir of interest at Google for the project, which started as more of a “service to humanity” idea than a profitable one, just evaporated. The scans that Google had already done became part of the HathiTrust repository of content, an important research and scholarship tool in the non-trade world without any recognition or impact on the trade world at all.

. . . .

And, of course, Google is the single most powerful source of “discovery” and many in publishing wonder if books overall would have benefited from Google being more “knowledgeable” about what is inside of them.

So, to this day, years after the litigation and the scanning program have concluded, there is a division of opinion in the publishing community. Some see Google as a bully and a villain, trying to make its own rules to benefit from publishers’ content and crippling the value of copyright. Others focus on the lost opportunity and believe publishers would actually have more valuable intellectual property (more valuable copyrights!) today if they’d just allowed the Google programs to develop and flourish.

. . . .

In the course of the discussion, a very knowledgeable and experienced veteran of publishing across education, professional, and trade offered the comment that “Google is a terrible partner.” I asked him (offline from the group discussion; he’s a friend) to amplify that.

My points of context for Google weren’t in publishing; they were in tech. My own most extensive experiences with the big three tech companies that publishers dealt with — Amazon, Apple, and Google — was working out their participation at publishing conferences.

. . . .

What I saw was that Apple was the most uptight; it was hard to get speakers because messaging was so tightly controlled by upper management.

Amazon would sometimes be very agreeable, but primarily when they had an agenda: some program they wanted to get across or some point they wanted to make. So they were often cooperative, but very much on their terms to put across their message du jour. In general, they wouldn’t do panels or Q&As. They needed to control the conversation and skillfully avoided being pushed to publicly discuss anything they didn’t want to talk about. But they were often available and always interesting, and unlike Apple (in my experience), would engage with you honestly about their agenda.

. . . .

Google was, in my experience, by far the most open and accessible of the three companies. You could tell them you wanted speakers or panelists to cover one subject or another and you’d get directed to people who could help you. And Google employed a pretty fair number of ex-publishing people who were conversant about issues from a perspective that publishers could relate to.

. . . .

What my friend said in response to my inquiry, in which I had only mentioned Google, was, “Google, Apple, and Amazon are all bad partners. Ingram, Baker & Taylor, and Firebrand are good partners.”

So much for my contextual frame.

But grouping the three to me made the point that my context was what mattered. Ingram, Baker & Taylor, and Firebrand all make their living in the book business. Google, Apple, and Amazon have a financial stake in the book business that amounts to a small rounding error to their overall financial performance.

. . . .

For the entire life of the book business until about fifteen minutes ago, it was very much a free-standing industry. The only larger-than-the-industry enterprises it had to deal with were the Post Office and United Parcel Service. Our authors, designers, typesetters, printers, and, most important of all, customers to which we shipped directly (the wholesalers and retailers and libraries) were part of the publishers’ world. They depended on the publishers as much as the publishers depended on them.

Amazon was the first piece of evidence — and still the most important piece of evidence — that the old world has disappeared.  . . . . They sell more than half of the books for most publishers, but all the books they sell probably amount to less than 5 percent of their total margin. And while Penguin Random House may be in the neighborhood of half the consumer book sales overall, they wouldn’t amount to nearly that big a percentage of Amazon’s book sales because Amazon gets a disproportionate share of professional and other niche markets and thus from publishers who don’t compete at all with PRH in the consumer market.

And because Amazon has very intentionally created a whole massive pool of consumer books that nobody else has, through their own publishing and enabling independent authors.

Link to the rest at The Shatzkin Files

PG has had direct business/legal dealings and negotiations with Apple and Amazon over the last 15 years or so. For context, he has also had business negotiations with Microsoft, Oracle, Hewlett-Packard and Intel in the tech world plus every major investment bank in New York (Goldman Sachs, Morgan Stanley, etc., etc.), most of the large accounting firms plus Disney, American Express and a bunch of other big companies.

To be clear, this doesn’t mean PG knows everything about negotiating intellectual property partnerships and other deals with large organizations, but he does know some things about that subject.

PG definitely has not represented any large publishers in their dealings with large tech companies. He has, however, represented a lot of authors in their dealings with large publishers.

Speaking generally, large publishers are not cut out to be good partners for tech companies.

Publishers are simply too rigid in their business vision and very much focused on the short term (which is strange for organizations that license copyrights, which extend far into the future).

This short term outlook is substantially affected by the fact that the Big Five publishers are all owned and controlled by other and larger media conglomerates. Four of the Big Five are owned by large European publishing corporations that are not known for their commitment to innovation and could not be described as tech-savvy in any sense. The fifth Big Five publisher, Simon & Schuster, is owned by CBS.

Each of these media conglomerates is heavily focused on this quarter’s and this year’s income, expenses and profits. They’re not what anyone would call forward-looking or focused on the long term. If they think about the long term at all, they’re convinced it will not be much different than last quarter.

(PG worked for a major subsidiary of a very, very large international media conglomerate for three unhappy years and knows that of which he speaks.)

This means that if Google sends someone to talk to the President of a Big Five publisher, Google is talking to a middle-manager in a much larger business organization. The Big Five President can do pretty much whatever he/she wants to do with Barnes & Noble and Ingram (as long as it doesn’t have an adverse impact on profits), but cutting a strategic deal with Google is way, way out of his/her job description.

Organizations like Google, Apple and Amazon quickly become frustrated with organizations that are not able to move rapidly.

Amazon’s Alexa Has A Data Dilemma: Be More Like Apple Or Google?

15 July 2017

From Fast Company:

Devices like Amazon Echo could someday turn into a treasure trove for developers that make voice assistant skills, but first companies have to figure out where they draw the line when it comes to weighing data sharing against consumer privacy.

Now that dilemma is heating up: Citing three unnamed sources, The Information reported this week that Amazon is considering whether to provide full conversation transcripts to Alexa developers. This would be a major change from Amazon’s current policy in which the company only provides basic information—such as the total number of users, the average number of actions they’ve performed, and rates of success or failure for voice commands. Amazon declined to comment to The Information regarding the claims, but the change wouldn’t be unprecedented. Google’s voice assistant platform already provides full transcripts to developers.

The potential move by Amazon underscores how it is caught between two worlds with its Alexa assistant, especially in regards to privacy. By keeping transcripts to itself, Amazon can better protect against the misuse of its customers’ data and avoid concerns about eavesdropping. But because Alexa already gives developers the freedom to build virtually any kind of voice skill, their inability to see what customers are saying becomes a major burden.

. . . .

With Google Assistant, developers can view a transcript for any conversation with their particular skill. Uber, for example, can look at all recorded utterances from the moment you ask for a car until the ride is confirmed. (It can’t, however, see what you’ve said to other apps and services.) Google’s own documentation confirms this, noting that developers can request “keyboard input or spoken input from end user” during a conversation.

For developers, this data can be of immense utility. It allows them to find out if users are commonly speaking in the wrong syntax, or asking to do things that the developer’s voice skill doesn’t support.

. . . .

In terms of sharing data with developers, Apple’s Siri voice assistant is on the opposite side of the spectrum from Google. Developers who work with SiriKit get no information about usage from Apple, not even for basic things like how many people use voice commands to access an app, or which voice commands are most commonly used.

. . . .

But keep in mind that Siri’s approach to third-party development is entirely different from that of Google and Amazon. Instead of letting developers build any kind of voice application, Apple only supports third-party voice commands in a handful of specific domains, such as photo search, workouts, ride hailing, and messaging. And instead of letting those apps drive the conversation, Apple controls the back-and-forth itself. The apps merely provide the data and some optional on-screen information.

Because these apps don’t communicate with users directly, there’s no need for them to have conversation transcripts in the first place. Instead, Apple can look at what users are trying to accomplish and use that data to expand Siri on its own.

The downside to this approach is that Siri just isn’t as useful as other virtual assistants.

Link to the rest at Fast Company 

If PG lived in China, he would be inclined not to use Alexa.

Apple Is Manufacturing a Siri Speaker to Outdo Google and Amazon

31 May 2017

From Bloomberg:

 Apple Inc. is already in your pocket, on your desk and underneath your television. Soon, a device embossed with “Designed by Apple in California” may be on your nightstand or kitchen counter as well.

The iPhone-maker has started manufacturing a long-in-the-works Siri-controlled smart speaker, according to people familiar with the matter. Apple could debut the speaker as soon as its annual developer conference in June, but the device will not be ready to ship until later in the year, the people said.

The device will differ from Inc.’s Echo and Alphabet Inc.’s Google Homespeakers by offering virtual surround sound technology and deep integration with Apple’s product lineup, said the people, who requested anonymity to discuss products that aren’t yet public.

Introducing a speaker would serve two main purposes: providing a hub to automate appliances and lights via Apple’s HomeKit system, and establishing a bulwark inside the home to lock customers more tightly into Apple’s network of services. That would help combat the competitive threat from Google’s and Amazon’s connected speakers: the Home and Echo mostly don’t support services from Apple. Without compatible hardware, users may be more likely to opt for the Echo or Home, and therefore use streaming music offerings such as Spotify, Amazon Prime Music or Google Play rather than Apple Music.

Link to the rest at Bloomberg

PG says competition keeps competitors sharp and is great for consumers.

Tech’s Frightful Five: They’ve Got Us

12 May 2017

From The New York Times:

A few weeks ago, I bought a new television. When the whole process was over, I realized something incredible: To navigate all of the niggling details surrounding this one commercial transaction — figuring out what to buy, which accessories I needed, how and where to install it, and whom to hire to do so — I had dealt with only a single ubiquitous corporation: Amazon.

It wasn’t just the TV. As I began combing through other recent household decisions, I found that in 2016, nearly 10 percent of my household’s commercial transactions flowed through the Seattle retailer, more by far than any other company my family dealt with. What’s more, with its Echos, Fire TV devices, audiobooks, movies and TV shows, Amazon has become, for my family, more than a mere store. It is my confessor, my keeper of lists, a provider of food and culture, an entertainer and educator and handmaiden to my children.

. . . .

This is the most glaring and underappreciated fact of internet-age capitalism: We are, all of us, in inescapable thrall to one of the handful of American technology companies that now dominate much of the global economy. I speak, of course, of my old friends the Frightful Five: Amazon, Apple, Facebook, Microsoft and Alphabet, the parent company of Google.

The five are among the most valuable companies on the planet, collectively worth trillions.

. . . .

 [L]ast week I came up with a fun game: If an evil, tech-phobic monarch forced you to abandon each of the Frightful Five, in which order would you do so, and how much would your life deteriorate as a result?

. . . .

When I went through the thought experiment, I found that dropping the first couple of tech giants was pretty easy — but after that the process became progressively more unbearable. For me, Facebook was the first to go. I tend to socialize online using Twitter, Apple’s messaging system, and Slack, the office-chat app, so losing Mark Zuckerberg’s popular service (and its subsidiaries, Instagram, WhatsApp and Messenger) was not such a big deal.

Next, for me, was Microsoft, which I found slightly more difficult to quit. I don’t normally use any Windows devices, but Microsoft’s word-processing program, Word, is an essential tool for me, and I’d hate to lose it.

In third place, full of regrets: Apple. There’s nothing I use more than my iPhone, and close behind are my MacBook and iMac 5K, which may be the best computer I’ve ever owned. Abandoning Apple would prompt deep and truly annoying rearrangements in my life, including braving Samsung’s bad software. But I could do it, grudgingly.

Link to the rest at The New York Times

Torching the Modern-Day Library of Alexandria

27 April 2017

From The Atlantic:

You were going to get one-click access to the full text of nearly every book that’s ever been published. Books still in print you’d have to pay for, but everything else—a collection slated to grow larger than the holdings at the Library of Congress, Harvard, the University of Michigan, at any of the great national libraries of Europe—would have been available for free at terminals that were going to be placed in every local library that wanted one.

At the terminal you were going to be able to search tens of millions of books and read every page of any book you found. You’d be able to highlight passages and make annotations and share them; for the first time, you’d be able to pinpoint an idea somewhere inside the vastness of the printed record, and send somebody straight to it with a link. Books would become as instantly available, searchable, copy-pasteable—as alive in the digital world—as web pages.

It was to be the realization of a long-held dream. “The universal library has been talked about for millennia,” Richard Ovenden, the head of Oxford’s Bodleian Libraries, has said. “It was possible to think in the Renaissance that you might be able to amass the whole of published knowledge in a single room or a single institution.” In the spring of 2011, it seemed we’d amassed it in a terminal small enough to fit on a desk.

“This is a watershed event and can serve as a catalyst for the reinvention of education, research, and intellectual life,” one eager observer wrote at the time.On March 22 of that year, however, the legal agreement that would have unlocked a century’s worth of books and peppered the country with access terminals to a universal library was rejected under Rule 23(e)(2) of the Federal Rules of Civil Procedure by the U.S. District Court for the Southern District of New York.When the library at Alexandria burned it was said to be an “international catastrophe.” When the most significant humanities project of our time was dismantled in court, the scholars, archivists, and librarians who’d had a hand in its undoing breathed a sigh of relief, for they believed, at the time, that they had narrowly averted disaster.

. . . .

Google’s secret effort to scan every book in the world, codenamed “Project Ocean,” began in earnest in 2002 when Larry Page and Marissa Mayer sat down in the office together with a 300-page book and a metronome. Page wanted to know how long it would take to scan more than a hundred-million books, so he started with one that was lying around. Using the metronome to keep a steady pace, he and Mayer paged through the book cover-to-cover. It took them 40 minutes.

Page had always wanted to digitize books. Way back in 1996, the student project that eventually became Google—a “crawler” that would ingest documents and rank them for relevance against a user’s query—was actually conceived as part of an effort “to develop the enabling technologies for a single, integrated and universal digital library.” The idea was that in the future, once all books were digitized, you’d be able to map the citations among them, see which books got cited the most, and use that data to give better search results to library patrons. But books still lived mostly on paper. Page and his research partner, Sergey Brin, developed their popularity-contest-by-citation idea using pages from the World Wide Web.
By 2002, it seemed to Page like the time might be ripe to come back to books. With that 40-minute number in mind, he approached the University of Michigan, his alma mater and a world leader in book scanning, to find out what the state of the art in mass digitization looked like. Michigan told Page that at the current pace, digitizing their entire collection—7 million volumes—was going to take about a thousand years. Page, who’d by now given the problem some thought, replied that he thought Google could do it in six.. . . .He offered the library a deal: You let us borrow all your books, he said, and we’ll scan them for you. You’ll end up with a digital copy of every volume in your collection, and Google will end up with access to one of the great untapped troves of data left in the world. Brin put Google’s lust for library books this way: “You have thousands of years of human knowledge, and probably the highest-quality knowledge is captured in books.” What if you could feed all the knowledge that’s locked up on paper to a search engine?

By 2004, Google had started scanning. In just over a decade, after making deals with Michigan, Harvard, Stanford, Oxford, the New York Public Library, and dozens of other library systems, the company, outpacing Page’s prediction, had scanned about 25 million books. It cost them an estimated $400 million. It was a feat not just of technology but of logistics.

. . . .

The stations—which didn’t so much scan as photograph books—had been custom-built by Google from the sheet metal up. Each one could digitize books at a rate of 1,000 pages per hour. The book would lie in a specially designed motorized cradle that would adjust to the spine, locking it in place. Above, there was an array of lights and at least $1,000 worth of optics, including four cameras, two pointed at each half of the book, and a range-finding LIDAR that overlaid a three-dimensional laser grid on the book’s surface to capture the curvature of the paper. The human operator would turn pages by hand—no machine could be as quick and gentle—and fire the cameras by pressing a foot pedal, as though playing at a strange piano.

What made the system so efficient is that it left so much of the work to software. Rather than make sure that each page was aligned perfectly, and flattened, before taking a photo, which was a major source of delays in traditional book-scanning systems, cruder images of curved pages were fed to de-warping algorithms, which used the LIDAR data along with some clever mathematics to artificially bend the text back into straight lines.

. . . .

In August 2010, Google put out a blog post announcing that there were 129,864,880 books in the world. The company said they were going to scan them all.

Of course, it didn’t quite turn out that way. This particular moonshot fell about a hundred-million books short of the moon. What happened was complicated but how it started was simple: Google did that thing where you ask for forgiveness rather than permission, and forgiveness was not forthcoming. Upon hearing that Google was taking millions of books out of libraries, scanning them, and returning them as if nothing had happened, authors and publishers filed suit against the company, alleging, as the authors put it simply in their initial complaint, “massive copyright infringement.”

. . . .

As Tim Wu pointed out in a 2003 law review article, what usually becomes of these battles—what happened with piano rolls, with records, with radio, and with cable—isn’t that copyright holders squash the new technology. Instead, they cut a deal and start making money from it. Often this takes the form of a “compulsory license” in which, for example, musicians are required to license their work to the piano-roll maker, but in exchange, the piano-roll maker has to pay a fixed fee, say two cents per song, for every roll they produce. Musicians get a new stream of income, and the public gets to hear their favorite songs on the player piano. “History has shown that time and market forces often provide equilibrium in balancing interests,” Wu writes.

But even if everyone typically ends up ahead, each new cycle starts with rightsholders fearful they’re being displaced by the new technology. When the VCR came out, film executives lashed out. “I say to you that the VCR is to the American film producer and the American public as the Boston strangler is to the woman home alone,” Jack Valenti, then the president of the MPAA, testified before Congress. The major studios sued Sony, arguing that with the VCR, the company was trying to build an entire business on intellectual property theft. But Sony Corp. of America v. Universal City Studios, Inc. became famous for its holding that as long as a copying device was capable of “substantial noninfringing uses”—like someone watching home movies—its makers couldn’t be held liable for copyright infringement.

The Sony case forced the movie industry to accept the existence of VCRs. Not long after, they began to see the device as an opportunity. “The VCR turned out to be one of the most lucrative inventions—for movie producers as well as hardware manufacturers—since movie projectors,” one commentator put it in 2000.
It only took a couple of years for the authors and publishers who sued Google to realize that there was enough middle ground to make everyone happy. This was especially true when you focused on the back catalog, on out-of-print works, instead of books still on store shelves. Once you made that distinction, it was possible to see the whole project in a different light. Maybe Google wasn’t plundering anyone’s work. Maybe they were giving it a new life. Google Books could turn out to be for out-of-print books what the VCR had been for movies out of the theater.If that was true, you wouldn’t actually want to stop Google from scanning out-of-print books—you’d want to encourage it. In fact, you’d want them to go beyond just showing snippets to actually selling those books as digital downloads.. . . .

Those who had been at the table crafting the agreement had expected some resistance, but not the “parade of horribles,” as Sarnoff described it, that they eventually saw. The objections came in many flavors, but they all started with the sense that the settlement was handing to Google, and Google alone, an awesome power. “Did we want the greatest library that would ever exist to be in the hands of one giant corporation, which could really charge almost anything it wanted for access to it?”, Robert Darnton, then president of Harvard’s library, has said.

Darnton had initially been supportive of Google’s scanning project, but the settlement made him wary. The scenario he and many others feared was that the same thing that had happened to the academic journal market would happen to the Google Books database. The price would be fair at first, but once libraries and universities became dependent on the subscription, the price would rise and rise until it began to rival the usurious rates that journals were charging, where for instance by 2011 a yearly subscription to the Journal of Comparative Neurology could cost as much as $25,910.Although academics and library enthusiasts like Darnton were thrilled by the prospect of opening up out-of-print books, they saw the settlement as a kind of deal with the devil. Yes, it would create the greatest library there’s ever been—but at the expense of creating perhaps the largest bookstore, too, run by what they saw as a powerful monopolist. In their view, there had to be a better way to unlock all those books. “Indeed, most elements of the GBS settlement would seem to be in the public interest, except for the fact that the settlement restricts the benefits of the deal to Google,” the Berkeley law professor Pamela Samuelson wrote.

Link to the rest at The Atlantic and thanks to Valerie for the tip.

Next Page »