From The Wall Street Journal:

Google is rolling out a package of new policies and services to help news publishers increase subscriptions, a move likely to warm its icy relationship with some of the biggest critics of its power over the internet.

Google said it will end this week its decade-old “first click free” policy that required news websites to give readers free access to articles from Google’s search results. The policy upset publishers that require subscriptions, believing it undercut their efforts to get readers to pay for news.

Google, a unit of Alphabet Inc., said it also plans tools to help increase subscriptions, including enabling users to log in with their Google passwords to simplify the subscription process and sharing user data with news organizations to better target potential subscribers.

With billions of people using its search, YouTube and other web properties, Google has an outsize influence on a wealth of industries and modern society.

. . . .

The new publisher rules are good news for the print industry, which has largely struggled to convert its business model to the internet as print advertising sales have plummeted in the digital age. Google and Facebook dominate the internet ad industry, and news organizations are increasingly reliant on those two tech giants for web traffic. Google says it drives 10 billion clicks a month to publishers’ sites.

Some newspapers even asked Congress this year to exempt them from antitrust laws so they could negotiate collectively with the tech giants.

. . . .

“We really recognize the transition to digital for publishers hasn’t been easy,” Google Chief Business Officer Philipp Schindler said in an interview. He said a strong news industry boosts the utility of Google search and helps Google’s ad business, which sells ads on news sites. “The economics are pretty clear: If publishers aren’t successful, we can’t be successful.”

. . . .

Kinsey Wilson, the former executive editor of USA Today who now advises New York Times Co. , said publishers must be careful about letting Google be the middleman to its readers. “Google can remove some friction,” he said, “but publishers have to stay vigilant.”

Link to the rest at The Wall Street Journal

From The Wall Street Journal:

Amazon.com Inc.  is pouring more resources into Alexa to maintain its edge as competition heats up among artificial-intelligence assistants, according to people familiar with the company’s thinking.

Amazon is adding hundreds of engineers to the Alexa program and giving it hiring preference over other divisions, the people said. It has also put Tom Taylor, a veteran Amazon executive known for scaling high-growth operations, in charge of the business, after the former Alexa chief retired.

Alexa powers Amazon’s Echo speaker device, which was the first of its kind when it was launched nearly three years ago. The Echo has about three-quarters of the U.S. market for smart speakers, with more than 11 million total devices sold through the end of last year, according to analyst estimates.

. . . .

On Wednesday, Amazon said it was teaming up with Microsoft to allow their voice-enabled digital assistants to work together, a move analysts said would help boost Alexa’s content. But the two won’t be sharing data.

Amazon draws its user data primarily from its retail website, which gives it a big advantage when it comes to shopping and other related machine learning, and its Echo devices.

Google gathers data from its widely used search engine in addition to its Android operating system for smartphones. It is directing its voice assistant at Amazon’s stronghold, e-commerce, with the partnership it announced with Wal-Mart Stores Inc. last week. Users of its Google Express shopping service will be able to order from the retail giant by voice via Google’s virtual assistant.

. . . .

Amazon also integrated Alexa with outside developers and products, adding more than 15,000 skills that allow consumers to ask Alexa to digitally shake a Magic 8-ball or turn on the lights. Alexa has also been added to devices ranging from Ford Motor Co. cars to Sears Holdings Corp.’s Kenmore refrigerators, which Google is also trying to do.

Still, Amazon needs Alexa to keep getting smarter. Josh Vickerson picked up a Dot late last year when it was on sale for $35. The 24-year-old originally tried out features like playing Jeopardy and integrating his Fitbit . Now he uses it to play videos, set timers and turn off the lights.

Link to the rest at The Wall Street Journal

Casa PG obtained its first Echo (which is currently marked down to $99) not long after it was first released. Since then, a flock of little Dots (also marked down) have shown up.

PG doesn’t claim to be a power user, but does have enough Dots that one is always within easy reach of his voice. They’re turning out to be very helpful and PG uses them for far fewer services than they can provide.

PG understands that his Echo and Dots will soon be able to control Casa PG’s Sonos speakers (very highly recommended). That will be wonderful. PG probably needs to buy a couple more Dots so whenever he whispers a command, Sonos starts playing.

From veteran publishing consultant Mike Shatzkin:

An online discussion forum that includes publishers and librarians and tech people usually sends me several emails a day. About 10 days ago, a conversation evolved about Google Book Search and the Google Library Project, two initiatives by the search giant that were initiated in the early part of the last decade.

Because both programs essentially gave Google a trove of book-published content for full text search, there was a wariness among the publishing community about them when they started. In time, publishers (through the AAP) sued Google and the course of the lawsuit ultimately led to a sharp curtailment of Google’s ability to just do the scanning. After a while, it appears the reservoir of interest at Google for the project, which started as more of a “service to humanity” idea than a profitable one, just evaporated. The scans that Google had already done became part of the HathiTrust repository of content, an important research and scholarship tool in the non-trade world without any recognition or impact on the trade world at all.

. . . .

And, of course, Google is the single most powerful source of “discovery” and many in publishing wonder if books overall would have benefited from Google being more “knowledgeable” about what is inside of them.

So, to this day, years after the litigation and the scanning program have concluded, there is a division of opinion in the publishing community. Some see Google as a bully and a villain, trying to make its own rules to benefit from publishers’ content and crippling the value of copyright. Others focus on the lost opportunity and believe publishers would actually have more valuable intellectual property (more valuable copyrights!) today if they’d just allowed the Google programs to develop and flourish.

. . . .

In the course of the discussion, a very knowledgeable and experienced veteran of publishing across education, professional, and trade offered the comment that “Google is a terrible partner.” I asked him (offline from the group discussion; he’s a friend) to amplify that.

My points of context for Google weren’t in publishing; they were in tech. My own most extensive experiences with the big three tech companies that publishers dealt with — Amazon, Apple, and Google — was working out their participation at publishing conferences.

. . . .

What I saw was that Apple was the most uptight; it was hard to get speakers because messaging was so tightly controlled by upper management.

Amazon would sometimes be very agreeable, but primarily when they had an agenda: some program they wanted to get across or some point they wanted to make. So they were often cooperative, but very much on their terms to put across their message du jour. In general, they wouldn’t do panels or Q&As. They needed to control the conversation and skillfully avoided being pushed to publicly discuss anything they didn’t want to talk about. But they were often available and always interesting, and unlike Apple (in my experience), would engage with you honestly about their agenda.

. . . .

Google was, in my experience, by far the most open and accessible of the three companies. You could tell them you wanted speakers or panelists to cover one subject or another and you’d get directed to people who could help you. And Google employed a pretty fair number of ex-publishing people who were conversant about issues from a perspective that publishers could relate to.

. . . .

What my friend said in response to my inquiry, in which I had only mentioned Google, was, “Google, Apple, and Amazon are all bad partners. Ingram, Baker & Taylor, and Firebrand are good partners.”

So much for my contextual frame.

But grouping the three to me made the point that my context was what mattered. Ingram, Baker & Taylor, and Firebrand all make their living in the book business. Google, Apple, and Amazon have a financial stake in the book business that amounts to a small rounding error to their overall financial performance.

. . . .

For the entire life of the book business until about fifteen minutes ago, it was very much a free-standing industry. The only larger-than-the-industry enterprises it had to deal with were the Post Office and United Parcel Service. Our authors, designers, typesetters, printers, and, most important of all, customers to which we shipped directly (the wholesalers and retailers and libraries) were part of the publishers’ world. They depended on the publishers as much as the publishers depended on them.

Amazon was the first piece of evidence — and still the most important piece of evidence — that the old world has disappeared.  . . . . They sell more than half of the books for most publishers, but all the books they sell probably amount to less than 5 percent of their total margin. And while Penguin Random House may be in the neighborhood of half the consumer book sales overall, they wouldn’t amount to nearly that big a percentage of Amazon’s book sales because Amazon gets a disproportionate share of professional and other niche markets and thus from publishers who don’t compete at all with PRH in the consumer market.

And because Amazon has very intentionally created a whole massive pool of consumer books that nobody else has, through their own publishing and enabling independent authors.

Link to the rest at The Shatzkin Files

PG has had direct business/legal dealings and negotiations with Apple and Amazon over the last 15 years or so. For context, he has also had business negotiations with Microsoft, Oracle, Hewlett-Packard and Intel in the tech world plus every major investment bank in New York (Goldman Sachs, Morgan Stanley, etc., etc.), most of the large accounting firms plus Disney, American Express and a bunch of other big companies.

To be clear, this doesn’t mean PG knows everything about negotiating intellectual property partnerships and other deals with large organizations, but he does know some things about that subject.

PG definitely has not represented any large publishers in their dealings with large tech companies. He has, however, represented a lot of authors in their dealings with large publishers.

Speaking generally, large publishers are not cut out to be good partners for tech companies.

Publishers are simply too rigid in their business vision and very much focused on the short term (which is strange for organizations that license copyrights, which extend far into the future).

This short term outlook is substantially affected by the fact that the Big Five publishers are all owned and controlled by other and larger media conglomerates. Four of the Big Five are owned by large European publishing corporations that are not known for their commitment to innovation and could not be described as tech-savvy in any sense. The fifth Big Five publisher, Simon & Schuster, is owned by CBS.

Each of these media conglomerates is heavily focused on this quarter’s and this year’s income, expenses and profits. They’re not what anyone would call forward-looking or focused on the long term. If they think about the long term at all, they’re convinced it will not be much different than last quarter.

(PG worked for a major subsidiary of a very, very large international media conglomerate for three unhappy years and knows that of which he speaks.)

This means that if Google sends someone to talk to the President of a Big Five publisher, Google is talking to a middle-manager in a much larger business organization. The Big Five President can do pretty much whatever he/she wants to do with Barnes & Noble and Ingram (as long as it doesn’t have an adverse impact on profits), but cutting a strategic deal with Google is way, way out of his/her job description.

Organizations like Google, Apple and Amazon quickly become frustrated with organizations that are not able to move rapidly.

From Fast Company:

Devices like Amazon Echo could someday turn into a treasure trove for developers that make voice assistant skills, but first companies have to figure out where they draw the line when it comes to weighing data sharing against consumer privacy.

Now that dilemma is heating up: Citing three unnamed sources, The Information reported this week that Amazon is considering whether to provide full conversation transcripts to Alexa developers. This would be a major change from Amazon’s current policy in which the company only provides basic information—such as the total number of users, the average number of actions they’ve performed, and rates of success or failure for voice commands. Amazon declined to comment to The Information regarding the claims, but the change wouldn’t be unprecedented. Google’s voice assistant platform already provides full transcripts to developers.

The potential move by Amazon underscores how it is caught between two worlds with its Alexa assistant, especially in regards to privacy. By keeping transcripts to itself, Amazon can better protect against the misuse of its customers’ data and avoid concerns about eavesdropping. But because Alexa already gives developers the freedom to build virtually any kind of voice skill, their inability to see what customers are saying becomes a major burden.

. . . .

With Google Assistant, developers can view a transcript for any conversation with their particular skill. Uber, for example, can look at all recorded utterances from the moment you ask for a car until the ride is confirmed. (It can’t, however, see what you’ve said to other apps and services.) Google’s own documentation confirms this, noting that developers can request “keyboard input or spoken input from end user” during a conversation.

For developers, this data can be of immense utility. It allows them to find out if users are commonly speaking in the wrong syntax, or asking to do things that the developer’s voice skill doesn’t support.

. . . .

In terms of sharing data with developers, Apple’s Siri voice assistant is on the opposite side of the spectrum from Google. Developers who work with SiriKit get no information about usage from Apple, not even for basic things like how many people use voice commands to access an app, or which voice commands are most commonly used.

. . . .

But keep in mind that Siri’s approach to third-party development is entirely different from that of Google and Amazon. Instead of letting developers build any kind of voice application, Apple only supports third-party voice commands in a handful of specific domains, such as photo search, workouts, ride hailing, and messaging. And instead of letting those apps drive the conversation, Apple controls the back-and-forth itself. The apps merely provide the data and some optional on-screen information.

Because these apps don’t communicate with users directly, there’s no need for them to have conversation transcripts in the first place. Instead, Apple can look at what users are trying to accomplish and use that data to expand Siri on its own.

The downside to this approach is that Siri just isn’t as useful as other virtual assistants.

Link to the rest at Fast Company 

If PG lived in China, he would be inclined not to use Alexa.

From Bloomberg:

 Apple Inc. is already in your pocket, on your desk and underneath your television. Soon, a device embossed with “Designed by Apple in California” may be on your nightstand or kitchen counter as well.

The iPhone-maker has started manufacturing a long-in-the-works Siri-controlled smart speaker, according to people familiar with the matter. Apple could debut the speaker as soon as its annual developer conference in June, but the device will not be ready to ship until later in the year, the people said.

The device will differ from Amazon.com Inc.’s Echo and Alphabet Inc.’s Google Homespeakers by offering virtual surround sound technology and deep integration with Apple’s product lineup, said the people, who requested anonymity to discuss products that aren’t yet public.

Introducing a speaker would serve two main purposes: providing a hub to automate appliances and lights via Apple’s HomeKit system, and establishing a bulwark inside the home to lock customers more tightly into Apple’s network of services. That would help combat the competitive threat from Google’s and Amazon’s connected speakers: the Home and Echo mostly don’t support services from Apple. Without compatible hardware, users may be more likely to opt for the Echo or Home, and therefore use streaming music offerings such as Spotify, Amazon Prime Music or Google Play rather than Apple Music.

Link to the rest at Bloomberg

PG says competition keeps competitors sharp and is great for consumers.

From The New York Times:

A few weeks ago, I bought a new television. When the whole process was over, I realized something incredible: To navigate all of the niggling details surrounding this one commercial transaction — figuring out what to buy, which accessories I needed, how and where to install it, and whom to hire to do so — I had dealt with only a single ubiquitous corporation: Amazon.

It wasn’t just the TV. As I began combing through other recent household decisions, I found that in 2016, nearly 10 percent of my household’s commercial transactions flowed through the Seattle retailer, more by far than any other company my family dealt with. What’s more, with its Echos, Fire TV devices, audiobooks, movies and TV shows, Amazon has become, for my family, more than a mere store. It is my confessor, my keeper of lists, a provider of food and culture, an entertainer and educator and handmaiden to my children.

. . . .

This is the most glaring and underappreciated fact of internet-age capitalism: We are, all of us, in inescapable thrall to one of the handful of American technology companies that now dominate much of the global economy. I speak, of course, of my old friends the Frightful Five: Amazon, Apple, Facebook, Microsoft and Alphabet, the parent company of Google.

The five are among the most valuable companies on the planet, collectively worth trillions.

. . . .

 [L]ast week I came up with a fun game: If an evil, tech-phobic monarch forced you to abandon each of the Frightful Five, in which order would you do so, and how much would your life deteriorate as a result?

. . . .

When I went through the thought experiment, I found that dropping the first couple of tech giants was pretty easy — but after that the process became progressively more unbearable. For me, Facebook was the first to go. I tend to socialize online using Twitter, Apple’s messaging system, and Slack, the office-chat app, so losing Mark Zuckerberg’s popular service (and its subsidiaries, Instagram, WhatsApp and Messenger) was not such a big deal.

Next, for me, was Microsoft, which I found slightly more difficult to quit. I don’t normally use any Windows devices, but Microsoft’s word-processing program, Word, is an essential tool for me, and I’d hate to lose it.

In third place, full of regrets: Apple. There’s nothing I use more than my iPhone, and close behind are my MacBook and iMac 5K, which may be the best computer I’ve ever owned. Abandoning Apple would prompt deep and truly annoying rearrangements in my life, including braving Samsung’s bad software. But I could do it, grudgingly.

Link to the rest at The New York Times

From The Atlantic:

You were going to get one-click access to the full text of nearly every book that’s ever been published. Books still in print you’d have to pay for, but everything else—a collection slated to grow larger than the holdings at the Library of Congress, Harvard, the University of Michigan, at any of the great national libraries of Europe—would have been available for free at terminals that were going to be placed in every local library that wanted one.

At the terminal you were going to be able to search tens of millions of books and read every page of any book you found. You’d be able to highlight passages and make annotations and share them; for the first time, you’d be able to pinpoint an idea somewhere inside the vastness of the printed record, and send somebody straight to it with a link. Books would become as instantly available, searchable, copy-pasteable—as alive in the digital world—as web pages.

It was to be the realization of a long-held dream. “The universal library has been talked about for millennia,” Richard Ovenden, the head of Oxford’s Bodleian Libraries, has said. “It was possible to think in the Renaissance that you might be able to amass the whole of published knowledge in a single room or a single institution.” In the spring of 2011, it seemed we’d amassed it in a terminal small enough to fit on a desk.

“This is a watershed event and can serve as a catalyst for the reinvention of education, research, and intellectual life,” one eager observer wrote at the time.On March 22 of that year, however, the legal agreement that would have unlocked a century’s worth of books and peppered the country with access terminals to a universal library was rejected under Rule 23(e)(2) of the Federal Rules of Civil Procedure by the U.S. District Court for the Southern District of New York.When the library at Alexandria burned it was said to be an “international catastrophe.” When the most significant humanities project of our time was dismantled in court, the scholars, archivists, and librarians who’d had a hand in its undoing breathed a sigh of relief, for they believed, at the time, that they had narrowly averted disaster.

. . . .

Google’s secret effort to scan every book in the world, codenamed “Project Ocean,” began in earnest in 2002 when Larry Page and Marissa Mayer sat down in the office together with a 300-page book and a metronome. Page wanted to know how long it would take to scan more than a hundred-million books, so he started with one that was lying around. Using the metronome to keep a steady pace, he and Mayer paged through the book cover-to-cover. It took them 40 minutes.

Page had always wanted to digitize books. Way back in 1996, the student project that eventually became Google—a “crawler” that would ingest documents and rank them for relevance against a user’s query—was actually conceived as part of an effort “to develop the enabling technologies for a single, integrated and universal digital library.” The idea was that in the future, once all books were digitized, you’d be able to map the citations among them, see which books got cited the most, and use that data to give better search results to library patrons. But books still lived mostly on paper. Page and his research partner, Sergey Brin, developed their popularity-contest-by-citation idea using pages from the World Wide Web.

By 2002, it seemed to Page like the time might be ripe to come back to books. With that 40-minute number in mind, he approached the University of Michigan, his alma mater and a world leader in book scanning, to find out what the state of the art in mass digitization looked like. Michigan told Page that at the current pace, digitizing their entire collection—7 million volumes—was going to take about a thousand years. Page, who’d by now given the problem some thought, replied that he thought Google could do it in six.. . . .He offered the library a deal: You let us borrow all your books, he said, and we’ll scan them for you. You’ll end up with a digital copy of every volume in your collection, and Google will end up with access to one of the great untapped troves of data left in the world. Brin put Google’s lust for library books this way: “You have thousands of years of human knowledge, and probably the highest-quality knowledge is captured in books.” What if you could feed all the knowledge that’s locked up on paper to a search engine?

By 2004, Google had started scanning. In just over a decade, after making deals with Michigan, Harvard, Stanford, Oxford, the New York Public Library, and dozens of other library systems, the company, outpacing Page’s prediction, had scanned about 25 million books. It cost them an estimated $400 million. It was a feat not just of technology but of logistics.

. . . .

The stations—which didn’t so much scan as photograph books—had been custom-built by Google from the sheet metal up. Each one could digitize books at a rate of 1,000 pages per hour. The book would lie in a specially designed motorized cradle that would adjust to the spine, locking it in place. Above, there was an array of lights and at least $1,000 worth of optics, including four cameras, two pointed at each half of the book, and a range-finding LIDAR that overlaid a three-dimensional laser grid on the book’s surface to capture the curvature of the paper. The human operator would turn pages by hand—no machine could be as quick and gentle—and fire the cameras by pressing a foot pedal, as though playing at a strange piano.

What made the system so efficient is that it left so much of the work to software. Rather than make sure that each page was aligned perfectly, and flattened, before taking a photo, which was a major source of delays in traditional book-scanning systems, cruder images of curved pages were fed to de-warping algorithms, which used the LIDAR data along with some clever mathematics to artificially bend the text back into straight lines.

. . . .

In August 2010, Google put out a blog post announcing that there were 129,864,880 books in the world. The company said they were going to scan them all.

Of course, it didn’t quite turn out that way. This particular moonshot fell about a hundred-million books short of the moon. What happened was complicated but how it started was simple: Google did that thing where you ask for forgiveness rather than permission, and forgiveness was not forthcoming. Upon hearing that Google was taking millions of books out of libraries, scanning them, and returning them as if nothing had happened, authors and publishers filed suit against the company, alleging, as the authors put it simply in their initial complaint, “massive copyright infringement.”

. . . .

As Tim Wu pointed out in a 2003 law review article, what usually becomes of these battles—what happened with piano rolls, with records, with radio, and with cable—isn’t that copyright holders squash the new technology. Instead, they cut a deal and start making money from it. Often this takes the form of a “compulsory license” in which, for example, musicians are required to license their work to the piano-roll maker, but in exchange, the piano-roll maker has to pay a fixed fee, say two cents per song, for every roll they produce. Musicians get a new stream of income, and the public gets to hear their favorite songs on the player piano. “History has shown that time and market forces often provide equilibrium in balancing interests,” Wu writes.

But even if everyone typically ends up ahead, each new cycle starts with rightsholders fearful they’re being displaced by the new technology. When the VCR came out, film executives lashed out. “I say to you that the VCR is to the American film producer and the American public as the Boston strangler is to the woman home alone,” Jack Valenti, then the president of the MPAA, testified before Congress. The major studios sued Sony, arguing that with the VCR, the company was trying to build an entire business on intellectual property theft. But Sony Corp. of America v. Universal City Studios, Inc. became famous for its holding that as long as a copying device was capable of “substantial noninfringing uses”—like someone watching home movies—its makers couldn’t be held liable for copyright infringement.

The Sony case forced the movie industry to accept the existence of VCRs. Not long after, they began to see the device as an opportunity. “The VCR turned out to be one of the most lucrative inventions—for movie producers as well as hardware manufacturers—since movie projectors,” one commentator put it in 2000.

It only took a couple of years for the authors and publishers who sued Google to realize that there was enough middle ground to make everyone happy. This was especially true when you focused on the back catalog, on out-of-print works, instead of books still on store shelves. Once you made that distinction, it was possible to see the whole project in a different light. Maybe Google wasn’t plundering anyone’s work. Maybe they were giving it a new life. Google Books could turn out to be for out-of-print books what the VCR had been for movies out of the theater.If that was true, you wouldn’t actually want to stop Google from scanning out-of-print books—you’d want to encourage it. In fact, you’d want them to go beyond just showing snippets to actually selling those books as digital downloads.. . . .

Those who had been at the table crafting the agreement had expected some resistance, but not the “parade of horribles,” as Sarnoff described it, that they eventually saw. The objections came in many flavors, but they all started with the sense that the settlement was handing to Google, and Google alone, an awesome power. “Did we want the greatest library that would ever exist to be in the hands of one giant corporation, which could really charge almost anything it wanted for access to it?”, Robert Darnton, then president of Harvard’s library, has said.

Darnton had initially been supportive of Google’s scanning project, but the settlement made him wary. The scenario he and many others feared was that the same thing that had happened to the academic journal market would happen to the Google Books database. The price would be fair at first, but once libraries and universities became dependent on the subscription, the price would rise and rise until it began to rival the usurious rates that journals were charging, where for instance by 2011 a yearly subscription to the Journal of Comparative Neurology could cost as much as $25,910.Although academics and library enthusiasts like Darnton were thrilled by the prospect of opening up out-of-print books, they saw the settlement as a kind of deal with the devil. Yes, it would create the greatest library there’s ever been—but at the expense of creating perhaps the largest bookstore, too, run by what they saw as a powerful monopolist. In their view, there had to be a better way to unlock all those books. “Indeed, most elements of the GBS settlement would seem to be in the public interest, except for the fact that the settlement restricts the benefits of the deal to Google,” the Berkeley law professor Pamela Samuelson wrote.

Link to the rest at The Atlantic and thanks to Valerie for the tip.

From Backchannel:

Books can do anything. As Franz Kafka once said, “A book must be the axe for the frozen sea inside us.”

It was Kafka, wasn’t it? Google confirms this. But where did he say it? Google offers links to some quotation websites, but they’re generally unreliable. (They misattribute everything, usually to Mark Twain.)

To answer such questions, you need Google Book Search, the tool that magically scours the texts of millions of digitized volumes. Just find the little “more” tab at the top of the Google results page — it’s right past Images, Videos, and News. Then click on it, find “Books,” and click on that.

. . . .

It turns out that the “frozen sea” quote is from Kafka’s Letters to Friends, Family, and Editors, in a missive to Oskar Pollak, dated January 27, 1904.

. . . .

Google Book Search is amazing that way. When it started almost 15 years ago, it also seemed impossibly ambitious: An upstart tech company that had just tamed and organized the vast informational jungle of the web would now extend the reach of its search box into the offline world. By scanning millions of printed books from the libraries with which it partnered, it would import the entire body of pre-internet writing into its database.

“You have thousands of years of human knowledge, and probably the highest-quality knowledge is captured in books,” Google cofounder Sergey Brin told The New Yorker at the time. “So not having that — it’s just too big an omission.”

. . . .

Today, Google is known for its moonshot culture, its willingness to take on gigantic challenges at global scale. Books was, by general agreement of veteran Googlers, the company’s first lunar mission. Scan All The Books!

In its youth, Google Books inspired the world with a vision of a “library of utopia” that would extend online convenience to offline wisdom. At the time it seemed like a singularity for the written word: We’d upload all those pages into the ether, and they would somehow produce a phase-shift in human awareness. Instead, Google Books has settled into a quiet middle age of sourcing quotes and serving up snippets of text from the 25 million-plus tomes in its database.

Google employees maintain that’s all they ever intended to achieve. Maybe so. But they sure got everyone else’s hopes up.

. . . .

When I started work on this story, I feared at first that Books no longer existed as a discrete part of the Google organization — that Google had actually shut the project down. As with many aspects of Google, there’s always been some secrecy around Google Books, but this time, when I started asking questions, it closed up like a startled turtle. For weeks there didn’t seem to be anyone around or available who could or would speak to the current state of the Books effort.

The Google Books “History” page trails off in 2007, and its blog stopped updating in 2012, after which it got folded into the main Google Search blog, where information about Books is nearly impossible to find. As a functioning and useful service, Google Books remained a going concern. But as a living project, with plans and announcements and institutional visibility, it seemed to have pulled a vanishing act. All of which felt weird, given the legal victory it had finally won.

When I talked to alumni of the project who’d left Google, several mentioned that they suspected the company had stopped scanning books. Eventually, I learned that there are, indeed, still some Googlers working on Book Search, and they’re still adding new books, though at a significantly slower pacethan at the project’s peak around 2010–11.

. . . .

LED lighting, not widely available at the project’s start, has helped. So has studying more efficient techniques for human operators to flip pages. “It’s almost like finger-picking on a guitar,” Jaskiewicz says. “So we find people who have great ways of turning pages — where is the thumb and that kind of stuff.”

. . . .

Like many tech-friendly bibliophiles, Sloan says he uses Google Books a lot, but is sad that it isn’t continuing to evolve and amaze us. “I wish it was a big glittering beautiful useful thing that was growing and getting more interesting all the time,” he says. He also wonders: We know Google can’t legally make its millions of books available for anyone to read in full — but what if it made them available for machines to read?

Machine-learning tools that analyze texts in new ways are advancing quickly today, Sloan notes, and “the culture around it has a real Homebrew Computer Club or early web feel to it right now.” But to progress, researchers need big troves of data to feed their programs.

“If Google could find a way to take that corpus, sliced and diced by genre, topic, time period, all the ways you can divide it, and make that available to machine-learning researchers and hobbyists at universities and out in the wild, I’ll bet there’s some really interesting work that could come out of that. Nobody knows what,” Sloan says. He assumes Google is already doing this internally. Jaskiewicz and others at Google would not say.

Link to the rest at Backchannel