Tuesday, April 7, 2009

Some questions related to Google News and the Associated Press

Yesterday I entered the following search in Google News: [Phish in mountain view]. The search results led me to click on this headline, which took me to the full story by the San Jose Mercury News about Phish's upcoming concert at Shoreline Amphitheatre.

Users like me are sent from different Google sites to newspaper websites at a rate of more than a billion clicks per month. These clicks go to news publishers large and small, domestic and international -- day and night.

And once a reader is on the newspaper's site, we work hard to help them earn revenue. Our AdSense program pays out millions of dollars to newspapers that place ads on their sites, and our goal is that our interest-based advertising technology will help newspapers make more from each click we send them by serving better, more relevant ads to their readers to generate higher returns.

The Associated Press (AP) recently issued a press release announcing plans to develop an initiative to "protect" the newspaper industry's content online. Since then, some readers, users and journalists have asked us if the AP's plan is about Google since we host complete AP articles. The answer is that it doesn't appear to pertain to Google since we host those articles in partnership with the AP. We announced that partnership in 2007 as part of an experiment in hosting articles on our site. In hosting agreements such as this, we pay news agencies and display the entire text of articles, such as this one from the AP about President Obama's visit to Turkey.

We drive traffic and provide advertising in support of all business models -- whether news sources choose to host their articles with us or on their own sites, and whether their business model is ad-supported or based on subscriptions. In all cases, for news articles we've crawled and indexed but do not host, we show users just enough to make them want to read more -- the headline, a "snippet" of a line or two of text and a link back to to the news publisher's website.

In the U.S., the doctrine of fair use enshrined in the US Copyright Act allows us to show snippets and links. The fair use doctrine protects transformative uses of content, such as indexing to make it easier to find [pdf]. Even though the Copyright Act does not grant a copyright owner a veto over such uses, it is our policy to allow any rightsholder, in this case newspaper or wire service, to remove their content from our index -- all they have to do is ask us or implement simple technical standards such as robots.txt or metatags.

As for Phish in Mountain View this summer, asking will get you nowhere because the tickets are already sold out.


Don said...

The big corporate content providers are going to have to make some tough decisions. They will need to focus on the core of their business: quality content. They should look to other non-essential areas for cost cutting. They should seriously consider going all digital, kill the printing presses, trucks, deliveries, most of the marketing and let sites like Google and such drive traffic. The bulk of their spending shoud be on content and the technical infrastructure to support their product.

If they don't follow that path, they will go under.

Kosher said...
This comment has been removed by the author.
Kosher said...

The problem is that sometimes the guy with no pockets wins. The guy with deep pockets is saying "why should I dump all of this money into content when some shmuck is giving away the same news for free?".

The guy with no pockets or even shallow pockets is just barely able to sustain his own life but hopes one day to have deep pockets. This guy also comes in lots of shapes and sizes.

The guy with deep/big pockets pays guys with shallow pockets to make content. When another big guy comes along and steps on lots of big guy pockets by giving lots of little guys the big guys job for free, other big guys get mad.

In the end, the big guys need to figure out how to pay the other big guys so that the little guys can get paid too. Maybe all of the big guys need to start paying the little guys instead of just the big guys too. Maybe there are just too many big guys and not enough work for the little guys since all the big guys are favoring the little guys with shallow pockets.

Thanks, Average Guy

hello_kittyhawk said...

I am very tired of watching these giant dinosaurs destroy the landscape with their wars of dominance.

The Associated Press is one more creature thinking only with its hindbrain, only dimly aware of its eventual extinction.

It would be so much nicer if they could all just go quietly ...

TomForemski said...

The problem is that online advertising doesn't pay enough to support the costs of news reporting.

If you are like Google and mostly rely on machines to harvest and publish content then there is enough revenues to pay for your costs of business.

If you need to hire journalists, editors, production people, photographers, etc to create and publish your content then you are out of luck. Online ads won't cover your costs of business.

If Google, and other aggregators started charging users of Google News, and shared that revenue with the newspapers, instead of sharing poorly performing AdSense revenue, then maybe that would give the content creators the resources to continue news reporting.

elogic said...

Very ridiculous and short sighted for the major content news publishers to try to hang their problems on Google's shoulders. Its obvious that Google is driving traffic to these content providers and that is a value add to the publishers not the reverse.

If these publishers feel the visibility of their headlines and story intros (which ultimately drive traffic to their sites to be monetized), then pull the content from Google, plain and simple.

Clearly that will have the opposite effect intended since they will now have (a) diminished traffic and (b) fewer eyeballs to monetize using (c) what they know is a non-viable strategy (100% advertising supported content).

Time to get creative folks - you need a better product and a better business model. Micropayments anyone?

Mike Saunders said...

One major problem with this analysis: Google isn't a news organization. It's not part of the Associated Press cooperative and doesn't contribute any newsgathering resource. It takes but doesn't give.

The AP's member organizations contribute reporting and original stories to the wire service in addition to fees they pay for various news feeds.

The bulk of AP content comes from its member organizations, and yes, Google news searches do route readers to this content. Often it's an AP wire story run on the site of a member organization.

Essentially, by graciously "hosting" wire service content, and -- ta da! -- tweaking search results to drive eyes to the hosted AP content, Google is siphoning off readers from hundreds, if not thousands of other sites.

It doesn't matter of the vampire smiles before the bite.

THAT'S the old media beef against Google, and it's a valid one. However, none of that absolves old media from its creaky reluctance to embrace technology or to revamp its money-bleeding distribution model. Google is only one of the myriad revenue-sucking forces assailing media -- and it's not even the worst of the bunch.

Many of the wounds were self-inflicted.

Roblimo said...

Google AdSense makes plenty of money for Google and shares hardly any of it with publishers. I'm sorry, but at the content provider end, AdSense is a sad joke.

My personal solution has been to move from making money doing reporting to earning a living doing (mostly online) ad/promo video work, with journalism as a hobby.

In my video/promo role, I love Google. It sends business to my clients. And that's what Google's for: selling things. Google is a merchant's friend. But it is no friend to anyone still trying to earn a living as an honest journalist, despite all the sanctimonious "we send you traffic" talk.

Google actively competes with news sites when it comes to selling ads -- and of course Google can sell for less since it is a scavenger, not a producer.

Not that I care. I am just a little schmuck in Floriduh, doing my best to ride the winds of change...

elogic said...

In all this talk of Google's "scavenger-ness" tell me how do you find the things you want to find without a search engine? I would imagine that is not a very friendly (or useful web experience).

If the problem that is driving newspapers into the ground is the far greater awareness and visibility of this content as a result of the exposure from sources like google, maybe they should pull all content off the web and go back to the local / regional model of the past.

Unfortunately its a changed world and that would surely drive newspapers to extinction at a much faster pace.

As someone who values good journalism (and a newspaper subscriber myself!) it is still clear that this is a business model problem. I do think newspapers need to charge for online content as ads just don't cut it (for now) - but the whole industry needs to cooperate or simply won't work.

Trust me, if people can't get access to good news content the willingness to pay will go up. This is no simple feat - its monumental, but I just don't see any other way.

Mike Saunders said...

@ elogic:

Search isn't the issue. We all love search. Search is good, search is kind.

The issue is that by hosting AP content, and then featuring it prominently via search, Google undercuts the news organizations that contributed to the creation of that content.

That's it.

David Gerard said...

"If Google, and other aggregators started charging users of Google News, and shared that revenue with the newspapers, instead of sharing poorly performing AdSense revenue, then"

... no-one would read it.

elogic said...

@ Mike Saunders

So you don't have an issue with the linking to newspaper content with the headlines / snippets (in accordance with fair use). Your main issue is with the hosting of full AP stories?

I think that is a fair point to consider but from my read of this situation, if there is joint ownership/development of content between AP and its member organizations, and AP is cutting deals with Google to present and profit from the stories it provides this seem like an issue between AP and its Network, not Google. Yes?

Mike Saunders said...

...a postscript:

Part of the media industry's beef is with the AP itself for allowing Google, Yahoo, and other aggregators to pay for content.

Sort of defeats the purpose of having a cooperative network of newsgathering organizations....

Bored Silly said...

HA! -- You people have REAL short memory span-- Have you heard: "those who RUSH to be the first, are certainly not the wisest " Google played you all, even now it plays the WhiteHouse with the new president. HOW you ask? Simple, Bush used IDA's to monitor you. Google monitors you by volume demographic comparison and then singling out case studies. SO since Bush is gone -Google needs political influence to remain- who better than that of the guy whose party runs the house. So the papers are worried they are dying- nothing new- actually it is 1995 news. What you need to watch is what google gets involved with in Washington. Be weary - power doesn't corrupt - ultimate power does.

Davidfree said...

I see the blame with AP (that is: with its owners).

If they choose to undercut their own business model (which for the last decades used to be: gather news, sell it to the newspapers, let THEM sell it to the ENDuser) Google is just the new newspaper stand that takes the opportunity.

However, as the newspapers around the world are DYING, somebody else will have to pay for AP´s services.

If that "somebody else" is Google, then Google IS indeed a NEWS ORGANISATION and as such is directly competing the the newspapers.

The fact that they also give some AD"cents" back to the newspapers is of no importance in this discussion, as with Ad"cents" no professional newspaper can afford its journalists...

So, now you can still find your "phish" news on SJ Mercury, but maybe soon not any more on LA Times, SF Chronicle, ....

But - maybe it really is enough to read the news on Google-owned AP?

Actually, Google could just buy a few large publishing houses around the world. Why not? They go cheap these days!

MHC said...

This probably isn't the place to do this, but since it was brought up: I'm not entirely happy with Google News. I've tried several times to get the newspaper I work for listed. It is still listed as a blog and not in the news sources.

Q-Notes (www.q-notes.com) is the premier LGBT newspaper of North and South Carolina. We're based in Charlotte, the region's largest city. It is a shame the paper still isn't included in Google News.

byork said...

Bravo... thank you for telling Murdoch and the other news barons the plain and simple truth. If they don't like the traffic google is sending them, there is a very simple solution... robots.txt

PosIT said...

Google is trying to change the subject, this isn't about ad $ or driving traffic. Google, you are missing the most basic and important point, and arguing with FUD, the old "they don't get it" parlay. If you have paid for the rights and or have an agreement to redistribute content, then fine, no argument. But when you are redistributing content without explicit rights UP FRONT you are infringing on the copyright. The copyright holder gets to set the policy, and if they do not post a policy other than the copyright then you can NOT reuse/redistribute the content, period. Your logic comes out like this: If you don't tell me I am not allowed to steal each time I want to steal then I am allowed to steal. Nope, you are simply stealing. Copyright law protects against non-approved distribution by default, that is the essence of that law. Frankly I am surprised that it has taken so long for information SOURCES, the people who have done the hard work, to challenge what is a pretty clear case of infringement. I realize it isn't just you, it is done by all kinds of web sites. But, well, I guess that is price you pay for being so big. Similarly I have noticed many music videos being taken down from Youtube recently. Hey, I liked some of those videos, but tough for me - if Youtube doesn't have the rights for redistribution then they don't! Instead of acting like a teenager trying to weasel out of a fib, show some backbone and say "We have made a mistake" and start acting responsibly. Google, get online and search "sues Google" and look at the list. Google, in this case you don't get it. Wake up or you might be Googling "class action suit."

Chandru said...

I dont agree with PostIT comments. If one newspaper or website quotes an article from another one, is it stealing the content, even though the original article's name and the newspaper have been mentioned and linked . In my opinion (and the copyright law), what google news is doing is actually a service to the news paper web site that people like me would never go to, if it was not on google news. News paper guys need to get over this fear and partner with Google to get the clicks.

Carlos said...

There is no doubt that Google News/ Google searches drive traffic to newspaper sites. There is also no doubt that Google has an agreement with AP. An agreement that may or may not siphon away some readers form news orgs. So how is this Google's wrong-doing? I dont see it. Also, how is it Google's fault that the news orgs haven't been able to monetize the traffic that it pushes their way? As far as online ads not paying very well, that is also not Google's fault. Newspaper's have greatly devalued online advertisements themselves by "giving" them away as add-ons to print ads or over-delivering ad impressions. Again, none of this is Google's fault.

elogic said...


Do you have examples of this illegal copyright "stealing" where full articles are being reprinted on Google web properties? That would be really useful to this dialogue but I don't think you'll find that... If you do, I'm 100% behind your statements and the media sources that have been infringed have a lock/stock legal case to recoup any associated losses.

In the case of the the AP article reprints, this is a negotiated partnership with AP/Google so that is clearly above board legally speaking. If the broader AP network has issues with that, shame on AP and on the disgruntled network members for not working that out within their own contractual relationships...

As far as Google's deliver of article title/snippets (Google News, Google search resutls), I think it would be hard to call those infringement since these fall within a "fair use" scenario. Furthermore, they drive traffic directly to the Media SOURCES themselves who can then monetize that traffic. If said media sources do not want that traffic, there is a very simple, low tech way to turn off the traffic spiggot. However, I think you'd be VERY hard pressed to find many executives at these media SOURCES that would want to go that route since it is somewhat akin to cutting the knees off of their online property...

Stepping back, I see these news article summaries and links delivered by google as really the modern equivalent of the traditional news stand / dispenser, allowing consumers to browse stories of interest before deciding to consume (or buy)... Google doesn't take a slice of the "sale" when a browser clicks to a news site and views the ad(s) on said newspaper website but certainly has its own way of monetizing this process (symbiotic value).

Since online advertising alone does not appear to offer enough revenue to support the cost of running and creating quality content, clearly there is a problem. And that problem falls squarely on the laps of the media companies themselves. The internet (not just Google) has shifted the ground beneath these companies, and they have few other choices then to redefine their traditional business model. If ads don't cut it, then subscriptions, micropayments and other, as yet to be dreamt up revenue sources must fill in the gaps.

I love good journalism and want to see it thrive and grow in this new era - and that means journalists need to get paid. But the industry is pointing the finger in the wrong direction. Online they say "Content is King" and that is true. But the King needs a new crown...

[Disclosure: I'm definitely not a Google employee just outraged at misguided media industry finger pointing]

David Gerard said...

GRUB STREET RETIREMENT VILLAGE, Times Old Roman, back when this were all fields (Mediocre Grauniad) — The Guardian Media Group has asked the Government to examine Google News and other content aggregators, claiming they contribute insufficiently to their income.

“The newspapers put their content up on the web for free and then Google, the freeloading bastards, tell people where to find it. We told them to pay up or stop using our stuff, and they said OK, they’d stop using our stuff!

“We need the Government to bring back balance, ‘balance’ defined as being able to make them give us money because we want it. You’d think the Internet wasn’t invented to give newspapers and record companies free money!”

The newspaper group argues that traffic from search engines doesn’t make up the cost of producing the content. “Ad revenue has collapsed, so search engine traffic doesn’t bring in enough views to pay for itself. Our inability to sell ads is clearly Google’s problem.”

The Guardian suggests the exploration of new models that “require fair acknowledgement of the value that our content creates, both on our own site through advertising and ‘at the edges’ in the world of search and aggregation. Basically, they should just give us money because we want it. And the music industry too. How about a bailout? Go on, gi’s it.”

My blog rant: http://tinyurl.com/dyun4n

LVCI said...

What ticks me off is if AP wants to carry on they need to disable embedding. It's such a simple solution!

Oilrigr said...

It's funny that all the other media orgs have backed AP, while choosing a different route based on working with Google. When your competitors egg you on AP, and then you hear them laughing behind your back, it's probably not a good thing. Go AP go!