Google's new Useragent for News
Hope it's the right section to post this... :thumbsup:
New User Agent for News
Quote:
Today we are announcing a new user agent for robots.txt called Googlebot-News that gives publishers even more control over their content.
Quote:
Site owners can just add Googlebot-News specific directives to their robots.txt file. Similar to the Googlebot and Googlebot-Image user agents, the new Googlebot-News user agent can be used to specify which pages of a website should be crawled and ultimately appear in Google News.
Quote:
Include pages in Google web search, but not in News:
User-agent: Googlebot
Disallow:
User-agent: Googlebot-News
Disallow: /
Full details:
http://googlewebmastercentral.blogspot.com/2009/12/new-user-agent-for-news.html
