Results 1 to 1 of 1

Thread: robots.txt keyword - Spam

  1. #1

    Thumbs down robots.txt keyword - Spam

    Make a search in for 'robots.txt'.
    the 6th result on the first page is : which reads the following
    User-agent: *
    Crawl-delay: 10
    The keyword 'robots.txt' has a global monthly search volume of 246,000.
    Isn't this like spamming of the Serps, why doesn't google do something about this. I thought robots.txt was for bots and not for humans.
    Anyways, what will a noob user who has no idea and wants to learn about Robots.txt presume about it from this?
    @mods : please move if its in the wrong category, couldn't find a more suitable one.
    Last edited by Snak3; 3 November, 2009 at 16:40 PM. Reason: typo : spelt robots.txt as roots.txt lol :D

Similar Threads

  1. Help with Robots.txt
    By 5starpix in forum Building
    Replies: 4
    Last Post: 11 February, 2010, 03:27 AM
  2. robots.txt help
    By Sami4u in forum Building
    Replies: 9
    Last Post: 27 September, 2009, 07:43 AM
  3. Robots.txt?
    By dmi in forum Managing
    Replies: 15
    Last Post: 9 September, 2009, 17:15 PM
  4. Block Robots and Web Downloaders with robots.txt
    By Will.Spencer in forum Managing
    Replies: 12
    Last Post: 6 June, 2009, 15:40 PM
  5. What is robots.txt file?
    By ltimranjaved in forum Managing
    Replies: 1
    Last Post: 26 May, 2009, 12:27 PM

Tags for this Thread


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts