Results 1 to 2 of 2

Thread: What is robots.txt file?

  1. #1
    ltimranjaved is offline Newbie Net Builder
    Join Date
    May 2009
    Location
    http://yourcurrentaffair.blogspot.com
    Posts
    10
    Thanks
    0
    Thanked 0 Times in 0 Posts

    What is robots.txt file?

    This is a text file used to control the bots of various search engines to access the files and pages present at your site.It is quite useful in search engine optimization.
    Its normally used to stop the search engine bots from accessing or indexing certain pages or stopping few of search engines.

    if you dont want to stop any bot then place an empty robots.txt file at the root directory of your site or just write User-agent: * in it.

    For e.g : www.yourcurrentaffair.blogspot.com/robots.txt

    and if u want to stop someone then you can inlclude the name of that bot which u want to stop accessing these files.

    the synatx which you have to write in order to block the bot is

    Disallow: /nameofwebsite/memberlist.php


    replace nameofwebsite with your name

  2. #2
    ltimranjaved is offline Newbie Net Builder
    Join Date
    May 2009
    Location
    http://yourcurrentaffair.blogspot.com
    Posts
    10
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: What is robots.txt file?

    so at last forum is alive :P

Similar Threads

  1. [WTS] PR1 File Mirror Site - Upload to 15 File Hosts
    By ChaoscripT in forum Sites
    Replies: 0
    Last Post: 26 September, 2010, 10:53 AM
  2. Help with Robots.txt
    By 5starpix in forum Building
    Replies: 4
    Last Post: 11 February, 2010, 02:27 AM
  3. robots.txt help
    By Sami4u in forum Building
    Replies: 9
    Last Post: 27 September, 2009, 07:43 AM
  4. Robots.txt?
    By dmi in forum Managing
    Replies: 15
    Last Post: 9 September, 2009, 17:15 PM
  5. Block Robots and Web Downloaders with robots.txt
    By Will.Spencer in forum Managing
    Replies: 12
    Last Post: 6 June, 2009, 15:40 PM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •