This is a text file used to control the bots of various search engines to access the files and pages present at your site.It is quite useful in search engine optimization.
Its normally used to stop the search engine bots from accessing or indexing certain pages or stopping few of search engines.
if you dont want to stop any bot then place an empty robots.txt file at the root directory of your site or just write User-agent: * in it.
For e.g : www.yourcurrentaffair.blogspot.com/robots.txt
and if u want to stop someone then you can inlclude the name of that bot which u want to stop accessing these files.
the synatx which you have to write in order to block the bot is
replace nameofwebsite with your name