Page 3 of 5 FirstFirst 12345 LastLast
Results 21 to 30 of 44

Thread: Block Web Content Scrapers and Downloaders

  1. I just have it on the home page -- that seems to be what most downloaders hit first.
    Submit Your Webmaster Related Sites to the NB Directory
    I swear, by my life and my love of it, that I will never live for the sake of another man, nor ask another man to live for mine.

  2. #22
    Google are using lots of different ones that's why.

  3. Quote Originally Posted by webmaster View Post
    Google are using lots of different ones that's why.
    It's pretty easy to tell which IP's are not really Google.
    Submit Your Webmaster Related Sites to the NB Directory
    I swear, by my life and my love of it, that I will never live for the sake of another man, nor ask another man to live for mine.

  4. #24
    I just snagged my second fish. This one is from China and is pretending to look like a browser.

    Code:
    address is 121.9.227.47, hostname is 121.9.227.47, agent is Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
    I've got bot-trap set up on two low traffic test sites. If I do not see anything detrimental over the next week, it will go on my main sites. Thus far it looks like a winner.

  5. #25

  6. Quote Originally Posted by thesyndicate View Post
    can they not hack from 777
    I do not understand what you are saying.

    What is the question?
    Submit Your Webmaster Related Sites to the NB Directory
    I swear, by my life and my love of it, that I will never live for the sake of another man, nor ask another man to live for mine.

  7. #27
    leaving a file open with 777 can not hackers just hack your server from that file.

  8. Not unless there is a vulnerable program to write to the file and putting something in that file has a benefit for them.
    Submit Your Webmaster Related Sites to the NB Directory
    I swear, by my life and my love of it, that I will never live for the sake of another man, nor ask another man to live for mine.

  9. #29
    Is this only for private servers or can i use it with shared hosting as well? Or is that a overkill?

  10. #30
    You should be able to use it on any server where you can use an .htaccess file.
    "Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote." -- Benjamin Franklin


Page 3 of 5 FirstFirst 12345 LastLast

Similar Threads

  1. Obfuscate Proxy Content to make harder to Block
    By tibbie in forum Web Proxies
    Replies: 4
    Last Post: 2 May, 2011, 08:34 AM
  2. New Content writer on the block!
    By AjiContent in forum Introduction Forum
    Replies: 0
    Last Post: 28 February, 2011, 09:19 AM
  3. Google on Content Scrapers
    By Kovich in forum Managing
    Replies: 18
    Last Post: 14 May, 2010, 06:58 AM
  4. Block Robots and Web Downloaders with robots.txt
    By Will.Spencer in forum Managing
    Replies: 12
    Last Post: 6 June, 2009, 15:40 PM
  5. How to Profit from Content Scrapers?
    By Shenron in forum Promoting
    Replies: 4
    Last Post: 12 March, 2009, 18:58 PM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •