Results 1 to 4 of 4

Thread: cURL scraper queries

  1. #1
    DickTracy is offline Newbie Net Builder
    Join Date
    Dec 2008
    Posts
    116
    Thanks
    12
    Thanked 19 Times in 12 Posts

    cURL scraper queries

    Hi

    I know someone with a small car hire company and they like to always quote a price lower than the competition.

    Someone is implementing a cURL solution for them which queries the competitors prices, deducts 5%, then quotes that price. I guess the query will be run less than 5-10 times a day.

    Sounds sensible, but is it risky? Do sites notice and care when their site is accessed like this (with a regular automated query)? Can they block the access? Get the site taken down?

    Or is it more or less common business practice?

    Cheers

  2. #2
    Nathan's Avatar
    Nathan is offline Work!
    Join Date
    Dec 2008
    Location
    New Zealand
    Posts
    392
    Blog Entries
    1
    Thanks
    37
    Thanked 120 Times in 68 Posts
    If the site that they are getting the info gets a lot of traffic and the cURL scraper is using a legit agent it would be hard to track it down and ban it.

    Scrapers are used a lot on the internet world and I don't think they would be taken down.
    Programmer, web developer and entrepreneur.

  3. Thanked by:

    DickTracy (1 January, 2009)

  4. #3
    Charles's Avatar
    Charles is offline Senior Net Builder
    Join Date
    Dec 2008
    Location
    On the Bike
    Posts
    1,211
    Blog Entries
    5
    Thanks
    311
    Thanked 311 Times in 212 Posts
    Is it risky? -- Yep.
    Do sites notice and care when their site is accessed like this? -- I can't see very many possible ways that they could notice.
    Can they block the access? -- They could (with difficulty), but your friend could always query them using a proxy server to regain access.
    Get the site taken down? -- They couldn't, it's not illegal.
    Is it more or less common business practice? -- I hope not! If everyone queried other company's Web sites like this, market value = drop 5%, drop 5%, drop 5% (etc.), $0.

  5. Thanked by:

    DickTracy (1 January, 2009)

  6. #4
    DickTracy is offline Newbie Net Builder
    Join Date
    Dec 2008
    Posts
    116
    Thanks
    12
    Thanked 19 Times in 12 Posts
    I'll leave them to it then.

    I hope not! If everyone queried other company's Web sites like this, market value = drop 5%, drop 5%, drop 5% (etc.), $0.
    That's a different issue for them, I never said they had a profitable business!

    Cheers

Similar Threads

  1. Tips on how to work efficiently with SQL queries
    By Dejvis in forum Web Design
    Replies: 0
    Last Post: 26 June, 2009, 15:03 PM
  2. 20-25% of Google queries have never been searched before
    By Shenron in forum Keyword Research
    Replies: 2
    Last Post: 13 April, 2009, 16:29 PM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •