Thread: cURL scraper queries

Hi

I know someone with a small car hire company and they like to always quote a price lower than the competition.

Someone is implementing a cURL solution for them which queries the competitors prices, deducts 5%, then quotes that price. I guess the query will be run less than 5-10 times a day.

Sounds sensible, but is it risky? Do sites notice and care when their site is accessed like this (with a regular automated query)? Can they block the access? Get the site taken down?

Or is it more or less common business practice?

Cheers

If the site that they are getting the info gets a lot of traffic and the cURL scraper is using a legit agent it would be hard to track it down and ban it.

Scrapers are used a lot on the internet world and I don't think they would be taken down.

Is it risky? -- Yep.
Do sites notice and care when their site is accessed like this? -- I can't see very many possible ways that they could notice.
Can they block the access? -- They could (with difficulty), but your friend could always query them using a proxy server to regain access.
Get the site taken down? -- They couldn't, it's not illegal.
Is it more or less common business practice? -- I hope not! If everyone queried other company's Web sites like this, market value = drop 5%, drop 5%, drop 5% (etc.), $0.

I'll leave them to it then.

I hope not! If everyone queried other company's Web sites like this, market value = drop 5%, drop 5%, drop 5% (etc.), $0.

That's a different issue for them, I never said they had a profitable business!

Cheers