twill: a simple scripting language for Web scraping

**Will.Spencer** · 13 March, 2009, 20:03 PM

twill looks like it has a lot of promise for making web scraping easier.

**stickycarrots** · 13 March, 2009, 20:55 PM

hmmmm...interesting...

I just use php to scrape my data

**Duegar** · 29 April, 2009, 05:56 AM

its a good tool thanks. But, in your experience, do people mind if you screen scrape their website?

**Will.Spencer** · 29 April, 2009, 16:32 PM

Originally Posted by Duegar

its a good tool thanks. But, in your experience, do people mind if you screen scrape their website?

I think it depends a lot on what your intent is.

Scraping a manufacturers site in order to get data you need to resell their products -- they probably won't mind.

Scraping a competitors site in order to clone it -- you're probably not making friends.