[thelist] browser requests

Stefan Waidele jun. Stefan at Waidele.info
Sat Jan 28 04:26:26 CST 2006


Robert schrieb:
> [...
 > [Do some technical wizardry in order to]
> block the requesting ip address - possibly for a
> short period, but enough to make the site undesirable to scrape. Any
> thoughts?

Why should you want to do this?
or
What is so bad about scraping?

IMO,
1. If a 'valid customer' thinks the site is so valuable that he/she 
wants to archive it on harddrive, why should I block that? (Think: 
Downlaod before out-of-town-trips, Broadband at work, but want to read 
at home, and other scenarios
2. If I block scraping, could I accidentally block search-engines? Or 
web-archives?
3. If somebody _really_ wants to scrape your site, he/she will do so. 
Even regular wget has options for random delays between requests, which 
would probably get around such limitations.

I think the harm that scrape-prevention does weights more than the good 
it brings.

But that's just my opinion.

Stefan

-- 
http://Stefan.Waidele.info
http://LinuxBasics.org
http://Krone-Neuenburg.de



More information about the thelist mailing list