[thelist] Spider Attack
Maximillian Schwanekamp
lists at neptunewebworks.com
Tue Mar 15 22:25:29 CST 2005
BMP wrote:
>I need someone to advise me about preventing robot spiders from using excess bandwidth on my website. The biggest abuser seems to be Mozilla Gecko. Any help would be greatly appreciated.
>
OK, I'll bite. Mozilla Gecko is not a spider, but currently the most
popular rival browser on the market. In some geek circles it has become
#1. If you're seeing lots of Gecko in your logs/stats, that is human
traffic, not spiders.
http://www.mozilla.org/
That said, here's an article that may get you going "How to block
spambots, ban spybots, and tell unwanted robots to go to hell" by Mark
Pilgrim. Some additional links in the comments to the article.
http://url123.com/25633
More information about the thelist
mailing list