[thelist] Trigger for Spambots IP Address Filtering

[J.J.SOLARI]

Mark Groen 1/04/05 17:19 -0800:

>that sounds like a lot of work, using mod_rewrite is too at first,
>but then it's just a matter of maintenance:
>
>http://diveintomark.org/archives/2003/02/26/
how_to_block_spambots_ban_spybots_and_tell_unwanted_robots_to_go_to_
hell

Mark,

The method explained in above URL is fine, but this is an all manual
process: one need to look first in the logs for suspicious queries,
then set up an individual rule for each would be robot.

However, one big drawback is that this method will not prevent a bad
robot from doing its job, i.e. query all possible files, thus not
preventing bandwidth 'bad' consumption. More there is no warranty
that bot names will not change at times.

The advantage of recording automatically IP addresses of
ill-behaving robots in such a way is that you can stop immediately
their predating the bandwidth whenever identified as such.


There are also a couple of pages which inspired greatly my
anti-badbot strategy:

<http://www.webmasterworld.com/forum23/1281.htm>
<http://www.kloth.net/internet/bottrap.php>
<http://www.danielwebb.us/software/bot-trap/>

JJS.