[thelist] User Agent question

Maximillian Schwanekamp anaxamaxan at neptunewebworks.com
Fri Feb 27 18:50:53 CST 2004

Got a user agent question - I'm setting up a 404 script for my client to
redirect traffic according to the url they entered.  For optimization's
sake, I would like to skip all processing if the user agent is a spider.  Is
there an easy way to differentiate between a spider and a browser (at least
for non-spoofed user agents!)??  I am thinking I can safely treat the UA as
a spider if the UA string has none of "Mozilla","MSIE","Opera","Gecko".  I
am not looking for perfect treatment - if a UA string is spoofed, fine.
But when a spider hits the site 50-100 times, there is no reason for the
script to be doing a database lookup every time.

Maximillian Von Schwanekamp
Websites for Profitable Microbusiness
voice: 541-302-1438
fax: 208-730-6504

More information about the thelist mailing list