[thelist] Common spider/bot USER_AGENT strings?

Simon at Sentient Technologies sentient at keysdigital.com
Thu Jul 12 07:31:47 CDT 2001


-- at 10:19 PM +0100 on 7/11/01, George Dillon wrote:
> 
> Is there a list anywhere of common spider/bot
> USER_AGENT strings I could add to this?

-----

George, 

This may be more than you need but...

	http://www.cyscape.com/browscap/

...has a file you can download for free
which will give you something to work
with.

HTH,

Simon
-- 

-------------------------------------------------------
Sentient Technologies - mailto:sentient at keysdigital.com
-------------------------------------------------------





More information about the thelist mailing list