[thelist] Common spider/bot USER_AGENT strings?
Simon at Sentient Technologies
sentient at keysdigital.com
Thu Jul 12 07:31:47 CDT 2001
-- at 10:19 PM +0100 on 7/11/01, George Dillon wrote:
>
> Is there a list anywhere of common spider/bot
> USER_AGENT strings I could add to this?
-----
George,
This may be more than you need but...
http://www.cyscape.com/browscap/
...has a file you can download for free
which will give you something to work
with.
HTH,
Simon
--
-------------------------------------------------------
Sentient Technologies - mailto:sentient at keysdigital.com
-------------------------------------------------------
More information about the thelist
mailing list