[thelist] Identify a Web Crawler's request
Daniel Champion
dchampion at clacks.gov.uk
Tue Jul 6 08:14:31 CDT 2004
> How do I identify a request from a web-crawler?
They will identify themselves with a user-agent string like any browser.
For a list see:
http://www.psychedelix.com/agents.html
HTH
--
Daniel Champion - Web Dev Mgr - Clackmannanshire Council
e: dchampion at clacks.gov.uk t: 01259 452258
f: 01259 452265 w: http://www.clacksweb.org.uk
**********************************************************************
This email and any attachments have been scanned for viruses prior to leaving Clackmannanshire Council.
Clackmannanshire Council will not be liable for any losses as a result of viruses being passed on.
www.clacksweb.org.uk
**********************************************************************
More information about the thelist
mailing list