[thelist] Bots killing my server

[Norman Bunn]

I have a calendar app that is running on my server that bots are 
accessing in spite of my robots.txt file saying otherwise (it's in 
cgi-bin).  Specifically, MSN and Yahoo are the problem.  I had this 
problem several year back and the update to the robots.txt file fixed it 
then, but not now.  What other avenues are available to me?  Here's the 
contents:

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /editor/
Disallow: /help/
Disallow: /images/
Disallow: /includes/
Disallow: /language/
Disallow: /mambots/
Disallow: /media/
Disallow: /modules/
Disallow: /templates/
Disallow: /installation/
Disallow: /cgi-bin/
User-agent: ia_archiver
Disallow: /
User-agent: msnbot
Crawl-delay: 30

Thanks,

Norman

-- 
---

Norman W. Bunn
norman.bunn at craftedsolutions.com
803.405.1008
----------------------------------------------
www.CraftedSolutions.com
Crafted Solutions, Inc.
Web Design & Development
Web Site Hosting & Custom Solutions
"Get the results the Internet promises;
 get the 'Net Result' from Crafted Solutions!"
----------------------------------------------