[thelist] Bots killing my server

Anthony Baratta anthony at baratta.com
Wed Feb 6 16:12:09 CST 2008


You might want to try this....

http://www.mcanerin.com/EN/search-engine/robots-txt.asp

One difference between the web-page generator and your code is that your default rule is first. That should be last, me thinks.




More information about the thelist mailing list