[thelist] robots.txt ???

D Ross dross202 at comcast.net
Tue Feb 22 02:45:08 CST 2005


I've just recently learned about putting a robots.txt file in the 
directory/root. I'm not real familiar with it and what to do or if it's 
actually a good thing or a waste of time. What do you folks think?

I found this page that helps set it up:
http://www.searchengineworld.com/robots/robots_tutorial.htm

Any other sites that explain differently or better?

Thanks,
David


More information about the thelist mailing list