On Sun, 16 Nov 2003 15:44:57 -0800, Diane Soini <dianesoini at earthlink.net> wrote: > How well is the robots.txt file obeyed? http://diveintomark.org/archives/2003/02/26/how_to_block_spambots_ban_spybots_and_tell_unwanted_robots_to_go_to_hell And this might help too, sorry - it was the next link unfollowed before I hit send.... hth Tony