[thelist] robots.txt
Diane Soini
dianesoini at earthlink.net
Mon Nov 17 19:55:53 CST 2003
> Re: [thelist] robots.txt
Thanks for the info. I don't have as much access to the log files as
probably most of you because I don't host the site myself. I suspect
the hosting company I use filters out some of these problems out of
their own self-interest.
I'm mostly concerned with email addresses being stolen. I appreciate
the information for how to detect if that is happening. I will try that.
I'm curious if these robots have the ability to browse cgi scripts and
read the script without executing it. My hosting company has it set up
that I must put cgi scripts into a directory of my site and not
cgi-bin. They have likely got some protective things in place around
this practice, but I'm still curious.
Spam isn't a huge problem for me, but it's the principle of it.
More information about the thelist
mailing list