[thelist] can I use a blank robots.txt file?
David U.
davidu at everydns.net
Wed Sep 11 14:35:01 CDT 2002
Dunstan Orchard wrote:
> Hi there,
>
> I've got an auto-email-404-error-script running on my site and my
> inbox gets full up with requests for a robotos.txt file.
>
> As I don't want to disallow anything on the site, is it ok to just
> put a blank robots.txt file up there?
>
> It won't do more harm than good or anything will it?
That, or you could just tell your 404-email thing to grep out the robots.txt
-davidu
More information about the thelist
mailing list