[thelist] can I use a blank robots.txt file?
Chris W. Parker
cparker at swatgear.com
Wed Sep 11 14:32:01 CDT 2002
> -----Original Message-----
> From: Dunstan Orchard [mailto:dunstan at 1976design.com]
> Sent: Wednesday, September 11, 2002 12:27 PM
>
> I've got an auto-email-404-error-script running on my site
> and my inbox gets
> full up with requests for a robotos.txt file.
>
> As I don't want to disallow anything on the site, is it ok to
> just put a blank
> robots.txt file up there?
>From http://www.robotstxt.org/
<quote>
To allow all robots complete access
User-agent: *
Disallow:
Or create an empty "/robots.txt" file.
</quote>
wow, google sure does work well! (robots.txt allow all)
chris.
More information about the thelist
mailing list