[thelist] ROR Sitemap Generator

Gilles Beauregard gilles at balour.org
Sat Jan 14 17:32:41 CST 2006


Allo!

> > Such as Google Sitemaps?
> > https://www.google.com/webmasters/sitemaps/docs/en/about.html
> >
> > regards.
> > -marc
> > --
> >
>
>I think the ROR generator does specify that it does search engines
>including AND other than Google.

Don't think so. Indexing robot use the robots.txt file to know what
to not included in the spidering. On the test I do this morning, all
page is included in the result page. This included all the /cgi-bin/
directory.

For myself, some part of any web site don't need to be indexed
by robot.

Gilles B.


---
Webmasters helping webmasters
http://www.balour.org/




More information about the thelist mailing list