[thelist] Hide entire directory from search engines?

Eduardo Hutter admforum at brasmontreal.net
Sat Mar 29 02:56:40 CDT 2008

Or if you want to disable specific bots and crawlers, you could use
htaccess to block them

* Hassan Schroeder wrote, On 28/03/2008 4:10 PM:
> On Fri, Mar 28, 2008 at 12:01 PM, Gersten, John <JGERSTEN at lchb.com> wrote:
>> Does anyone know of a way to conceal entire directories from crawlers, etc.?
> Password protecting it would be one, since robots.txt is only a serving
> suggestion  :-)

More information about the thelist mailing list