[thelist] Hide entire directory from search engines?
Eduardo Hutter
admforum at brasmontreal.net
Sat Mar 29 02:56:40 CDT 2008
Or if you want to disable specific bots and crawlers, you could use
htaccess to block them
* Hassan Schroeder wrote, On 28/03/2008 4:10 PM:
> On Fri, Mar 28, 2008 at 12:01 PM, Gersten, John <JGERSTEN at lchb.com> wrote:
>
>
>> Does anyone know of a way to conceal entire directories from crawlers, etc.?
>>
>
> Password protecting it would be one, since robots.txt is only a serving
> suggestion :-)
>
>
More information about the thelist
mailing list