[thelist] Hide entire directory from search engines?

Dejan Kozina dejan at kozina.com
Fri Mar 28 22:05:36 CDT 2008

If your including is done with PHP or some other server-side scripting 
not bound by .htaccess rules you can use this:

order allow,deny
deny from all

This prevents Apache from serving any content directly toward the 
network, while scripting engines are bound only by file-system 


Gersten, John wrote:
> It's not specific crawlers/bots I'm worried about; I need to stick some "include" content somewhere, and if I put it "above" the site root (which I realize is optimal for security, etc.) then my paths get long, plus there are some issues of convenience in terms of creating the pages that pull toward a "downward" location. So while I'm happy to have the crawlers find and index the pages using the pulled content, I don't want to confuse things by having the material show up twice as far as the bots are concerned. Does that make sense?
> I have a feeling I'll end up moving the content above the site root in the end, but I was hoping for something to support what is essentially my construction-laziness.
> -----Original Message-----
> From: thelist-bounces at lists.evolt.org on behalf of Eduardo Hutter
> Sent: Sat 3/29/08 0:56
> To: thelist at lists.evolt.org
> Subject: Re: [thelist] Hide entire directory from search engines?
> Or if you want to disable specific bots and crawlers, you could use
> htaccess to block them
> * Hassan Schroeder wrote, On 28/03/2008 4:10 PM:
>> On Fri, Mar 28, 2008 at 12:01 PM, Gersten, John <JGERSTEN at lchb.com> wrote:
>> Password protecting it would be one, since robots.txt is only a serving
>> suggestion  :-)
Dejan Kozina Web design studio
Dolina 346 (TS) - I-34018 Italy
tel./fax: +39 040 228 436 - cell.: +39 348 7355 225 skype: dejankozina
http://www.kozina.com/  - e-mail: dejan at kozina.com

More information about the thelist mailing list