[thelist] Google is indexing me......

[Michael Pemberton]

It may be possible to create an alias for that url and then close off
access to that directory.  I haven't got access to my apache box at the
moment, I can try see if it works when I get a chance.

This should work for a single directory.  I don't know what you can do
when there are too many variations.

CDitty wrote:
> 
> Which directory?  Afraid I don't understand completely.  The forum
> directory or the one below?  The forums can be accessed anywhere on the
> site and the one I listed below was just one of hundreds of directories in
> the error log.
> 
> At 10:16 AM 9/9/2001, you wrote:
> >It might even be worth using http-agent detection to block the spider
> >from accessing that directory.  This will stop it from putting 404 pages
> >in the google result.
> >
> >CDitty wrote:
> > >
> > > As far as I can tell, there are not any users yet hitting these links.  It
> > > is just the Google spider causing all these errors.   Since it is the
> > > spider causing problems, my 404 page will have no effect.
> > >
> > > Will the mod_rewrite take care of links like this -
> > > /One_Time/Household_Items/Lawn_and_Garden/\forum\ ?
> > >
> > > Chris
> > >
> > > At 09:57 AM 9/9/2001, you wrote:
> > >
> > > >One option would be to just go with the flow and use mod_rewrite to
> > > >correct the URL.
> > > >
> > > >That is, if you have mod_rewrite available to you. The other option is
> > > >using the 404 document, but I don't think that will fix your errorlog
> > > >problem
> > > >
> > > >hth,
> > > >
> > > >--
> > > >mattwarden
> > > >mattwarden.com
> > > >
> > > >On Sep 9, CDitty had something to say about [thelist] Google is
> > indexing...
> > > >
> > > > >Google is indexing me and is filling up my error log.  For some reason,
> > > > >they keep trying to hit my forums and are trying to call it with a
> > \forum
> > > > >instead of /forum (see below).
> > > > >
> > > > >[Sun Sep 9 01:12:53 2001] [error] [client 216.239.46.153] File does not
> > > > >exist: /One_Time/Household_Items/Lawn_and_Garden/\forum\
> > > > >
> > > > >
> > > > >They are doing this on almost every single page.  Can anyone offer any
> > > > >advice on fixing this?  I have verified all my links and they are
> > > > >correct.  I would hate to have to block Google as I get tons of traffic
> > > > >from them.
> > > >
> > > >
> > > >---------------------------------------
> > > >For unsubscribe and other options, including
> > > >the Tip Harvester and archive of TheList go to:
> > > >http://lists.evolt.org Workers of the Web, evolt !
> > >
> > > ---------------------------------------
> > > For unsubscribe and other options, including
> > > the Tip Harvester and archive of TheList go to:
> > > http://lists.evolt.org Workers of the Web, evolt !
> >
> >--
> >Michael Pemberton
> >mpember at phreaker.net
> >ICQ: 12107010
> >
> >
> >---------------------------------------
> >For unsubscribe and other options, including
> >the Tip Harvester and archive of TheList go to:
> >http://lists.evolt.org Workers of the Web, evolt !
> 
> ---------------------------------------
> For unsubscribe and other options, including
> the Tip Harvester and archive of TheList go to:
> http://lists.evolt.org Workers of the Web, evolt !

-- 
Michael Pemberton
mpember at phreaker.net
ICQ: 12107010