[thelist] Google is indexing me......

CDitty mail at redhotsweeps.com
Sun Sep 9 10:24:13 CDT 2001


Which directory?  Afraid I don't understand completely.  The forum 
directory or the one below?  The forums can be accessed anywhere on the 
site and the one I listed below was just one of hundreds of directories in 
the error log.

At 10:16 AM 9/9/2001, you wrote:
>It might even be worth using http-agent detection to block the spider
>from accessing that directory.  This will stop it from putting 404 pages
>in the google result.
>
>CDitty wrote:
> >
> > As far as I can tell, there are not any users yet hitting these links.  It
> > is just the Google spider causing all these errors.   Since it is the
> > spider causing problems, my 404 page will have no effect.
> >
> > Will the mod_rewrite take care of links like this -
> > /One_Time/Household_Items/Lawn_and_Garden/\forum\ ?
> >
> > Chris
> >
> > At 09:57 AM 9/9/2001, you wrote:
> >
> > >One option would be to just go with the flow and use mod_rewrite to
> > >correct the URL.
> > >
> > >That is, if you have mod_rewrite available to you. The other option is
> > >using the 404 document, but I don't think that will fix your errorlog
> > >problem
> > >
> > >hth,
> > >
> > >--
> > >mattwarden
> > >mattwarden.com
> > >
> > >On Sep 9, CDitty had something to say about [thelist] Google is 
> indexing...
> > >
> > > >Google is indexing me and is filling up my error log.  For some reason,
> > > >they keep trying to hit my forums and are trying to call it with a 
> \forum
> > > >instead of /forum (see below).
> > > >
> > > >[Sun Sep 9 01:12:53 2001] [error] [client 216.239.46.153] File does not
> > > >exist: /One_Time/Household_Items/Lawn_and_Garden/\forum\
> > > >
> > > >
> > > >They are doing this on almost every single page.  Can anyone offer any
> > > >advice on fixing this?  I have verified all my links and they are
> > > >correct.  I would hate to have to block Google as I get tons of traffic
> > > >from them.
> > >
> > >
> > >---------------------------------------
> > >For unsubscribe and other options, including
> > >the Tip Harvester and archive of TheList go to:
> > >http://lists.evolt.org Workers of the Web, evolt !
> >
> > ---------------------------------------
> > For unsubscribe and other options, including
> > the Tip Harvester and archive of TheList go to:
> > http://lists.evolt.org Workers of the Web, evolt !
>
>--
>Michael Pemberton
>mpember at phreaker.net
>ICQ: 12107010
>
>
>---------------------------------------
>For unsubscribe and other options, including
>the Tip Harvester and archive of TheList go to:
>http://lists.evolt.org Workers of the Web, evolt !





More information about the thelist mailing list