[thelist] googlebot strange behaviour
listman at hazard0us.org
listman at hazard0us.org
Thu Aug 29 02:44:00 CDT 2002
I 've been using URL mapping on a website, for a while now to generate
friendly URLs.
/script?type=utils maps to something like -> /home/pages/utils/index.html
Google incidentally has refused to index this website beyond the 1st page.
I was checking up the http logs for this site recently and i realized
googlebot is making requests on :
/home/page/utils/ (and returning with a 404)
instead of
/home/page/utils/index.html (inspite of all the HREFs pointing to
/home/pages/utils/index.html)
Other search spiders like Scooter (altavista) pick up the links properly.
Anybody ever had a similar problem ?
(For some technical reasons I cannot create a valid mapping to
/home/page/utils/ that could be identical
to /home/page/utils/index.html - so that route is out of the question).
------
Ashok Hariharan
http://www.unganisha.org
------
More information about the thelist
mailing list