[thelist] What search bots follow [was: Gray Hairs - Google Mini]
kasimir-k
evolt at kasimir-k.fi
Wed Feb 15 13:30:39 CST 2006
Asif Suria scribeva in 15/02/2006 18:25:
>>From what I have heard, as search engines have a ton
> of sites to crawl, they sometimes will only crawl a
> couple of pages per site (most recently updated pages,
> pages with the most links coming from outside) and try
> to get to the rest later
All this got me curious, and I thgought I'd check with a site just
launched: http://udumood.com/
I had a look at the access logs, and Googlebot has been a daily visitor
there - but only requesting index page - not following to other pages.
The URLs look like this: http://udumood.com/index.html?About_Samuli
Which I believe should be just fine for G-bot. But why doesn't it follow
the links then? Any ideas?
A search for words "udu peace space" will bring the site at number one,
so Google hasn't any objections to the site per se.
Then there's msnbot - it has visited all pages, but still it has
diffiulties showing it in its results. A search for word 'udu', 'mood'
and 'music' will bring my site in the top, as I have a link to
udumood.com, but udumood.com itself is nowhere to be seen... Only if I
wrap the three words in quotes msn search gives it - so msn has it, but
is utterly reluctant to give it... Any ideas on this?
thanks,
.k
More information about the thelist
mailing list