[thelist] What search bots follow [was: Gray Hairs - Google Mini]

kasimir-k evolt at kasimir-k.fi
Wed Feb 15 13:30:39 CST 2006


Asif Suria scribeva in 15/02/2006 18:25:
>>From what I have heard, as search engines have a ton
> of sites to crawl, they sometimes will only crawl a
> couple of pages per site (most recently updated pages,
> pages with the most links coming from outside) and try
> to get to the rest later 

All this got me curious, and I thgought I'd check with a site just 
launched: http://udumood.com/

I had a look at the access logs, and Googlebot has been a daily visitor 
there - but only requesting index page - not following to other pages. 
The URLs look like this: http://udumood.com/index.html?About_Samuli
Which I believe should be just fine for G-bot. But why doesn't it follow 
the links then? Any ideas?

A search for words "udu peace space" will bring the site at number one, 
so Google hasn't any objections to the site per se.

Then there's msnbot - it has visited all pages, but still it has 
diffiulties showing it in its results. A search for word 'udu', 'mood' 
and 'music' will bring my site in the top, as I have a link to 
udumood.com, but udumood.com itself is nowhere to be seen... Only if I 
wrap the three words in quotes msn search gives it - so msn has it, but 
is utterly reluctant to give it... Any ideas on this?

thanks,
.k



More information about the thelist mailing list