[thelist] What search bots follow [was: Gray Hairs - Google Mini]

Christian Heilmann codepo8 at gmail.com
Thu Feb 16 06:25:05 CST 2006

On 2/16/06, Ian Anderson <ian at zstudio.co.uk> wrote:
> kasimir-k wrote:
> > But why doesn't it follow the links then? Any
> > ideas?
> I met someone from (what seems a contradiction in terms) a reputable SEO
> company, who are Google partners. He explained that this is normal
> practice, and that new sites are indexed gradually over a period of
> time. Guessing at the reason, I think this is to reduce the load on the
> crawler, so all new URLs are added to the index as rapidly as possible,
> but without depth, and the depth gets added thereafter as a series of
> lower priority passes.
> What you say corroborates what he said, which is interesting.

This would also stop black hat SEO (what an idiotic term) practices
dead in their tracks. The first thing bad SEO companies do is bombard
the page with hundreds of links with keywords in them.

Chris Heilmann
Blog: http://www.wait-till-i.com
Writing: http://icant.co.uk/
Binaries: http://www.onlinetools.org/

More information about the thelist mailing list