[thelist] Are search engine spiders JavaScript-enabled?

Kevin D. White nonzero at well.com
Fri Nov 9 10:23:05 CST 2001

Check your server logs.  The answer is in there.  Most legit crawlers
clearly identify themselves.  It's also worth noting that many search sites
(Yahoo, Google) do not use crawlers to gather site info.

Side note: Your method of redirection is going to trap JS-enabled people on
your site.  Every time they hit the back button, they are going to hit the
redirect page again.  Investigate the location.replace() method to solve
this problem.
----- Original Message -----
From: "Ben Gustafson" <cbg3 at earthlink.net>
> Now, when a search engine spider visits default.asp, which version of the
page is it going to crawl: The version built with js==true, the other
version, or both? Is it going to ignore the js==true version since calling
that version involves a URL with a query string and a browser redirect (to
the same page)? Or are the questions not even relevant?

More information about the thelist mailing list