[thelist] browser requests

Mark Groen evolt at markgroen.com
Sat Jan 28 13:15:37 CST 2006


On Sat, 2006-01-28 at 14:03 -0500, Robert Vreeland wrote:

> Ok, to rephrase my question: Will a browser request a url if it is only
> located inside a javascript function that is never called?

hmm, web browser or bot? According to Google:
Googlebot follows HREF links and SRC links.
http://www.google.com/webmasters/bot.html

On the other hand, a web browser follows whatever is clicked/entered on,
so if it's publicly accessible, yes, if it is called.

As mentioned, even simple tools like wget can crawl your site regardless
of whether you have a robots file or not, the only way to keep it from
being seen eventually is through htaccess and a protected directory.

As a general rule, I've found that bots don't parse/follow javascript
links, the processing required is more than it's worth methinks. That
doesn't mean a person can't set up a bot to do just that though, if they
are so inclined.

hope that helps!

-- 
cheers,

        Mark




More information about the thelist mailing list