[thelist] How does Google access protected sites

[Joshua Olson]

----- Original Message -----
From: "Feingold Josh S" <Josh.S.Feingold at irs.gov>
Sent: Tuesday, August 13, 2002 12:11 PM


> I noticed that Google gets access to sites that usually need a username
and
> password to enter.  Does anyone know how Google manages to do this?
>
> Josh

I have a theory.  Perhaps they actually open the door for the google spider
using it's IP or something else.  Porno sites, for example, would benefit
from indexing by the search engines.

-joshua