[thelist] Removing previous posts from thelist archives.

Christian Heilmann codepo8 at gmail.com
Fri Aug 11 04:15:12 CDT 2006


On 8/11/06, Micky Hulse <micky at ambiguism.com> wrote:
> Christian Heilmann wrote:
> > <tip author="Chris Heilmann" type="Demo URLs on the list">
> > Every URL you post on the list will end up in the archive and thus
> > will be spidered by search engines and found by non-list members (yes,
> > and spammers). Therefore make sure that you don't post any sensitive
> > URLs with client information.
> > One common trick around that is to provide the links as
> > h**p://www.example.com although it is very likely that spambots do
> > recognize that pattern by now.
> > </tip>
>
> @Amanda
>
> Ouch... sorry.  :(
>
> @Christian
>
> Do you think snipping the urls would help?
>
> <http://snipurl.com/teindex.php>

I doubt it. The snipurl would be spidered and that one redirects...

Funnily enough it is much harder to get URLs not spidered than it is
getting them spidered. That is why some web sites like kelkoo.com have
Javascript: links. This is not done out of lack of knowledge or not
caring about the inaccessibility of them, but to avoid spidering of
pages that shouldn't be spidered.

Then again, the google bot is scarily good, and at times we discovered
that it does follow Javascript links and even flash redirects. 10
points for thorough scanning, 0 points as that takes away the "google
is the blind millionaire" argument when talking to clients about
scripting dependent links.

One option would be to set up a redirect with password.

-- 
Chris Heilmann
Book: http://www.beginningjavascript.com
Blog: http://www.wait-till-i.com
Writing: http://icant.co.uk/



More information about the thelist mailing list