[thelist] My first Robot

Arlen.P.Walker at jci.com Arlen.P.Walker at jci.com
Tue Jul 23 13:13:01 CDT 2002

>If you put the robot meta tag mentioned on robotstxt.org on all the pages
>your website,
> 1. would the site continue to get found by search engines and
>  2. would it stop email addresses listed on your site from getting
>Naturally, I am looking for a YES answer to these questions.

You won't get it. ;{>}

1) If you use the "noindex" value on the tag, the page will *not* be found
by any search engines obeying the protocol. You're telling them not to
index the page, so they won't. If you just use "nofollow", then the page
will be found, but none of the pages it links to will be indexed. After
all, you told it not to follow the links it found.

2) The REP is voluntary. Well-behaved robots obey the protocol; ill-behaved
ones do not. Which robot belongs to the spammer? ;{>}

Have fun,
Chief Managing Director In Charge, Department of Redundancy Department
DNRC 224

Arlen.P.Walker at JCI.Com
In God we trust; all others must provide data.
Opinions expressed are mine and mine alone.
If JCI had an opinion on this, they'd hire someone else to deliver it.

More information about the thelist mailing list