[thelist] To Crawl... or Not To Crawl [Solved]

Rob Smith rob.smith at lexjet.com
Wed Jul 12 08:40:06 CDT 2006


Just to follow up for future reference, the Mini will crawl redirected
URL's and if there is a specific piece of information on any particular
page that you don't want the Mini to crawl, simply put these tags around
the data to hide and the Mini will simply ignore the data:

Do not crawl this<!--googleoff: all--> sensitive<!--googleon:all-->
data.

Thanks,

Rob Smith
LexJet
rob.smith at lexjet.com
http://www.lexjet.com
(800)453-9538
(941)330-1210 Int'l
(941)330-1220 Fax
1680 Fruitville Road, 3rd Floor
Sarasota, FL 34236




More information about the thelist mailing list