[thelist] robots and the whitehouse.gov site

[Damien COLA]

Because if the site has no commercial interest in having more traffic,
they'd want people to find only the start page and look for information
with the embed navigation.

-----Original Message-----
My question is, why would anyone ever want to disallow their content 
from being indexed?  I can understand the want to disallow 
yourdomain.com/personalstuff or something like that, but why pretty much

all your content?