[thelist] Google is indexing me......

Patrick Berry pberry at piratehaven.org
Sun Sep 9 20:31:28 CDT 2001

>>> Michael Pemberton had the thought that... <<<

> It might even be worth using http-agent detection to block the spider
> from accessing that directory.  This will stop it from putting 404 pages
> in the google result.

Use a robots.txt file.



More information about the thelist mailing list