[thelist] Crawling for headers

[James Q. Stansfield]

    If you can still get a license, M$' SiteServer 3 does this perfectly.
It'll build an index based on any Meta tag you specify and you can point it
to crawl directories and/or websites. Unfortunately, SS3 is now discontinued
and the repalcement is M$ Commerce Server and it is very expensive
comparatively.

    //James
----- Original Message -----
From: "Peter Johansson" <peter at johansson.org>
To: <thelist at lists.evolt.org>
Sent: Tuesday, March 19, 2002 8:14 AM
Subject: [thelist] Crawling for headers


> Hi,
>
> I'm in need of a tool that can crawl through websites and look for
> specific meta-data in the pages. For instance to identify which pages that
> have meta-data with expiry dates and that have already expired, or to
> locate pages that don't contain specific meta-data at all. The more
> generic the better.
>
> It's intended for a large intranet containing of a significant number of
> sites, hosted at separate locations, so a simple find and grep won't do.
>
> Anyone knows of such a tool?
>
> Regards,
> Peter
>
> --
> For unsubscribe and other options, including
> the Tip Harvester and archive of thelist go to:
> http://lists.evolt.org Workers of the Web, evolt !
>