[thelist] Crawler pages: listing the files in a site

[Jon Haworth]

Praise Xenu!

Checks your links, then builds a nested-<ul> list of every page as part of
the report.

http://home.snafu.de/tilman/xenulink.html

HTH
Jon


-----Original Message-----
From: Tony Page [mailto:zamba at zamba.com]
Sent: 01 June 2001 09:54
To: thelist at lists.evolt.org
Subject: [thelist] Crawler pages: listing the files in a site


Is there an easy way to construct a crawler page, ie. a webpage containg
simple links to every (static) page in a website? I've found these very
effective in facilitating search engine etc spidering in the past, but I'd
like to extend this technique to some larger websites. A scanning program of
some kind would be useful, as this would make updating easier as well.
Any ideas, anybody?
Tony Page


**********************************************************************
'The information included in this Email is of a confidential nature and is 
intended only for the addressee. If you are not the intended addressee, 
any disclosure, copying or distribution by you is prohibited and may be 
unlawful. Disclosure to any party other than the addressee, whether 
inadvertent or otherwise is not intended to waive privilege or confidentiality'

**********************************************************************