[thelist] Downloading a whole site

Simon Willison cs1spw at bath.ac.uk
Sun Jul 27 11:51:13 CDT 2003


Hi John,

Sunday, July 27, 2003, 10:36:09 AM, you wrote:
> Favourite tool for downloading a whole site?

WGET. It's a Unix command line tool but it's available for Windows as
well (as a pre-compiled binary). It's incredibly powerful but since
it's command line only it can be a bit difficult to use; that said,
the learning curve is well worth it. You can set it up to suck down
whole sites recursively, rewriting relative URLs and replicating
directory structures all with a single (if lengthy) command. There's
also excellent documentation available for it.

WGET for Windows:
http://www.qtm.net/~twegscheid/wget.html

A tip on using it from my blog archives:
http://simon.incutio.com/archive/2002/07/02/wgetTip

WGET Manual:
http://cclib.nsu.ru/projects/gnudocs/gnudocs/wget/wget_toc.html

Hope that helps,

Simon Willison
-- 
http://simon.incutio.com/



More information about the thelist mailing list