Distributed bandwidth (was RE: [Theforum] Re: [Content] Mail Alert! - Sat Jun 22 31226)

William Anderson neuro at well.com
Mon Jun 24 04:03:25 CDT 2002

---- Original Message ----
From: ".jeff" <jeff at members.evolt.org>
To: <theforum at lists.evolt.org>
Sent: Monday, June 24, 2002 8:11 AM
Subject: RE: Distributed bandwidth (was RE: [Theforum] Re: [Content]
Mail Alert! - Sat Jun 22 31226)

> beau,
> let me preface this reply by saying that william (neuro)

hullooo!  </ian-mcaskill>

> and i have
> talked at length about this in irc today.  i've been listening to all
> the conversations about this issue and have most of a data model
> together that will address most everything.
> [snip]
> i found kb listings for the top-level folders, but no information on
> the total number of files.  dean, can you chime in with some figures
> for us?

bear in mind this includes extra cruft like cfm, html etc ...

[neuro at leo beo]$ find | wc -l

[neuro at leo beo]$ ls -lF | grep / | wc -l

(there are two support directories for beo files, one directory for
unchecked browsers, all else are browsers = 115 browsers)

[neuro at leo beo]$ du -sch .
4.6G    .

[neuro at leo beo]$ du -sch ie
886M    ie

[neuro at leo beo]$ du -sch mozilla
960M    mozilla

so you can see where the weighted distribution lies.

> [snip]
> our copy of the archive would sit behind some form of login or
> accessible via http or anonymous ftp at all. our copy would act as
> the master copy.

imho, rsync is the first and best service to use for mirrorers to grab and
update their parts, or all, of the archive.  http and ftp should be 'last

> if a mirror finds and makes available a browser we
> don't have in our master copy, someone (hopefully the mirror) would
> let us know and we'd grab a copy to put in our private collection
> just in case the mirror should ever go offline and not come back.

in a similar vein, we'd still keep our mirror up-to-date and stable, even
for browsers we'd never want to mirror again, such as IE or Mozilla, due to
the bandwidth cost.  For example, Mozilla have an excellent archive of their
milestone, beta, RC and now final releases.  It makes sense to point to
ftp.mozilla.org in the first instance for as many iterations of mozilla as
possible.  But if the moz.org ftp site went away for some reason, we'd know
that we'd still be able to use our distributed mirrors to serve up the older
mozilla browsers.

> [snip]

great spec and write up jeff - i was about to do the same until i saw this
post :)  thanks!

_ __/|   ___  ___ __ _________  "Hell hath no fury like a woman scorned
\`O_o'  / _ \/ -_) // / __/ _ \  for Sega." -- Brodie, 'Mallrats'
=(_ _)=/_//_/\__/\_,_/_/  \___/ @ well.com :: William Anderson
   U - Ack! Phttpt! Thhbbt!     http://neuro.me.uk/

More information about the theforum mailing list