[thesite] recently on thelist

Daniel J. Cody djc at starkmedia.com
Mon Jan 15 16:48:31 CST 2001

Seth Bienek wrote:

>> would it be possible to walk through like 100 msgs at a time rather than
>> try to parse the whole thing at once?

> Yes, it's possible.  The only drawback I see is that we'd have to save the
> loop count in an application variable. (You're talking about running it
> every x minutes, right?)
Actually, just for the first time when we do the initial load.. After 
that, we can do it every x minutes and won't need a loop cus it will be 
a new file(if i'm reading your last paragraph correctly)

> Yes, big time.  I was processing at a rate of about 8-10 messages/second on
> my end.. When I copied the files up there and ran a couple of tests using
> the database, it was running about 10 seconds PER message (basically taking
> 100x longer).  That's one reason I was thinking that parsing it into XML and
> then tossing that into the DB might be better..  I dunno though, might work

Ok, i'll see where i can get tonite on that..

Sound good?


More information about the thesite mailing list