[thesite] Tips Database

[Dean Mah]

Daniel J. Cody writes:

> i have the mbox file from the first year of thelist, and it may be
> cool to put that in there..


It's kind of a pain in the ass and will take some time.  The biggest
problem is that since the tips do not have a rigid structure, tip
author's have tended to abuse the format and each chunk needs to be
manually inspected especially if higher-level processing is to be done
on them.


> also, was checking out the data, and it looks like you've got just
> the <tip>'s in there. would i be a serious bitch if i asked that you
> just take the entire message body and insert it into thelist table?
> then we could run through that, grep out tips, and associate them
> with a messageid from thelist table..
>
> does that make sense? i just figure if we should get everything we
> can out of the mbox's while you're kind enough to do this :)


Do you want every message from the mbox inserted into THELIST table?

Currently, I am only inserting the messages with the tips in them.
I'm also doing processing of the tips.  (This is modified Tip
Harvester code.)  Is there a point in doing this?


Dean