[thelist] Very Large MySQL Table

Fred Jones fredthejonester at gmail.com
Mon Feb 23 13:27:06 CST 2009

This solution I think might be almost as CPU intensive as just searching. :)

I am still trying to get the text file into MySQL so I can massage it
a tiny bit, and then spit it out again as a text file and then try the
grep solution suggested.


> Off the top of my head -- SQL gurus feel free to point and laugh.
> You could also index it with a simple 26-digit number -- each digit is
> the number of times a particular letter appears (up to 9, anyway).
> Then it's a single index.  To check for the frequency of "e", look at
> the 5th digit... and so forth.
> Or go binary -- 0s or 1s simply indicate "contains this letter" or not.
> Then you could have a separate index of the alphabet based on
> frequency within the English language, and based on that, search the
> *least* common letters first.  If searching for the string "quaker",
> it would look for "q" first -- not because it's the first in the
> string, but because it's the least common letter.  if it's not present
> move on to the second least-common.
> This allows you to eliminate candidates without grepping the whole
> string -- you're just pulling particular digits (by position) out of
> the index number.

More information about the thelist mailing list