[thelist] Very Large MySQL Table

Steve Bigelow steve.bigelow at tkcsb.com
Mon Feb 23 21:59:40 CST 2009

I haven't been following this thread too closely, so apologies if this has
already been suggested...

Would a full text search indexer, like Sphinx Search
(http://www.sphinxsearch.com/), work for you?

It would probably take a long time to create the index, but once it's
created the search results should be almost instantaneous.

Steve Bigelow
TKCSB Images

-----Original Message-----
From: thelist-bounces at lists.evolt.org
[mailto:thelist-bounces at lists.evolt.org] On Behalf Of Fred Jones
Sent: Monday, February 23, 2009 12:27 PM
To: thelist at lists.evolt.org
Subject: Re: [thelist] Very Large MySQL Table

This solution I think might be almost as CPU intensive as just searching. :)

I am still trying to get the text file into MySQL so I can massage it
a tiny bit, and then spit it out again as a text file and then try the
grep solution suggested.


> Off the top of my head -- SQL gurus feel free to point and laugh.
> You could also index it with a simple 26-digit number -- each digit is
> the number of times a particular letter appears (up to 9, anyway).
> Then it's a single index.  To check for the frequency of "e", look at
> the 5th digit... and so forth.
> Or go binary -- 0s or 1s simply indicate "contains this letter" or not.
> Then you could have a separate index of the alphabet based on
> frequency within the English language, and based on that, search the
> *least* common letters first.  If searching for the string "quaker",
> it would look for "q" first -- not because it's the first in the
> string, but because it's the least common letter.  if it's not present
> move on to the second least-common.
> This allows you to eliminate candidates without grepping the whole
> string -- you're just pulling particular digits (by position) out of
> the index number.

* * Please support the community that supports you.  * *

For unsubscribe and other options, including the Tip Harvester 
and archives of thelist go to: http://lists.evolt.org 
Workers of the Web, evolt ! 

More information about the thelist mailing list