[thelist] Very Large MySQL Table
Steve Bigelow
steve.bigelow at tkcsb.com
Mon Feb 23 21:59:40 CST 2009
I haven't been following this thread too closely, so apologies if this has
already been suggested...
Would a full text search indexer, like Sphinx Search
(http://www.sphinxsearch.com/), work for you?
It would probably take a long time to create the index, but once it's
created the search results should be almost instantaneous.
Steve Bigelow
TKCSB Images
-----Original Message-----
From: thelist-bounces at lists.evolt.org
[mailto:thelist-bounces at lists.evolt.org] On Behalf Of Fred Jones
Sent: Monday, February 23, 2009 12:27 PM
To: thelist at lists.evolt.org
Subject: Re: [thelist] Very Large MySQL Table
This solution I think might be almost as CPU intensive as just searching. :)
I am still trying to get the text file into MySQL so I can massage it
a tiny bit, and then spit it out again as a text file and then try the
grep solution suggested.
Thanks.
> Off the top of my head -- SQL gurus feel free to point and laugh.
>
> You could also index it with a simple 26-digit number -- each digit is
> the number of times a particular letter appears (up to 9, anyway).
>
> Then it's a single index. To check for the frequency of "e", look at
> the 5th digit... and so forth.
>
> Or go binary -- 0s or 1s simply indicate "contains this letter" or not.
>
> Then you could have a separate index of the alphabet based on
> frequency within the English language, and based on that, search the
> *least* common letters first. If searching for the string "quaker",
> it would look for "q" first -- not because it's the first in the
> string, but because it's the least common letter. if it's not present
> move on to the second least-common.
>
> This allows you to eliminate candidates without grepping the whole
> string -- you're just pulling particular digits (by position) out of
> the index number.
--
* * Please support the community that supports you. * *
http://evolt.org/help_support_evolt/
For unsubscribe and other options, including the Tip Harvester
and archives of thelist go to: http://lists.evolt.org
Workers of the Web, evolt !
More information about the thelist
mailing list