[thelist] Very Large MySQL Table

Chris Dorer cdorer at gmail.com
Sat Mar 7 16:54:59 CST 2009


You may want to do some research on the term map/reduce and then investigate

Your searching would be easier, then also, couple hadoop with lucene or solr
and then your searches will be fast.


On Mon, Feb 9, 2009 at 5:38 PM, Fred Jones <fredthejonester at gmail.com>wrote:

> You may recall that I recently asked about running searches against
> domain names. I have now gotten a .com zone file from Verisign, which
> is basically a list of all registered .com domain names. I imported it
> into my local MySQL. There are ~ 185,000,000 domain names and it's too
> much for my workstation to run searches on. Just running count(*)
> takes 10 minutes. It's 6G of data.
> We want to run searches basically to find domain names with certain
> words or phrases.
> We don't have a dedicated server for this but his web account does
> have disk space for 6Gs, but I am wondering if we will be able to run
> searches. His idea was to break it up into 8 pieces, and then to run
> each search 8 times, against just that one eighth of the full list.
> Anyone have any better idea?
> Thanks!
> --
> * * Please support the community that supports you.  * *
> http://evolt.org/help_support_evolt/
> For unsubscribe and other options, including the Tip Harvester
> and archives of thelist go to: http://lists.evolt.org
> Workers of the Web, evolt !

You always have the option of pitching baseballs at empty spray paint cans
in a cul-de-sac in a Cleveland suburb.

More information about the thelist mailing list