[thelist] Very Large MySQL Table

[Chris Dorer]

That's VERY HUGE.

You may want to do some research on the term map/reduce and then investigate
hadoop.

Your searching would be easier, then also, couple hadoop with lucene or solr
and then your searches will be fast.

Chris

On Mon, Feb 9, 2009 at 5:38 PM, Fred Jones <fredthejonester at gmail.com>wrote:

> You may recall that I recently asked about running searches against
> domain names. I have now gotten a .com zone file from Verisign, which
> is basically a list of all registered .com domain names. I imported it
> into my local MySQL. There are ~ 185,000,000 domain names and it's too
> much for my workstation to run searches on. Just running count(*)
> takes 10 minutes. It's 6G of data.
>
> We want to run searches basically to find domain names with certain
> words or phrases.
>
> We don't have a dedicated server for this but his web account does
> have disk space for 6Gs, but I am wondering if we will be able to run
> searches. His idea was to break it up into 8 pieces, and then to run
> each search 8 times, against just that one eighth of the full list.
>
> Anyone have any better idea?
>
> Thanks!
> --
>
> * * Please support the community that supports you.  * *
> http://evolt.org/help_support_evolt/
>
> For unsubscribe and other options, including the Tip Harvester
> and archives of thelist go to: http://lists.evolt.org
> Workers of the Web, evolt !
>



-- 
You always have the option of pitching baseballs at empty spray paint cans
in a cul-de-sac in a Cleveland suburb.