[thelist] Very Large MySQL Table

[Phil Turmel]

Phil Turmel wrote:
> Hi Fred,
> 
> Fred Jones wrote:
>> Data is a flat text file like this:
>>
>> aaa NS 1.2.3.4
>> bbb NS 5.6.7.8
>>
>> meaning that aaa has DNS pointed to 1.2.3.4 etc.
>>
>>> They don't use relational databases - relational databases are relatively
>>> slow in the "database" world.
> 
> Have you considered using other tools?
> 
> May I suggest grep?
> 
> First, take the zone file and strip off all but the domain name, like so:
> 
> cut -d\  -f1 <zone.txt >domains.txt
> 
> (note the two spaces after the "d\")
> 
> Then you can grep for patterns to your heart's content:
> 
> grep copperhydride <domains.txt
> 
> 
> I'm betting both the size of domains.txt and the grep time are better
> than any database solution by at least an order of magnitude.
> 
> HTH,
> 
> Phil

Thinking about this some more... This approach is likely to be
disk-limited, not cpu-limited, so I'd used a compressed domains file,
like so:

cut -d\  -f1 <zone.txt |gzip >domains.gz
zcat domains.gz |grep copperhydride


You might even have enough excess cpu to do multiple keyword searches
together (using alternate pattern regexp notation) after a single zcat:

zcat domains.gz |grep 'copperhydride\|nickelcadmium'

Please let us know how it turns out.

Phil