[thelist] sleep, etc to make a script not take resources

[James Hardy]

2009/10/13 Hassan Schroeder <hassan.schroeder at gmail.com>:
>
> door #3: Since the regular expression comparison is by far the most
> computationally intensive part of this, parse the CSV file locally and
> generate a script of insert statements; upload that, run on DB server.
>
> In a loop with sleep()s included, if it makes you feel better. :-)
>

In my experience of MySQL, using a LOAD DATA IN FILE statement[1] to
insert structured (eg CSV) data uses almost no resources and takes
very little time compared with generated MySQL scripts with thousands
of insert statements, which tend to eat up resources and take an age.

In this case, I would use a simple grep (or awk if more manipulation
is required) command to generate a CSV with only the matching lines
locally and then simply import this data.

[1] http://dev.mysql.com/doc/refman/5.1/en/load-data.html

-- 
James Hardy