[thelist] sleep, etc to make a script not take resources
Jason Handby
jason.handby at corestar.co.uk
Tue Oct 13 15:42:54 CDT 2009
> door #3: Since the regular expression comparison is by far the most
> computationally intensive part of this, parse the CSV file locally and
> generate a script of insert statements; upload that, run on DB server.
>
> In a loop with sleep()s included, if it makes you feel better. :-)
Another option that just occurred to me: import your entire CSV file
into a database table, then use the DB engine itself to do the pattern
matching and inserting into your other table. MySQL supports regular
expressions, for example:
http://dev.mysql.com/doc/refman/5.1/en/regexp.html
If you can come up with the right database query, it'll probably be more
efficient to do it that way than to script it yourself. Presumably
something like
INSERT INTO my_table
SELECT f1, f2, f3, f_examineme
FROM temp_table
WHERE f_examineme REGEXP 'pattern'
Jason
More information about the thelist
mailing list