[thelist] sleep, etc to make a script not take resources

Jason Handby jason.handby at corestar.co.uk
Tue Oct 13 15:42:54 CDT 2009

> door #3: Since the regular expression comparison is by far the most
> computationally intensive part of this, parse the CSV file locally and
> generate a script of insert statements; upload that, run on DB server.
> In a loop with sleep()s included, if it makes you feel better. :-)

Another option that just occurred to me: import your entire CSV file
into a database table, then use the DB engine itself to do the pattern
matching and inserting into your other table. MySQL supports regular
expressions, for example:


If you can come up with the right database query, it'll probably be more
efficient to do it that way than to script it yourself. Presumably
something like

 INSERT INTO my_table
	SELECT f1, f2, f3, f_examineme
	FROM temp_table
	WHERE f_examineme REGEXP 'pattern'


More information about the thelist mailing list