[thelist] sleep, etc to make a script not take resources

Jason Handby jason.handby at corestar.co.uk
Tue Oct 13 15:42:54 CDT 2009


> door #3: Since the regular expression comparison is by far the most
> computationally intensive part of this, parse the CSV file locally and
> generate a script of insert statements; upload that, run on DB server.
> 
> In a loop with sleep()s included, if it makes you feel better. :-)

Another option that just occurred to me: import your entire CSV file
into a database table, then use the DB engine itself to do the pattern
matching and inserting into your other table. MySQL supports regular
expressions, for example:

  http://dev.mysql.com/doc/refman/5.1/en/regexp.html


If you can come up with the right database query, it'll probably be more
efficient to do it that way than to script it yourself. Presumably
something like

 INSERT INTO my_table
	SELECT f1, f2, f3, f_examineme
	FROM temp_table
	WHERE f_examineme REGEXP 'pattern'



Jason




More information about the thelist mailing list