Python Write Dynamically Huge Files Avoiding 100% Cpu Usage

I am parsing a huge CSV approx 2 GB files with the help of this great stuff. Now have to generate dynamic files for each column in a new file where column name as file name. So I w

Solution 1:

There is number of ways to achieve this:

• Nice the process - plain and simple.

• cpulimit - just pass your script and cpu usage as parameters:

  cpulimit -P /path/to/your/script -l 20

• Python's resource package to set limits from the script. Bear in mind it works with absolute CPU time.