Long (>20million Element) Array Summation In Python Numpy

I am new to python and numpy so please excuse me if this problem is so rudimentary! I have an array of negative values (it is sorted): >>>neg [ -1.53507843e+02 -1.5320001

Solution 1:

As noted in the comments, you get float roundoff problems from summing up many millions of equal-signed numbers. One possible way around this could be to mix positive and negative numbers in the combined array, so that any intermediate results while summing up always stay roughly within the same order of magnitude:

neg = -100*numpy.random.rand(20e6)
pos = -neg
combined = numpy.zeros(len(neg)+len(pos))
combined[::2] = negcombined[1::2] = pos

Now combined.sum() should be pretty close to zero.

Maybe this approach will also help to improve the precision in the computation of the standard deviation.