Runtimewarning: Invalid Value Encountered In Greater

I tried to implement soft-max with the following code (out_vec is a numpy vector of floats): numerator = np.exp(out_vec) denominator = np.sum(np.exp(out_vec)) out_vec = numerator/d

Solution 1:

Your problem is caused by the NaN or Inf elements in your out_vec array. You could use the following code to avoid this problem:

if np.isnan(np.sum(out_vec)):
    out_vec = out_vec[~numpy.isnan(out_vec)] # just remove nan elements from vector
out_vec[out_vec > 709] = 709
...

or you could use the following code to leave the NaN values in your array:

out_vec[ np.array([e > 709if ~np.isnan(e) elseFalsefor e in out_vec], dtype=bool) ] = 709

Solution 2:

In my case the warning did not show up when calling this before the comparison (I had NaN values getting compared)

np.warnings.filterwarnings('ignore')

Solution 3:

IMO the better way would be to use a more numerically stable implementation of sum of exponentials.

from scipy.misc import logsumexp
out_vec = np.exp(out_vec - logsumexp(out_vec))

Solution 4:

If this happens because of your NaN value, then this might help:

out_vec[~np.isnan(out_vec)] = out_vec[~np.isnan(out_vec)] > 709

This does the greater operation for none NaN values and the rest remains the same. If you need the rest to be False, then do this too:

out_vec[np.isnan(out_vec)] = False