Numpy: Stop Numpy.array() From Trying To Reconcile Elements. Create Ndarry From List Without Trying To Merge / Reconcile The Elements

I have two 2d matrices in a list, which i want to convert to a numpy array. Below are 3 examples a,b,c . >>> import numpy as np >>> a = [np.zeros((3,5)), np.zeros

Solution 1:

Numpy still complains even if you explicitly pass object as the dtype:

>>> np.array(b, dtype=object)
Traceback (most recent calllast):
  File "<stdin>", line 1, in<module>
ValueError: could not broadcast input arrayfrom shape (3,5) into shape (3)

Essentially, numpy is not really written around using dtype=object, it always assumes you want an array with a primitve numeric or structured dtype.

So I think your only option is something like:

>>> arr = np.empty(len(b), dtype=object)
>>> arr[:] = b
>>> arr
array([array([[0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.]]),
       array([[0., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0., 0., 0., 0., 0.]])], dtype=object)

And just for fun, you can use the actual np.ndarray type constructor, although this isn't very easy:

>>> np.ndarray(dtype=object, shape=len(b), buffer=np.array(list(map(id, b)),dtype=np.uint64))
array([array([[0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.]]),
       array([[0., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0., 0., 0., 0., 0.]])], dtype=object)

And note, that relies on a CPython implementation detail, that id is simply the address of the python object. So mostly I'm just showing it for fun.

Solution 2:

In the latest version we are starting to see a warning:

In [185]: np.__version__                                                                             
Out[185]: '1.19.0'
                                                
In [187]: np.array([np.zeros((3,5)), np.zeros((2,9))])                                               
/usr/local/bin/ipython3:1: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  #!/usr/bin/python3
Out[187]: 
array([array([[0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.]]),
       array([[0., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0., 0., 0., 0., 0.]])], dtype=object)

It still makes the object dtype array. In the matching first dimension case we get the warning and error.

In [188]: np.array([np.zeros((3,5)), np.zeros((3,9))])                                               
/usr/local/bin/ipython3:1: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object'when creating the ndarray
  #!/usr/bin/python3
---------------------------------------------------------------------------
ValueError                                Traceback (most recent calllast)
<ipython-input-188-b6a4475774d0>in<module>----> 1 np.array([np.zeros((3,5)), np.zeros((3,9))])

ValueError: could not broadcast input arrayfrom shape (3,5) into shape (3)

Basically np.array tries, as first step, to make a multidimensional numeric array. Failing that it takes two routes - make an object dtype array or failure. Details are buried in compiled code.

The preallocate and assignment is the best way if you want full control over how the object array is created.

In [189]: res=np.empty(2,object)                                                                     
In [191]: res[:] = [np.zeros((3,5)), np.zeros((3,9))]