[FIXED] Store numpy.array in cells of a Pandas.DataFrame

Issue

I have a dataframe in which I would like to store 'raw' numpy.array:

df['COL_ARRAY'] = df.apply(lambda r: np.array(do_something_with_r), axis=1)

but it seems that pandas tries to 'unpack' the numpy.array.

Is there a workaround? Other than using a wrapper (see edit below)?

I tried reduce=False with no success.

EDIT

This works, but I have to use the 'dummy' Data class to wrap around the array, which is unsatisfactory and not very elegant.

class Data:
    def __init__(self, v):
        self.v = v

meas = pd.read_excel(DATA_FILE)
meas['DATA'] = meas.apply(
    lambda r: Data(np.array(pd.read_csv(r['filename'])))),
    axis=1
)

Solution

Use a wrapper around the numpy array i.e. pass the numpy array as list

a = np.array([5, 6, 7, 8])
df = pd.DataFrame({"a": [a]})

Output:

             a
0  [5, 6, 7, 8]

Or you can use apply(np.array) by creating the tuples i.e. if you have a dataframe

df = pd.DataFrame({'id': [1, 2, 3, 4],
                   'a': ['on', 'on', 'off', 'off'],
                   'b': ['on', 'off', 'on', 'off']})

df['new'] = df.apply(lambda r: tuple(r), axis=1).apply(np.array)

Output :

     a    b  id            new
0   on   on   1    [on, on, 1]
1   on  off   2   [on, off, 2]
2  off   on   3   [off, on, 3]
3  off  off   4  [off, off, 4]

df['new'][0]

Output :

array(['on', 'on', '1'], dtype='<U2')

Answered By - Bharath