[FIXED] How do I change a value in a .npz file?

Issue

I want to change one value in an npz file.

The npz file contains several npy's, I want all but one ( 'run_param' ) to remain unchanged and I want to save over the original file.

This is my working code:

DATA_DIR = 'C:\\Projects\\Test\\data\\'
ass_file = np.load( DATA_DIR + 'assumption.npz' )
run_param = ass_file['run_param']

print ass_file['run_param'][0]['RUN_MODE']
ass_file['run_param'][0]['RUN_MODE'] = 1        (has no effect)
print ass_file['run_param'][0]['RUN_MODE']

print run_param[0]['RUN_MODE']
run_param[0]['RUN_MODE'] = 1
print run_param[0]['RUN_MODE']

This produces:

0
0
0
1

I can't seem to change the value in the original npy.

My code to save afterward is:

np.savez( DATA_DIR + 'assumption.npz', **ass_file )   #
ass_file.close()

How to make this work?

Solution

Why your code did not work

What you get from np.load is a NpzFile, which may look like a dictionary but isn't. Every time you access one if its items, it reads the array from file, and returns a new object. To demonstrate:

>>> import io
>>> import numpy as np
>>> tfile = io.BytesIO()  # create an in-memory tempfile
>>> np.savez(tfile, test_data=np.eye(3))  # save an array to it
>>> tfile.seek(0)  # to read the file from the start
0
>>> npzfile = np.load(tfile)
>>> npzfile['test_data']
array([[ 1.,  0.,  0.],
       [ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])
>>> id(npzfile['test_data'])
65236224
>>> id(npzfile['test_data'])
65236384
>>> id(npzfile['test_data'])
65236704

The id function for the same object is always the same. From the Python 3 Manual:

id(object) Return the “identity” of an object. This is an integer which is guaranteed to be unique and constant for this object during its lifetime. ...

This means that in our case, each time we call npz['test_data'] we get a new object. This "lazy reading" is done to preserve memory and to read only the required arrays. In your code, you modified this object, but then discarded it and read a new one later.

---

So what can we do?

If the npzfile is this weird NpzFile instead of a dictionary, we can simply convert it to a dictionary:

>>> mutable_file = dict(npzfile)
>>> mutable_file['test_data'][0,0] = 42
>>> mutable_file
{'test_data': array([[ 42.,   0.,   0.],
                     [  0.,   1.,   0.],
                     [  0.,   0.,   1.]])}

You can edit the dictionary at will and save it.

Answered By - jarondl