[FIXED] Why cpython exposes 'PyTuple_SetItem' as C-API if tuple is immutable by design?

Issue

Tuple in python is immutable by design, so if we try to mutate a tuple object, python emits following TypeError which make sense.

>>> a = (1, 2, 3)
>>> a[0] = 12
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment

So my question is, if tuple is immutable by design why cpython exposes PyTuple_SetItem as C-API?.

From the documentation it's described as

int PyTuple_SetItem(PyObject *p, Py_ssize_t pos, PyObject *o)

Insert a reference to object o at position pos of the tuple pointed to by p. Return 0 on success. If pos is out of bounds, return -1 and set an IndexError exception.

Isn't this statement exactly equal to tuple[index] = value in python layer?. If the goal was to create a tuple from collection of items we could have use PyTuple_Pack.

Additional note:

After lot of trial and error with ctypes.pythonapi I managed to mutate tuple object using PyTuple_SetItem

import ctypes

from ctypes import py_object

my_tuple = (1, 2, 3)
newObj = py_object(my_tuple)

m = "hello"

# I don't know why I need to Py_DecRef here. 
# Although to reproduce this in your system,  no of times you have 
# to do `Py_DecRef` depends on no of ref count of `newObj` in your system.
ctypes.pythonapi.Py_DecRef(newObj)
ctypes.pythonapi.Py_DecRef(newObj)
ctypes.pythonapi.Py_DecRef(newObj)

ctypes.pythonapi.Py_IncRef(m)



PyTuple_SetItem = ctypes.pythonapi.PyTuple_SetItem
PyTuple_SetItem.argtypes = ctypes.py_object, ctypes.c_size_t, ctypes.py_object

PyTuple_SetItem(newObj, 0, m)
print(my_tuple) # this will print `('hello', 2, 3)`

Solution

Similarly, there is a PyTuple_Resize function with the warning

Because tuples are supposed to be immutable, this should only be used if there is only one reference to the object. Do not use this if the tuple may already be known to some other part of the code. The tuple will always grow or shrink at the end. Think of this as destroying the old tuple and creating a new one, only more efficiently.

Looking at the source, there is a guard on the function

if (!PyTuple_Check(op) || Py_REFCNT(op) != 1) {
    .... error ....

Sure enough, this is only allowed when there is only 1 reference to the tuple - that reference being the thing that thinks its a good idea to change it. So, a tuple is "mostly immutable" but C code can change it in limited circumstances to avoid the penalty of creating a new tuple.

Answered By - tdelaney