I'm repeatedly calling a python module I wrote in C++ using the Python C API. My python program repeatedly calls my module's pyParse function, which does a bunch of stuff and retus a PyTuple that contains more PyTuple objects as elements. Every retued object ends up with a PyObject->refcnt of 1, so you know the object should be deleted when it goes out of scope in python. I repeatedly call this module with something like the following python code:
import my_module #this is my c++ module.
import os
path = 'C:/data/'
for filename in os.listdir(path):
data = my_module.pyParse(path+filename)
The longer this loop runs, the more the memory usage blows up. Every iteration produces about 2kb of tuples (which should be destroyed at end of every iteration). Yet when I take "heap snapshots" and compare an early one to another many more iterations later, you can see the allocation of memory called by PyTuple_New and other python objects keeps growing.
Yet because every retued object has 1 as a reference count, I would expect that it would be destroyed after going out of scope in python. Is there something I am missing? Or does anyone know how to possibly debug this and get a better handle on what's going on? I'm desperate!