[Bug]: NWBFile.objects is cached on first access and never invalidated

[pauladkisson]

Adding new objects to an NWB file causes nwb_file.objects to become stale.

Details (AI-generated)

What happened?

NWBFile.objects is populated once, lazily, and nothing ever invalidates it
(src/pynwb/file.py:587-591):

@property
def objects(self):
    if self.__obj is None:
        self.all_children()
    return self.__obj

all_children() rebuilds self.__obj and returns a fresh walk; objects returns whatever the
first walk happened to see. So the first read of .objects decides its contents for the life of the
NWBFile — containers added afterwards are absent from it, and containers removed afterwards are
still in it.

I expected .objects to reflect the file as it currently stands, the way all_children() does. At
minimum I did not expect merely reading the property to change what a later read returns.

Where I hit it

I'm adding NWB export to GuPPy (LernerLab/GuPPy#357),
which reads an existing NWB file, adds the fiber-photometry derived data to it via a NeuroConv
interface, and writes the result:

nwbfile = read_nwb(path=source_nwbfile_path)
interface.add_to_nwbfile(nwbfile=nwbfile, metadata=interface.get_metadata())
configure_and_write_nwbfile(nwbfile=nwbfile, nwbfile_path=nwbfile_path, backend="hdf5")

The output was valid and the data was correct, but every dataset the interface added — the derived
traces, the transients tables, the events table — came out with compression=None and
chunks=None, while the datasets that came from the source file were gzip.

The cause is the cache. Early in add_to_nwbfile, the interface resolves GuPPy's store ids against
the file's existing FiberPhotometryResponseSeries by reading nwbfile.objects — a read-only
inspection, with no intent to mutate anything. That froze the cache at 42 objects. The finished file
had 85. NeuroConv's backend configuration then walks .objects to decide which datasets to wrap in
an H5DataIO, saw the 42-object snapshot, and wrote the other 43 containers with backend defaults.

Nothing raised, so this is only visible by inspecting the written file's filters. On a real
multi-hour session those uncompressed traces are among the largest objects in the file. The
NeuroConv-side report is catalystneuro/neuroconv#1909;
we'll switch our call sites to all_children(), but the trap is easy to fall into from anywhere:
whether an inspection of an NWBFile corrupts a later inspection depends on which of the two
accessors each unrelated piece of code happened to pick.

Steps to Reproduce

"""`NWBFile.objects` is computed once and never invalidated."""

import numpy as np
from pynwb.testing.mock.base import mock_TimeSeries
from pynwb.testing.mock.file import mock_NWBFile

nwbfile = mock_NWBFile()

nwbfile.objects  # anything that inspects the file freezes the cache here

nwbfile.add_acquisition(mock_TimeSeries(name="AddedAfter", data=np.arange(10)))
print("in objects:      ", any(o.name == "AddedAfter" for o in nwbfile.objects.values()))
print("in all_children: ", any(c.name == "AddedAfter" for c in nwbfile.all_children()))
print("in objects after all_children:", any(o.name == "AddedAfter" for o in nwbfile.objects.values()))

# Nested containers behave the same way.
module = nwbfile.create_processing_module(name="behavior", description="")
nwbfile.objects
module.add(mock_TimeSeries(name="Nested", data=np.arange(10)))
print("nested in objects:", any(o.name == "Nested" for o in nwbfile.objects.values()))

# And removals never leave the cache.
timeseries = mock_TimeSeries(name="Removed", data=np.arange(10))
nwbfile.add_acquisition(timeseries)
nwbfile.all_children()
del nwbfile.acquisition["Removed"]
timeseries.reset_parent()
print("removed still in objects:", any(o.name == "Removed" for o in nwbfile.objects.values()))

in objects:       False
in all_children:  True
in objects after all_children: True
nested in objects: False
removed still in objects: True

Note the third line: all_children() repopulates __obj as a side effect, so an unrelated call
elsewhere can also silently un-stale the cache. Whether .objects is correct depends on the
interleaving of reads and writes across otherwise independent code.

Environment

• pynwb 4.0.0 (objects is unchanged on dev as of this writing), hdmf 6.1.0
• Python 3.13.13, conda, macOS

🤖 Generated with Claude Code