Units waveforms dimension order is mislabeled in add_unit docstring and TestUnitsIO (actual order is (num_spikes, num_electrodes, num_samples))

[ehennestad]

Summary

For the doubly-indexed Units.waveforms column, add_unit interprets a 3-D per-unit input as (num_spikes, num_electrodes, num_samples): dim 0 = spike events (→ waveforms_index_index), dim 1 = electrodes / waveforms per spike (→ waveforms_index), dim 2 = samples. The stored dataset is 2-D [num_waveforms, num_samples].

Two places document/label this backwards, and one test fixture is internally inconsistent. Because the round trip is faithful and nothing validates the electrode dimension, following the documented order silently produces a transposed, self-inconsistent file with no error.

1. add_unit docstring (src/pynwb/misc.py)

The waveforms parameter doc says:

Individual waveforms for each spike. If the dataset is three-dimensional, the third dimension shows the response from different electrodes …

But for a 3-D add_unit input the last dimension is samples, not electrodes:

import numpy as np
from pynwb.misc import Units

u = Units(name='units')
u.add_unit(spike_times=[0.], waveforms=np.arange(1*5*7).reshape(1, 5, 7))  # (spikes, D1, D2)
wf = u['waveforms']
print(np.asarray(wf.target.target.data).shape)  # (5, 7)  -> (num_waveforms, num_samples)
print(list(np.asarray(wf.target.data)))          # [5]     -> electrodes per spike
# => D1 (5) is electrodes, D2 (7) is samples

2. tests/integration/hdf5/test_misc.py::TestUnitsIO.setUpContainer

The fixture labels the outer dim as electrodes and the middle dim as spikes (# elec 1, # elec 2, # spike 1 …), i.e. (electrodes, spikes, samples) — the reverse of what add_unit does. Running the exact fixture:

waveforms data shape  : (18, 3)
waveforms_index       : [3, 6, 10, 14, 18]   # electrodes per spike event
waveforms_index_index : [2, 5]               # spike events per unit

So unit 1's (2,3,3) becomes 2 spike events × 3 electrodes, and unit 2's (3,4,3) becomes 3 spike events × 4 electrodes — dim 0 is spikes, dim 1 is electrodes. The comments are swapped.

The same fixture is also internally inconsistent with spike_times: both units declare 3 spike_times, but unit 1's waveforms imply 2 spike events and unit 2's middle dim is 4. TestUnitsIO only asserts spike_times/obs_intervals (never the waveforms), so this is never caught.

3. Concrete failure: following the docstring silently transposes the data

2 electrodes, 10 samples, 3 spikes, and 2 electrode references. Correct vs docstring-order:

# CORRECT: (num_spikes=3, num_electrodes=2, num_samples=10), electrodes=[0, 1]
#   waveforms.data  : (6, 10)
#   waveforms_index : [2, 4, 6]      # 2 electrodes/spike -> matches the 2 electrode refs

# DOCSTRING ORDER: (num_spikes=3, num_samples=10, num_electrodes=2), electrodes=[0, 1]
#   add_unit does NOT raise
#   waveforms.data  : (30, 2)        # each "waveform" is now length 2, not 10
#   waveforms_index : [10, 20, 30]   # claims 10 electrodes/spike, but electrodes column has 2

No exception is raised; the electrode and sample axes are transposed on disk, and the waveforms' electrode count (10) is inconsistent with the electrodes column (2) — which the schema requires to match.

Suggested fixes

• Correct the add_unit waveforms docstring (and the shared waveforms_desc) to state the input order (num_spikes, num_electrodes, num_samples) for the doubly-indexed column.
• Fix the TestUnitsIO fixture comments, make the waveforms consistent with spike_times, and consider asserting the resulting waveforms_index / waveforms_index_index so the structure is actually validated.
• Optionally, warn when the waveforms' per-spike electrode count is inconsistent with the length of a unit's electrodes entry. I also ran nwbinspector on a file created with the wrong electrode/sample order following the docs, and there was no check for electrode count matching waveform shape.

Environment

• pynwb 4.0.0

🤖 Generated with Claude Code