Skip to content

Add a variable: search operator over assetsSummary.variableMeasured - #2885

Open
bendichter wants to merge 1 commit into
search-operator-standardfrom
search-operator-variable
Open

Add a variable: search operator over assetsSummary.variableMeasured#2885
bendichter wants to merge 1 commit into
search-operator-standardfrom
search-operator-variable

Conversation

@bendichter

Copy link
Copy Markdown
Member

Follow-up to #2882 and #2884, stacked on #2884. Adds a variable: operator over assetsSummary.variableMeasured.

I surveyed the archive before picking this field. Of the fields left in assetsSummary, variableMeasured is populated on about 77% of dandisets and carries 22 distinct values in a 100-dandiset sample, led by ProcessingModule (39), ElectrodeGroup (20), OpticalChannel (17), ImagingPlane (17), Units (15), and ElectricalSeries (14), with SpatialSeries, LFP, PupilTracking, and EyeTracking further down. It answers a genuinely different question from the operators we already have: approach: and technique: describe how the experiment was run, while this describes what is actually in the files, so "which dandisets contain sorted units" or "which have eye tracking" becomes answerable. Against production metadata, variable:LFP returns 22 dandisets, all of them genuinely LFP recordings.

The one structural difference is that variableMeasured is an array of bare strings (NWB neurodata types) rather than objects with a name, so its jsonpath selects the elements themselves and _jsonpath_name_match is renamed to _jsonpath_match, since it no longer always matches a name. variableMeasured being bare strings is guaranteed by dandischema (Optional[List[str]]), and a 100-dandiset production sample across schema versions 0.4.4 through 0.6.4 found no rows using the older PropertyValue shape.

On naming, I went with variable: rather than variable_measured: to match the existing convention, where technique: comes from measurementTechnique and standard: from dataStandard. Underscores show up only in the date operators, where the suffix does real work distinguishing before from after. Happy to rename if you would rather be explicit; it is a one-line change in the parser plus the frontend list.

For the record, the fields I looked at and skipped: numberOfSamples is populated on 8% of dandisets and numberOfCells on none of the 100 I sampled, so operators over either would return almost nothing. numberOfBytes and numberOfFiles are always present, but a size filter already exists in the faceted sidebar. numberOfSubjects is populated on 93% and is worth having, but it is a threshold operator rather than a substring match and is already written up as num_subjects: on the advanced-search-counts branch (#2827), so it belongs there.

🤖 Generated with Claude Code

variable: answers a different question from the operators we already
have: approach: and technique: describe how the experiment was run,
while variableMeasured describes what is actually in the files, so
"which dandisets contain sorted units" or "which have eye tracking"
becomes answerable (variable:Units, variable:EyeTracking).

In a survey of the archive, variableMeasured is populated on about 77%
of dandisets and carries 22 distinct values in a 100-dandiset sample,
led by ProcessingModule, ElectrodeGroup, OpticalChannel, ImagingPlane,
Units, and ElectricalSeries.

The one structural difference from the other summary fields is that
variableMeasured is an array of bare strings rather than objects with a
name, so its jsonpath selects the elements themselves and
_jsonpath_name_match becomes _jsonpath_match. The name variable: (not
variable_measured:) follows the existing convention, where technique:
comes from measurementTechnique and standard: from dataStandard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bendichter
bendichter force-pushed the search-operator-standard branch from 35a0e2e to b9413f5 Compare August 14, 2026 20:57
@bendichter
bendichter force-pushed the search-operator-variable branch from 03009b5 to 844a0be Compare August 14, 2026 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant