Add a variable: search operator over assetsSummary.variableMeasured - #2885
Open
bendichter wants to merge 1 commit into
Open
Add a variable: search operator over assetsSummary.variableMeasured#2885bendichter wants to merge 1 commit into
bendichter wants to merge 1 commit into
Conversation
variable: answers a different question from the operators we already have: approach: and technique: describe how the experiment was run, while variableMeasured describes what is actually in the files, so "which dandisets contain sorted units" or "which have eye tracking" becomes answerable (variable:Units, variable:EyeTracking). In a survey of the archive, variableMeasured is populated on about 77% of dandisets and carries 22 distinct values in a 100-dandiset sample, led by ProcessingModule, ElectrodeGroup, OpticalChannel, ImagingPlane, Units, and ElectricalSeries. The one structural difference from the other summary fields is that variableMeasured is an array of bare strings rather than objects with a name, so its jsonpath selects the elements themselves and _jsonpath_name_match becomes _jsonpath_match. The name variable: (not variable_measured:) follows the existing convention, where technique: comes from measurementTechnique and standard: from dataStandard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
bendichter
force-pushed
the
search-operator-standard
branch
from
August 14, 2026 20:57
35a0e2e to
b9413f5
Compare
bendichter
force-pushed
the
search-operator-variable
branch
from
August 14, 2026 20:57
03009b5 to
844a0be
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #2882 and #2884, stacked on #2884. Adds a
variable:operator overassetsSummary.variableMeasured.I surveyed the archive before picking this field. Of the fields left in
assetsSummary,variableMeasuredis populated on about 77% of dandisets and carries 22 distinct values in a 100-dandiset sample, led byProcessingModule(39),ElectrodeGroup(20),OpticalChannel(17),ImagingPlane(17),Units(15), andElectricalSeries(14), withSpatialSeries,LFP,PupilTracking, andEyeTrackingfurther down. It answers a genuinely different question from the operators we already have:approach:andtechnique:describe how the experiment was run, while this describes what is actually in the files, so "which dandisets contain sorted units" or "which have eye tracking" becomes answerable. Against production metadata,variable:LFPreturns 22 dandisets, all of them genuinely LFP recordings.The one structural difference is that
variableMeasuredis an array of bare strings (NWB neurodata types) rather than objects with aname, so its jsonpath selects the elements themselves and_jsonpath_name_matchis renamed to_jsonpath_match, since it no longer always matches a name.variableMeasuredbeing bare strings is guaranteed bydandischema(Optional[List[str]]), and a 100-dandiset production sample across schema versions 0.4.4 through 0.6.4 found no rows using the olderPropertyValueshape.On naming, I went with
variable:rather thanvariable_measured:to match the existing convention, wheretechnique:comes frommeasurementTechniqueandstandard:fromdataStandard. Underscores show up only in the date operators, where the suffix does real work distinguishing before from after. Happy to rename if you would rather be explicit; it is a one-line change in the parser plus the frontend list.For the record, the fields I looked at and skipped:
numberOfSamplesis populated on 8% of dandisets andnumberOfCellson none of the 100 I sampled, so operators over either would return almost nothing.numberOfBytesandnumberOfFilesare always present, but a size filter already exists in the faceted sidebar.numberOfSubjectsis populated on 93% and is worth having, but it is a threshold operator rather than a substring match and is already written up asnum_subjects:on theadvanced-search-countsbranch (#2827), so it belongs there.🤖 Generated with Claude Code