Skip to content

feat(esm-tools-plus): ESM-Tools-plus STAC browser customisations (Option A, comparison, personal collections, data preview, search)#1

Open
siligam wants to merge 39 commits into
mainfrom
feat/data-preview-component
Open

feat(esm-tools-plus): ESM-Tools-plus STAC browser customisations (Option A, comparison, personal collections, data preview, search)#1
siligam wants to merge 39 commits into
mainfrom
feat/data-preview-component

Conversation

@siligam
Copy link
Copy Markdown

@siligam siligam commented May 18, 2026

Context

This branch contains all ESM-Tools-plus-specific customisations on top of the upstream stac-browser. It is developed as part of the ESM-Tools-plus/simcat initiative, which adds a STAC-based experiment catalog to ESM-Tools.

The browser is configured to talk to the ESM-Tools-plus catalog API (esm_catalog) and is aware of the Option A catalog layout (one collection per experiment, collapsed from per-component collections). All changes here are domain-specific additions; nothing intentionally breaks upstream compatibility.

Reviewer tip: commits are grouped below by feature area. Each area is self-contained and can be read independently. The biggest single changes are PersonalCollections.vue (~850 lines) and SearchFilter.vue (~700 lines added), both net-new features rather than modifications to existing code.


Feature areas

1 · Catalog cards — Option A layout

Files: src/components/Catalog.vue, src/theme/page.scss, src/locales/en/texts.json

  • modelComponents() reads data.components[] (Option A list) first, falls back to legacy data.model string, then parses known component names from the collection ID
  • Component badges moved to their own row below the title so the experiment name gets full width
  • Compare button promoted to a badge-style toggle in the card title row (was a bottom-of-card checkbox that overlapped the timestamp)
  • Add-to-Collection star added alongside the Compare button
  • "Search for Collections" tab renamed to "Search for Experiments"
  • apiCatalogPriority: 'collections' set so Browse uses /collections instead of child links (no duplicate cards)
  • Bug fix: card-body subgrid span increased from 4 → 6 rows (overall card 7 → 9) to prevent text overlap when the quick-facts row (CO₂, run length, etc.) is present

2 · Collection comparison

Files: src/components/CollectionComparison.vue, src/components/CompareButton.vue, src/store/comparison.js

  • Side-by-side comparison table for up to 3 collections, surfacing namelist parameters, temporal extent, components, and other metadata
  • stacId (bare STAC ID like basic-002) is now used as the comparison key — previously getBrowserPath() was used, producing API URLs like /collections//collections/basic-002/items that returned 404
  • Collection metadata fetched directly from GET /collections/{id} instead of looking up by URL in the Vuex database (which always missed)
  • nml:parameters and nml:groups blobs excluded from the comparison table (aggregate dicts that stretched the layout off-screen); NML values are fetched by scanning up to 50 items for one that carries nml: properties, skipping coupler items that carry none
  • Column headers show collection ID (monospace, bold) as primary with title below; colour-coded top border per column

3 · Search & quick filters

Files: src/components/SearchFilter.vue, src/models/cql2/operators/logical.js

  • Quick filters for component, experiment type, output frequency, CO₂/CH₄/N₂O ranges (uses nml:echam:radctl:*vmr field names)
  • Multi-select for experiment type and output frequency
  • Metadata property search with <datalist> autocomplete
  • Quick filter results wired to search submission and collection-level handling
  • Bug fix: field name modelcomponent (renamed in Option A refactor)
  • Bug fix: CO₂/CH₄/N₂O field names corrected (nml:echam:radctl:co2vmr, not nml:radctl:co2vmr)
  • Bug fix: Active filters preview label and AND/OR separator now update reactively when the radio toggle changes (were hardcoded strings/CSS ::before)
  • CQL2 NOT fix: CqlNot.toText() now emits NOT (inner) correctly

4 · Personal collections

Files: src/components/PersonalCollections.vue, src/views/PersonalCollectionsPage.vue, src/components/AddToCollection.vue, src/components/TreeNode.vue

  • Per-user experiment lists stored via the catalog API's personal collections endpoint
  • Treebeard-style tree view for browsing saved collections
  • Add/remove experiments from the card grid via the ★ star button
  • Success/error toast feedback on all operations
  • Anonymous fallback when no auth is configured

5 · Data preview

Files: src/components/DataPreview.vue, src/views/Item.vue, src/views/Catalog.vue

  • Interactive data preview panel embedded in the item and collection views
  • Connects to a configurable vizServer URL (set in config.js)
  • Enabled via vizServer config option (null = disabled)

6 · Dask dashboard

Files: src/components/DaskDashboard.vue, src/views/DaskDashboardPage.vue

  • Embedded Dask cluster management dashboard accessible from the nav bar
  • Shows workers, memory, task graph

7 · HPC asset display

Files: src/components/HrefActions.vue

  • hpc:system asset property used to label the "Copy URL" button (e.g. "Copy URL for albedo")
  • Falls back to path-based detection (/albedo/ → albedo, /work/ → levante)

8 · Infrastructure

  • Docker publish workflow: raw tag fallback for non-semver tags
  • config.js updated for VM deployment (catalogUrl → 10.7.0.13:23006)
  • E2E test suite: new specs for data preview, search filters, homepage; Playwright infrastructure fixes

Test plan

  • Collection grid: experiment names display in full width, component badges on second row
  • Collection grid: no text overlap between description and datetime on cards with CO₂/run-length quick-facts
  • Comparison: select 2–3 collections via Compare button, verify Parameters tab shows NML values
  • Search: Additional filters → switch AND/OR, verify preview label updates immediately without submitting
  • Search: quick filter by component, experiment type, or variable; verify results
  • Personal collections: star an experiment, reload, verify it persists in My Collections tree
  • HPC assets: file:// asset copy button shows system name label

🤖 Generated with Claude Code

siligam and others added 30 commits March 13, 2026 00:42
Adds a dedicated info-variant badge showing the item's collection ID
on every item card. Reads from the top-level `collection` field of the
STAC Item object, which is always populated for items returned by the
ESM Catalog API.

This makes it easy to visually distinguish items across experiments
when viewing cross-collection search results (e.g. after applying
Additional Filters in the Search for Items tab). The badge is rendered
first, before file-format and deprecated badges, using variant="info"
so it is visually distinct from both.

The collection badge is independent of the showKeywordsInItemCards
setting — it is always shown when the item has a collection field.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
STAC Browser previously serialised CQL2 filters as cql2-text when
attaching them to GET requests (e.g. GET /collections/{id}/items).
The ESM Catalog API only implements a cql2-json parser, so the text
filter was silently ignored and the Submit button appeared to have
no effect.

Switch value.toText() → value.toJSON() in Utils.addFiltersToLink so
GET requests carry ?filter-lang=cql2-json&filter={...} — consistent
with the POST /search path and supported by the API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous commit changed value.toText() to value.toJSON() expecting
the API to parse JSON-encoded filters on GET requests. The API now
supports cql2-text natively via _parse_cql2_text(), so the stac-browser
change is no longer needed and is reverted to keep standard behaviour.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CqlNot inherited toText() from CqlOperator which uses Array.join() with
the operator as separator. For a single-element args array, join() returns
just the element text with no separator — silently dropping the NOT.

Override toText() in CqlNot to produce the correct CQL2-text syntax:
  NOT (inner expression)

Without this fix, the "Negate filter" checkbox had no effect on GET
requests because the NOT was lost before the filter reached the API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduces a reusable component that fetches metadata from a visualization
server and displays interactive previews of geospatial data. Features include
variable selection, time dimension slider, colormap options, and error handling.
- Add DataPreview as a collapsible section in the Item view
- Lazy-load preview content only when section is expanded
- Add vizServer configuration option for visualization server URL
- Update config schema to include vizServer property
…andling

- Add CQL operator imports for quick filter conversion
- Integrate buildQuickFilters() into onSubmit() - was defined but never called
- Convert quick filter objects to proper CQL operators (>=, <=, like, in)
- AND-combine quick filters with manual filters
- DataPreview: detect collections vs items and pass collection_id
- Fix config loading by adding CONFIG option to yargs and correcting case sensitivity
- Create test-specific config (config.test.js) without hardcoded albedo URLs
- Add CQL2 conformances to test fixtures for Quick Filters visibility
- Register BFormCheckboxGroup in vite auto-import
- Add quick filter fields to getDefaults() for reset functionality
- Fix test selectors for grouped queryables, data preview, and homepage
- Remove spatial extent UI and tests (all climate models are global)
- Add new e2e tests for search filters and data preview
- Add collection comparison, glossary tooltips, and namelist tree components
- Frontend now uses RESTful URL pattern for Panel interactive preview
- Update e2e test mocks to match new URL scheme
- Generate /preview/collection/{id}/panel URL for collections
- Remove isCollection guard from interactiveUrl
- Add taller iframe for collection previews
- PersonalCollections.vue: CRUD collections, drag-and-drop tree, labels, sharing
- TreeNode.vue: recursive tree node with expand/collapse, hover actions
… tab

- Fetch item properties (nml: keys) from STAC API for parameter diff
- Add Visual Compare tab with iframe to viz server comparison endpoint
- Tab navigation between Parameters and Visual Compare views
loadPaleoPresets() returned early without setting fallback presets when
baseUrl was empty (common in collection search context).
Presets were empty until async loadPaleoPresets completed. Initialize
with fallback values directly in data() so they show on both tabs.
Prevents baseUrl from being null when $store.state.user is undefined.
- Add /collections/personal route with PersonalCollectionsPage view
- Add My Collections bookmark button in header nav
- Add AddToCollection button on Item and Collection pages
- Set default user in store for NoAuth mode
- DaskDashboard.vue: cluster list, create/scale/delete, auto-refresh
- /compute route with CPU icon in header nav
- Supports Local, SLURM, Gateway, and Existing cluster types
Replace single-value <b-form-select> dropdowns for Experiment Type and
Output Frequency with <b-form-checkbox-group buttons> matching the existing
Model Components style. Multiple values can now be selected simultaneously
(e.g. "3-hourly AND 6-hourly") and are submitted as a CQL2 IN filter.

Also fix namelist dropdown chip clicks: Object.assign({}, q, {shortId})
was stripping the Queryable prototype chain, causing getOperators() to
throw TypeError when a chip was clicked. Fix by using
Object.create(Object.getPrototypeOf(q)) as the base object so all
prototype methods remain callable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add a native HTML <datalist> element to the metadata search input in
MetadataGroups.vue. The datalist is populated with all human-readable
property labels (e.g. "CO2 Volume Mixing Ratio") sorted alphabetically,
providing browser-native autocomplete without any extra dependencies.

A unique datalist ID per component instance (using $.uid) ensures
multiple MetadataGroups on the same page (item + asset) do not conflict.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Cql stores its root operator as this.filters (plural), but onSubmit()
was reading filters.filter (singular) → undefined → crash with
"can't access property toJSON, arg is undefined" when any Additional
Filter was active alongside Quick Filters.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Every CRUD operation in PersonalCollections.vue was silent — errors
went to console.error only, successes had no confirmation at all.

Changes:
- Add a BAlert notification banner below the card title that
  auto-dismisses after 4 s (success=green, warning=orange, danger=red)
- Add per-modal inline BAlert for create/edit/share/add-items errors;
  the modal stays open (bvEvent.preventDefault()) so the user can
  correct the input without re-opening it
- Improve apiRequest() error parsing: extracts FastAPI's detail field
  from JSON error responses instead of showing raw HTTP status text
- Add BAlert to component imports/registrations

Operations covered and their feedback:
  New Collection  → success toast / inline modal error
  New Folder      → success toast / inline modal error
  Edit (rename/update) → success toast / inline modal error
  Delete          → warning toast / danger toast on failure
  Share           → success toast / inline modal error
  Revoke share    → warning toast / inline modal error
  Add items       → success toast with item count / inline modal error
  Create label    → success toast / inline labels modal error
  Delete label    → warning toast / inline labels modal error
  Move (drag/drop) → danger toast on failure

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TreeResponse returns {roots:[...]} but both PersonalCollections.vue
and AddToCollection.vue were reading data?.nodes which is always
undefined, making the tree perpetually empty in the browser.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- TreeNode: emit view-node when a collection row is clicked; bubble
  the event through nested TreeNode instances
- PersonalCollections: handle view-node by opening a detail modal
  that fetches and lists the collection's item IDs
- Add a drag-drop hint below the tree so users know folders can be
  used to organise collections

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Both components must use the same default username ('anonymous') so
that add-items requests go to the correct /users/{username}/... path
and pass the ownership check.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- View modal: each item ID is now a router-link to /collections/{id}
  so users can navigate directly to the catalog entry; modal closes on
  click
- Share modal: add an explanatory note clarifying that no notification
  is sent, show the direct API URL the recipient can use to access the
  shared collection, and note that a browser-side "Shared with me"
  view is planned

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- config.js: point catalogUrl to localhost:23006 (SSH-tunnelled port)
  rather than the direct server hostname, which caused CORS failures
- config.js + vite.config.js: add allowedHosts: true so the dev server
  accepts requests from any hostname (needed for SSH tunnel access)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…atalog cards

Replace the bottom-of-card Compare checkbox (which overlapped the timestamp)
with a badge-style toggle button in the card title row alongside the model
component badges. Also add a compact Add-to-Collection star button (★) in
the same row, always visible on collection cards.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ments

Items on the landing page were stored with /collections/{id} URLs because
AddToCollection received only data.id. Now it receives data.getBrowserPath()
which includes the correct prefix (e.g. /experiments/basic-001).

PersonalCollections view modal updated to use stored paths directly when
they start with '/', with fallback for legacy plain-ID entries.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…sed collections

- modelComponents() reads data.components list first (Option A), falls
  back to data.model string, then parses known component names from ID
- Rename "Search for Collections" tab to "Search for Experiments"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
siligam and others added 9 commits April 25, 2026 18:31
…ts tab

- Move component badges to their own row below the card title so the
  experiment name gets full horizontal width (was squished to ~0 px
  by flex-shrink-0 badge row)
- Set apiCatalogPriority='collections' so Browse uses /collections
  endpoint instead of child links (no duplicate cards)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Quick filter was querying field 'model' but items store their model
component in properties.component (renamed during Option A refactor).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Browser was using 'nml:radctl:*vmr' but items store these under
'nml:echam:radctl:*vmr' (echam component prefix required).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Use hpc:system from the asset (if present) to label the "Copy URL"
button for file:// assets, so users see e.g. "Copy URL for albedo"
instead of the generic "Copy URL for file system (local)".

Falls back to path-based detection (/albedo/ → albedo, /work/ →
levante) for existing catalog items that predate the asset-level field.
Scales to any HPC system without hardcoding names in the UI.

Also update config.js to target the VM deployment
(catalogUrl → 10.7.0.13:23006, vizServer → null).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs caused the comparison view to silently produce an empty
Parameters tab:

1. Catalog.vue used collectionId (getBrowserPath() → '/collections/
   basic-002') as the key passed to the comparison Vuex store.
   CollectionComparison.vue then built API URLs like
   /collections//collections/basic-002/items, which returned 404,
   so no NML parameters were ever loaded.

   Fix: add stacId computed (= data.id = 'basic-002') and use it for
   canCompare, isSelectedForComparison, and toggleComparison.
   collectionId (browser path) is left unchanged for AddToCollection.

2. loadCollectionData tried rootState.database[id] to get collection
   metadata, but the Vuex database is keyed by full URL, not bare ID,
   so the lookup always missed.

   Fix: fetch collection metadata directly from the STAC API
   (GET /collections/{id}) to get the title and other fields.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The collection title (long descriptive text) was the primary header,
with the short ID in small secondary text — making it hard to tell
columns apart. Swapped them: collection ID (e.g. basic-001) is now
the bold primary header in monospace, with the title truncated below
it. Also adds a colour-coded top border per column (blue/orange/green)
so columns are visually distinct at a glance.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- allParameters getter now excludes nml:parameters and nml:groups, which
  are collection-level aggregate dicts rather than individual comparable
  values; their huge JSON blobs were stretching the table and hiding the
  second collection column off-screen
- fetchItemParameters now fetches up to 50 items and picks the first one
  with nml: properties, avoiding OASIS/coupler items (returned first by
  default) that carry no namelist keys and left columns blank

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The card body was allocated span 4 rows in the subgrid, but after the
nml:parameters enrichment added the quick-facts row (CO2, run length,
etc.), cards with all rows visible (title + model-badges + intro +
quick-facts + datetime) had 5 children with only 4 tracks. The 5th
child (datetime) had no allocated row and overlapped content above it.

Increase card-body from span 4 → span 6 and overall card span from
7 → 9 to accommodate all currently-rendered conditional rows without
overflow.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… AND/OR toggle

The label "Active filters (AND):" was hardcoded, and the "AND" prefix
between filter lines was a static CSS ::before pseudo-element. Neither
updated when the user switched to "Match any filters (or)".

Replace both with reactive template bindings on filtersAndOr so the
label and operator keyword update immediately on radio change.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@siligam siligam changed the title feat: ESM-Tools STAC browser customisations (Option A, comparison, personal collections, data preview, search) feat(esm-tools-plus): ESM-Tools-plus STAC browser customisations (Option A, comparison, personal collections, data preview, search) May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants