Skip to content

CS-10009 PR 2: migrate realm-endpoints/ tests to explicit fixture#4790

Merged
lukemelia merged 2 commits into
mainfrom
cs-10009-migrate-realm-endpoints
May 13, 2026
Merged

CS-10009 PR 2: migrate realm-endpoints/ tests to explicit fixture#4790
lukemelia merged 2 commits into
mainfrom
cs-10009-migrate-realm-endpoints

Conversation

@lukemelia
Copy link
Copy Markdown
Contributor

Second slice of CS-10009. Stack: PR 1 (#4772) landed the fixture plumbing + 2 representative migrations; this PR migrates the remaining 9 callers under packages/realm-server/tests/realm-endpoints/.

Per-file decisions

File Fixture Rationale
cancel-indexing-job-test.ts blank Job-queue ops only — no card content
dependencies-test.ts blank Writes its own dependencies-card.gts
directory-test.ts realistic Asserts on dir/bar.txt, dir/foo.txt, dir/subdir/ from tests/cards/dir/
info-test.ts (4 modules) blank _info returns testRealmInfo-shape; doesn't read cards
lint-test.ts blank POST source bodies, no realm reads
mtimes-test.ts blank mtimes(testRealmPath) round-trip; works on any fixture
permissions-test.ts blank (was fileSystem: {}) _permissions endpoints
search-test.ts (4 modules) simple _search queries for Person/Mango
user-test.ts (2 modules) blank _user endpoints don't read card content

Skipped (per Linear plan: callers that pass their own fileSystem inline are left alone):

  • invalidate-urls-test.ts — already migrated to simple in PR 1.
  • markdown-test.ts, publishability-test.ts, reindex-test.ts — each test slot defines its content inline.

Drive-by: dead copySync(cards) blocks

Four files (directory-test, info-test, permissions-test, user-test) had a hooks.beforeEach(() => { dir = dirSync(); copySync(cards, dir.name); }) block whose result was never read — the local dir was immediately overwritten by args.dir from onRealmSetup, so the kitchen-sink copy was dropped on the floor. Removed in those four files (alongside the now-unused dirSync / copySync / join imports).

For directory-test, the realm's actual content STILL needs to be the kitchen sink — that's now made explicit via fixture: 'realistic' rather than relying on the implicit default.

Test plan

  • CI runs every migrated test against the named fixture and they pass.
  • CI runs every other realm-server test file unchanged.

🤖 Generated with Claude Code

One PR per directory per the CS-10009 plan; this one covers
packages/realm-server/tests/realm-endpoints/.

Per-file migration:
- cancel-indexing-job-test: fixture: 'blank' (job-queue ops only).
- dependencies-test: fixture: 'blank' (writes its own file).
- directory-test: fixture: 'realistic' (asserts on tests/cards/dir/
  contents); the redundant `dirSync + copySync(cards)` hook ahead of
  setupPermissionedRealmCached was dead code (the local `dir` was
  immediately overwritten by args.dir in onRealmSetup) — removed.
- info-test: fixture: 'blank' on all 4 modules; same dead-copySync
  removal as directory-test.
- lint-test: fixture: 'blank' (POSTs source bodies, no realm reads).
- mtimes-test: fixture: 'blank' (compares mtimes(testRealmPath)
  round-trip; works on any fixture).
- permissions-test: fixture: 'blank' (was already fileSystem: {});
  same dead-copySync removal.
- search-test: fixture: 'simple' on all 4 implicit-default modules
  (they all query Person/Mango which the simple fixture has). The
  three modules that pass their own `fileSystem` inline are
  untouched.
- user-test: fixture: 'blank' on both modules (GET and POST _user
  endpoints don't read realm card content); same dead-copySync
  removal.

Skipped (per Linear plan: callers that pass their own `fileSystem`
inline are left alone): invalidate-urls-test (already simple from
PR 1), markdown-test, publishability-test, reindex-test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lukemelia lukemelia marked this pull request as draft May 12, 2026 18:04
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 12, 2026

Host Test Results

    1 files      1 suites   1h 47m 24s ⏱️
2 658 tests 2 643 ✅ 15 💤 0 ❌
2 677 runs  2 662 ✅ 15 💤 0 ❌

Results for commit 20868f4.

Realm Server Test Results

    1 files  ±    0      1 suites  ±0   10m 51s ⏱️ -10s
1 334 tests +    6  1 334 ✅ +   12  0 💤 ±0  0 ❌  -  6 
1 413 runs   - 1 401  1 413 ✅  - 1 385  0 💤 ±0  0 ❌  - 16 

Results for commit 20868f4. ± Comparison against earlier commit 925e68f.

@lukemelia
Copy link
Copy Markdown
Contributor Author

Per-file benchmarks

Single run each via packages/realm-server/scripts/measure-test-file.sh <file> 1 on local hardware (Apple Silicon, host/synapse/realm dev stack running), comparing main (HEAD e32387deff) vs this branch.

File New fixture Before (s) After (s) Δ Status before → after
cancel-indexing-job-test blank 454.73 418.31 -36.4 (-8.0%) FAIL → FAIL †
dependencies-test blank 207.28 166.06 -41.2 (-19.9%) FAIL → ✅ OK
directory-test realistic (kept) 328.60 331.94 +3.3 (+1.0%) FAIL → FAIL †
info-test blank 505.06 419.69 -85.4 (-16.9%) FAIL → FAIL †
lint-test blank 702.65 662.25 -40.4 (-5.8%) FAIL → ✅ OK
mtimes-test blank 166.13 122.67 -43.5 (-26.2%) FAIL → ✅ OK
permissions-test blank (was fileSystem: {}) 385.86 383.35 -2.5 (-0.6%) OK → OK
search-test simple 1398.83 1375.58 -23.3 (-1.7%) FAIL → FAIL †
user-test blank 397.14 354.74 -42.4 (-10.7%) FAIL → ✅ OK
Total 4546.28 4234.59 -311.7 (-6.9%)

† All FAIL→FAIL cases are local-env-specific: realistic-fixture template builds occasionally exceed qunit's 60s testTimeout on local hardware, the offending realm holds port 4444, and subsequent realms cascade into EADDRINUSE. The original code had the same property (it's not introduced by this PR — the same files FAIL on main too). CI hardware doesn't exhibit it.

Reliability gains

Four files flip from local-FAIL to local-OK: dependencies, lint, mtimes, user. That's the more interesting signal than the wall-time deltas — moving these off the heavy realistic fixture means their template builds complete inside the qunit 60s timeout window even on local hardware. The migration removes a real flakiness floor.

Expected non-movers

  • directory-test is kept on realistic (it asserts on dir/bar.txt and dir/subdir/ from tests/cards/dir/), so a flat wall time is the right answer.
  • permissions-test was already fileSystem: {} (semantically the same realm shape as blank), so the swap is a no-op for content.
  • search-test is simple but only saves 1.7% — the test runs many search queries against the indexed data, so test-work dominates over realm boot here.

Caveats

  • Single runs only (each file is multi-minute locally); medians would tighten variance but not change the direction.
  • Local hardware shows much wider variance than CI; treat these as order-of-magnitude / sign-of-delta signal, not as production-ready microbenchmarks.

🤖 Generated with Claude Code

CI surfaced two miscategorizations from the initial survey:

info-test depends on `testRealmInfo.name === 'Test Realm'`, which
comes from the `realm.json` RealmConfig card instance in
tests/cards/, NOT from `.realm.json`. Neither `blank` nor `simple`
ships that instance, so the realm boots with the "Unnamed Workspace"
fallback and every 200-with-permission assertion fails the deepEqual.
All 4 info-test modules now use `realistic`.

search-test's `QUERY request (public realm) > public readable realm`
module exercises pagination (expects 3 Person results), file-meta
filtering across multiple FileDefs, and sparse-fieldset
backward-compat — all of which need the kitchen-sink content.
Reverted just that one sub-module to `realistic`; the other three
search-test modules only ever query for Mango and stay on `simple`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lukemelia lukemelia marked this pull request as ready for review May 12, 2026 21:19
@lukemelia lukemelia requested review from a team, backspace, Copilot and habdelra and removed request for habdelra May 12, 2026 21:19
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Migrates the remaining packages/realm-server/tests/realm-endpoints/* tests to use the explicit named realm fixtures (blank / simple / realistic) via setupPermissionedRealmCached, removing previously dead temp-dir + copySync(cards) setup blocks.

Changes:

  • Updated realm-endpoints tests to declare an explicit fixture rather than relying on the implicit default or inline fileSystem: {}.
  • Removed unused tmp-dir copying logic (dirSync/copySync/join and dir plumbing) where the copied directory wasn’t actually used.
  • Adjusted fixture selection per-module in search-test (mix of realistic and simple) to match test expectations.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file
File Description
packages/realm-server/tests/realm-endpoints/cancel-indexing-job-test.ts Switches realm setup to fixture: 'blank' for job-queue endpoint tests.
packages/realm-server/tests/realm-endpoints/dependencies-test.ts Uses fixture: 'blank' and continues writing its own test card file.
packages/realm-server/tests/realm-endpoints/directory-test.ts Makes kitchen-sink dependency explicit via fixture: 'realistic'; removes unused temp copying.
packages/realm-server/tests/realm-endpoints/info-test.ts Adds explicit fixture selection (currently realistic) and removes unused temp copying.
packages/realm-server/tests/realm-endpoints/lint-test.ts Uses fixture: 'blank' for POST-only lint endpoint tests.
packages/realm-server/tests/realm-endpoints/mtimes-test.ts Uses fixture: 'blank' for mtimes endpoint tests.
packages/realm-server/tests/realm-endpoints/permissions-test.ts Replaces fileSystem: {} with fixture: 'blank'; removes unused temp copying.
packages/realm-server/tests/realm-endpoints/search-test.ts Splits fixtures: one module uses realistic, other modules use simple; documents rationale.
packages/realm-server/tests/realm-endpoints/user-test.ts Uses fixture: 'blank'; removes unused temp copying and dir plumbing.
Comments suppressed due to low confidence (2)

packages/realm-server/tests/realm-endpoints/info-test.ts:46

  • setupPermissionedRealmCached is configured with fixture: 'realistic', but this test file only exercises the _info endpoint and does not appear to depend on any realm card/file content. Using the kitchen-sink fixture adds unnecessary setup cost; consider switching this to fixture: 'blank' (or, if realistic is actually required, reconcile this with the PR description which states info-test uses blank).
    module('public readable realm', function (hooks) {
      setupPermissionedRealmCached(hooks, {
        fixture: 'realistic',
        permissions: {
          '*': ['read'],
        },
        realmURL,
        onRealmSetup,
      });

packages/realm-server/tests/realm-endpoints/search-test.ts:73

  • This module is using fixture: 'realistic', but the PR description's per-file decisions state that search-test uses the simple fixture for its modules. Please reconcile the PR description with the code (or adjust the fixture selection). Given the pagination/file-meta assertions in this module, realistic may be intentional, in which case the PR description table should be updated.
      module('public readable realm', function (hooks) {
        // Uses `realistic` because tests in this module depend on
        // multiple Person instances (pagination), FileDef instances
        // beyond person.gts (file-meta queries), and the kitchen-sink
        // content variety (sparse-fieldsets backward-compat). Sibling
        // modules below only ever query for Mango and stay on `simple`.
        setupPermissionedRealmCached(hooks, {
          fixture: 'realistic',
          permissions: {
            '*': ['read'],
          },
          realmURL: testRealmURLFor('test/'),
          onRealmSetup,
        });

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@lukemelia lukemelia merged commit 1794cef into main May 13, 2026
99 checks passed
lukemelia added a commit that referenced this pull request May 13, 2026
* `ServerEndpointsTestOptions.fixture` now imports `RealmFixtureName`
  from the shared `tests/helpers` rather than re-declaring the literal
  union — fixture-name additions land in one place and can't drift.
* `user-and-catalog-test.ts`: drop the stale "Same pattern as info-test
  in PR 2 (#4790)" reference. The constraint above it (realistic-only)
  already states the why; PR-number links rot post-merge.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants