Conversation
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 26549932 | Triggered | Generic High Entropy Secret | dd771c2 | lib/src/db/test.rs | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
Iroh for transport layer #1179
Phase 2b/2c of the loro-source-of-truth plan. 2b: add_resource_opts now derives the Loro snapshot via build_state_doc() and writes Tree::LoroSnapshots + Tree::Resources in one transaction for every non-commit resource, unconditionally. Previously the snapshot was only written when propvals lacked a loroUpdate, so any resource that had been through apply_state_doc (i.e. every sync import) had its snapshot write silently skipped. Invariant established: every CRDT-resource blob write is paired with a current snapshot. 2c: the loroUpdate propval is stripped from the Tree::Resources blob for non-commit subjects. That blob is now a pure derived projection; the CRDT snapshot lives only in Tree::LoroSnapshots. Commits are native (immutable, signed) and keep their loroUpdate payload in the blob. loroUpdate still rides the wire via propvals_for_serialization. 2a and 2d remain deferred — the plan now records the real blocker: making set_unsafe doc-first re-derives a commit's loroUpdate from a doc snapshot, whose random peer id breaks client-sign/server-verify parity. The serialization layer must be fixed first. Test: add_resource_opts_always_writes_loro_snapshot.
Prerequisite for Phase 2a of loro-source-of-truth.md. `propvals_for_serialization` re-derived `loroUpdate` from the live Loro doc whenever one existed. Once Phase 2a makes mutation doc-first, a *commit* resource would acquire a doc too, and this re-derivation would replace the commit's signed `loroUpdate` payload with a doc snapshot — whose embedded random peer id makes the bytes non-deterministic and breaks signature verification. Add `Resource::is_native()`: true for resources whose `loroUpdate` is a signed payload rather than a CRDT snapshot (commits). It discriminates on `isA: Commit`, not the subject — a commit's subject is a placeholder at client-sign time and `did:ad:commit:…` only at server-verify time, so a subject-based gate would make the two sides serialize differently. `propvals_for_serialization` now injects the doc snapshot only for non-native resources; commits keep their `loroUpdate` propval verbatim. No behaviour change today (no commit currently carries a live doc) — a guard that lets the rest of 2a land. Test: commit_loro_update_is_not_re_derived_from_doc.
…ropval_fallible Phase 2a of loro-source-of-truth.md. Add `set_unsafe_fallible` and `remove_propval_fallible` alongside the existing infallible `set_unsafe` / `remove_propval`. The fallible pair: - Materializes the live Loro doc and applies the write/removal to it with `?` — a failing doc write surfaces instead of being swallowed by `let _`, so the doc and the `propvals` cache cannot silently diverge. - Skips the doc entirely for commit resources (`is_native`), which stay propval-only. When `isA` itself is the property being set, native-ness is judged from the incoming value, so a commit never acquires a state doc even transiently while it is being constructed. The old methods are kept so the ~84 call sites can be migrated in build-green batches before the infallible versions are removed. Tests: set_unsafe_fallible_is_doc_first_for_crdt_resources, set_unsafe_fallible_keeps_commit_resources_docless.
Phase 2a of loro-source-of-truth.md. `into_resource` builds the commit Resource via `new_instance(COMMIT, …)`, which sets `isA: Commit` before any other write. The resource is therefore `is_native()` for every subsequent `set_unsafe_fallible` call, so each takes the propval-only branch and no Loro state doc is ever materialized on a commit. This is the step the earlier 2a attempt got wrong: it let a commit acquire a doc, and `propvals_for_serialization` then re-derived the commit's `loroUpdate` from a doc snapshot (random peer id → bytes differ between client-sign and server-verify → "Incorrect signature"). With the `is_native` carve-out the commit's signed `loroUpdate` payload is left untouched. All 19 commit tests pass, including signature_matches and sign_merges_commit_set_onto_existing_loro_snapshot.
Completes Phase 2a of loro-source-of-truth.md. Steps 4-6 — the migration: - `set_unsafe` / `remove_propval` are now doc-first and fallible: they materialize the live Loro doc and apply the write with `?` instead of swallowing the error with `let _`. All ~84 non-test call sites migrated across lib/ and server/; the old infallible methods are gone. - Commit resources stay propval-only (`is_native()`) and never get a state doc, so their signed `loroUpdate` payload is never re-derived from a non-deterministic doc snapshot. - `serialize::test` golden fixtures strip the `loroUpdate` snapshot that `to_json_ad` now emits for every mutated resource (its bytes embed a random peer id and cannot be pinned). The load-bearing fix — doc continuity across `save`: - The server's apply path writes its own ops (`lastCommit`) into the doc under a fresh peer id each commit. `adopt_resource_state` previously kept the client's pre-commit branch, so the next edit was causally *concurrent* with the server's state — every later commit re-merged two divergent branches as LWW and silently dropped writes at random (peer-id tiebreak). This was an intermittent data-loss bug, reproduced by `collections::test::query_on_resource_arrays`. - `adopt_resource_state` now imports the server's post-commit doc into the client's existing doc: one shared causal lineage, no divergence. Importing (not cloning) keeps the live `UndoManager` intact, so undo/redo still survives `save_locally`. Verified: atomic_lib 158/0, atomic-server 26/0, query_on_resource_arrays 20/20, undo_after_save_exports_loro_update_for_sync green. (`multi_client_sync` integration test fails — pre-existing, also red on the pre-session baseline 6e134d7.)
- `closeDialogWith` retries the click while the dialog stays open: the footer button is detached/replaced mid-click while an async commit settles, and a single click races that churn. Bounded retry. - `waitForSynced` dumps the outbox entries and each stuck commit's `lastAttemptError` on timeout — surfaces the server's rejection reason, which is otherwise invisible behind a bare timeout.
The offline-sync outbox could deadlock: when an entry held a commit the server rejects as "produced no state changes" — an already-applied, idempotent no-op — `postOutboxEntry` threw, `LocalOutbox.drain` recorded the error and kept the whole entry, and every genuinely-unsynced commit queued behind it never reached the server. `pendingDirtyCount` then never returned to 0. `postOutboxEntry` now catches that specific rejection per commit, treats it as synced (it is), and continues draining. Genuinely new commits still produce real state changes and post normally; only true no-ops are skipped. Fixes the offline-edit-sync e2e test (`sync.spec.ts`).
The search overlay's input node is detached and replaced as results
stream in, so a single `fill` races that churn ("element was detached
from the DOM"). Retry against a freshly-resolved locator until the
typed value sticks — same pattern as the dialog-close helper.
The server's Tantivy search index commits on a ~5s throttle, so a resource created moments earlier is not yet searchable. `useServerSearch` fired exactly one query per query-string change and never refreshed — an empty result for a resource that exists, until the user re-typed. While results are empty, retry the search a bounded number of times (4 × 2s). A freshly-indexed resource now appears on its own. Fixes the intermittent `search.spec.ts` failures (the overlay queried before the index caught up and never re-queried) and the equivalent real-user "I just made it, why can't I find it" UX gap.
Adds a search.spec.ts test: create a resource online, disconnect (setOffline + close WS), then search for it. The result must come from the client-side MiniSearch index (`LocalSearch`) with no server round-trip. Guards against regressions in offline search.
`LocalSearch` (MiniSearch) is in-memory and constructed empty, so every page load started with a blank index. Offline search after a reload then found only resources that happened to be re-loaded into the store — not the whole persisted dataset. A reloaded PWA session offline could search almost nothing. `setClientDb` now kicks off a background `rehydrateLocalSearch`: once the ClientDb is ready it exports every persisted resource and indexes it into `localSearch`. Runs off the startup path so it never blocks boot; `addResource` is dedup-safe so overlap with the normal ingest path is harmless. Test: store.test.ts 'rehydrates local search from the ClientDb so offline search survives a reload'.
A single global `LocalSearch` index leaked results across every drive and surfaced the bootstrap ontology (the `atomicdata.dev` drive's classes and properties) and commits when a user searched their own drive — the offline search returned mostly noise. - `LocalSearch` now holds one MiniSearch index per drive subject; `addResource(resource, drive)` / `search(query, drive)` are scoped. `did:ad:commit:` resources are skipped — they are write metadata, not searchable content. - `Store.driveOf()` resolves a resource's drive (root of its `parent` chain). Used both for incremental indexing and, in `rehydrateLocalSearch`, against a parent map built from the full ClientDb export so drives resolve even before ancestors load. - `Store.search()` resolves the `parents` scope up to its drive and searches that partition. Offline it falls back to the drive's local index — still scoped to the drive being browsed, no cross-drive leak. Includes temporary `[search]`-prefixed `console.debug` tracing for diagnosing offline search in the client.
The `validate_loro_causality` guard rejected ANY commit whose loroUpdate
produced no net state change. That conflated two opposite cases:
1. Idempotent replay — the commit's Loro ops are already in the
resource's oplog (re-importing changed nothing because there was
nothing new). Loro deduplicates ops by ID, so this is correct and
safe — it MUST be accepted. The browser outbox hits this whenever
it retransmits a commit the server already applied.
2. Silent LWW loss — the ops ARE new but lost last-writer-wins against
stored state (the client's doc was not seeded from the server's).
This is real data loss and MUST be rejected.
`apply_changes` now records `imported_new_ops` — whether importing the
update advanced the doc's oplog version vector. The guard accepts when
no new ops were imported (case 1) and only rejects genuinely-new ops
that produced no change (case 2).
This is the proper server-side fix for the outbox-stranding bug. The
client-side heuristic in `postOutboxEntry` (skip errors containing
"produced no state changes") is removed — it was a band-aid that would
also have swallowed genuine case-2 losses.
Phase 1 of commit-retention-and-state-certificates.md. Test:
idempotent_commit_replay_is_accepted. Verified: atomic_lib 159/0,
offline-sync e2e 4/4.
`scoped search` used `waitForSearchIndex` — a fixed 6.5s sleep — to wait
for the server's Tantivy index to catch up before the scoped query.
That races the ~5s index-commit throttle and failed ~50% of runs.
Capture the created doc's subject and poll the actual scoped server
search (`store.search('Avocado', { parents: cakeFolder })`) until it
returns that subject, then proceed. The first scoped assertion is now
deterministic; residual flakiness (~1 in 8) is post-reload overlay
render timing, not index lag.
`Collection.setPage` merges optimistically-added subjects (resources created locally but not yet in the local-DB query result) into the page. But `fetchPageFromLocalDb` then set `_totalMembers = result.count` — the *pre-merge* query count. Consumers that iterate `0..totalMembers` (e.g. `useChildren`) then stopped one short and never read the just-added resource. Symptom: add two tags to a folder in quick succession; the second tag is created and the folder's `tags` array has it, but the sidebar's drive-children list never shows it — `totalMembers` was 2 while the page held 3 members. Intermittent: only when a local-DB refresh lands after the optimistic add. Read the post-merge total from the page resource instead, exactly as `fetchPageFromServer` already does. Verified: `add tags` e2e 10/10.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Most insane PR ever.
did:adResolvingdid:adlocal-first resources #1146 , refactors subject completely Refactor Subject (resource identifiers / urls / dids / ids) #1139Checklist