fix(assets): link shared assets atomically instead of check-then-act - #7039
Conversation
`asset(shared=True)` created the symlink into `assets/external/` with a sequence that was racy at every step: the `exists()`/`is_symlink()` guard, the `unlink()` in the `FileExistsError` handler, and the retry `symlink_to()` after it. Concurrent compiles into one working directory — pytest-xdist workers, parallel builds, containers on a shared bind mount — lost those races and aborted the compile with `FileNotFoundError` from the `unlink()`, or with an unhandled `FileExistsError` from the retry. Build the link under a unique temporary name in the destination directory and `os.replace()` it into place, which atomically overwrites whatever the loser of the race left behind and needs no retry. Errors still propagate, so `asset()` cannot return a path with no symlink behind it. The old guard is dropped: `exists()` follows the link, so a destination pointing at some other existing file made it skip rather than repoint. The replacement fast path compares `readlink()` against the intended target, keeping the "no needless re-creation for file watchers" property while actually converging on the right target. The `FileExistsError` comment attributed this to docker bind mounts; that is one cause, but the general one is concurrency, with no container involved.
Merging this PR will not alter performance
Comparing Footnotes
|
Greptile SummaryThe PR replaces the racy shared-asset check-and-link sequence with a uniquely staged symlink and atomic destination replacement.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| reflex/assets.py | Introduces staged atomic replacement for shared-asset symlinks and routes shared asset creation through the new helper. |
| tests/units/assets/test_assets.py | Adds focused regression tests for concurrency, destination convergence, platform retry behavior, filename limits, and temporary-link cleanup. |
| news/7039.bugfix.md | Documents the externally visible correction to concurrent shared-asset linking. |
Reviews (9): Last reviewed commit: "test(assets): compare the untouched link..." | Re-trigger Greptile
There was a problem hiding this comment.
All reported issues were addressed
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed across 2 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed across 2 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed across 2 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed across 2 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
Problem
rx.asset(shared=True)symlinks the asset intoassets/external/with acheck-then-act sequence that loses every race it can enter:
dst_file.unlink()raisesFileNotFoundErrorwhen another process removedthe link between this one's
FileExistsErrorand itsunlink().symlink_to()raisesFileExistsErrorwhen another processrecreated it in the same window — unhandled, so it escapes too.
exists()follows the link, so a destination pointing at some other file that happens
to exist makes it skip instead of repointing.
asset()runs during page evaluation for every shared asset, so any concurrentcompile hits this. In our CI it is pytest-xdist workers each standing up an app
instance against a shared checkout: one split of four fails, the other three
pass on the same commit, and which split fails moves between runs.
The existing comment blames docker bind mounts. That is one cause; the general
one is concurrency, with no container involved — and reading as a niche
environment quirk is why the path was left non-idempotent.
Fix
Build the link under a unique temporary name in the destination directory and
rename it into place.
os.replace()is atomic on POSIX and overwrites whateverthe loser of the race left behind, so there is no
unlink(), no retry, and nowindow another process can invalidate:
No
exceptwas widened — errors still propagate, soasset()can never returna path with no symlink behind it and fail later as a 404 on the built asset.
The
exists()/is_symlink()guard is dropped. The replacement fast pathcompares
readlink()against the intended target, which keeps the "don't churnthe file watcher by re-creating a correct link" property that
remove_stale_external_asset_symlinks()cares about, while actually convergingon the right target when a concurrent writer pointed the link elsewhere.
Verification
Both reported failures reproduce deterministically before the fix, via a
monkeypatched competitor that writes to the destination around each
symlinkcall:
test_shared_asset_survives_concurrent_removalFileNotFoundErroratdst_file.unlink()test_shared_asset_survives_concurrent_recreationFileExistsErrorfrom the retrytest_shared_asset_converges_on_correct_target[symlink_to_decoy]test_shared_asset_converges_on_correct_target[regular_file]test_shared_asset_is_thread_safeUnmocked multi-process repro — 8 concurrent processes, 64 tasks x 300 compiles
into one working directory, with a competitor unlinking the destination:
29/64 workers fail before, 0/64 after.
Full unit suite passes (8130 passed, 75.86% coverage);
ruffandpyrightclean.Notes for review
symlink, where
exists()previously left it alone. This is the intendedconvergence guarantee, and it is the proper fix for the bind-mount case the
old comment described rather than the retry. That path is generated territory.
symlink_toandreplaceleaves a dot-prefixed
.tmplink pointing at a valid target, whichremove_stale_external_asset_symlinks()won't reap since it only collectsbroken links. Two-syscall window; happy to add a sweep if reviewers want it.
remove_stale_external_asset_symlinks()isitself check-then-act —
path.unlink()anddirpath.rmdir()will raise underthe same concurrent compiles. Left for a follow-up to keep this diff local.
os.replaceusesMOVEFILE_REPLACE_EXISTINGsoreplacing an existing link should work, but this was only exercised on Linux.
Symlink-privilege requirements are unchanged.