dicos.rs 2.0.0 — standards-conformant codecs, hardened parsing, honest docs - #17
Merged
Conversation
jpeg2k raw-DWT and jpegls Go-compatible bitstreams are pinned by checked-in fixtures + byte-identity/decode regression tests, before any conformance work changes the coders (plan Workstream 0). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces .expect() panics on surface/adapter/device acquisition with an rfd error dialog and a low-power fallback-adapter retry; Camera::MIN/ MAX_DISTANCE consts shared by camera.rs and renderer.rs (plan WS4 B1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Step 1-3 of the T.800 conformance work: Jpeg2kOptions::validate() (cb exps 2..=10 sum<=12, multi-tile rejected), Annex-D context init and C.3.4 BYTEIN marker handling in the MQ coder, floor-semantics 5/3 lifting with conformant multi-level entry points (degenerate regions, hoisted scratch). Legacy raw-DWT pipeline preserved byte-identically behind private truncating-division copies; crate-wide dead_code allow replaced with targeted TODO(t800) allows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
J1-J3b of the conformance work: SOF3/SOS contract enforcement (precision 2..=16, Nf=1, Ss 1..=7, Se=0, Ah=0, Al<P), arithmetic-shift predictors 5-7, EncodeOptions with predictor/point-transform/precision selection, reduced-domain point transform (P' = P - Pt), and the Table H.2 SSSS=16 no-appended-bits case both directions. Also fixes a pre-existing Huffman EOF short-circuit that broke long codes at end of stream. Restart emission reserved for J4 (rejected until then). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
WS3 A1+A2: CodecError/DicosError gain #[non_exhaustive] plus typed variants (Backend, BadPreamble, NestingTooDeep, UnexpectedTag, LengthExceedsLimit/Buffer, UndefinedLengthInFixedSq, Truncated); reader distinguishes clean EOF from partial headers via a byte counter, verifies skip counts, and parses all element sequences through one parse_elements_until; pixel-data integrity downgrades become ParseWarning values surfaced by parse_with_warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Precinct/band/code-block partition math via the general Eq. B-15 form (proven equivalent to the ceil-difference simplification), Mallat placement verified against the conformant DWT packing; canonical tag-tree coder with incremental cross-packet state, proptest-verified bit-exact round-trips. Both unwired until tier-2 (plan WS1 steps 4+6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Completes J4: DRI emission and validation (Ri must be a multiple of samples-per-row), RSTn cycling with order checking, predictor reset at restart rows (first-line semantics), BitReader stops at any marker instead of silently skipping RSTn, BitWriter pad_and_align refactor. Round-trip matrix + error-path tests; 82 unit tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
L0-L2: 1.0.0/Go-compatible scan loops and bit I/O frozen verbatim in legacy.rs (public API reroutes through it, fixtures byte-identical); bitstream.rs rewritten for T.87 single-bit stuffing + A.5.3 limited Golomb with structured marker signaling; context.rs rewritten per spec (A[Q]=max(2,(RANGE+32)/64) init, A.12/A.13 updates, near-aware A.3.3 quantizer with both MAXVAL branches, run contexts with Nn stats). New paths unwired until L3/L4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
WS4 B3: egui drawing moves to ui/{sidebar,slice_panel,controls,
band_slider} with a thin dispatcher; UiState decomposes into
VolumeLibrary/TransferState/RenderSettings/UiActions in state.rs
(RenderSettings is pub, ready to ride RenderParams in B4). app.rs
drops from 1227 to 584 lines; behavior unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
WS4 B4: the eight per-frame-resynced VolumeRenderer fields are gone -- render() reads RenderParams::settings (RenderSettings) directly; same-dimension layer switches reuse the existing GPU texture via queue.write_texture with no allocation or bind-group rebuild. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Full rewrite: 4-row stripe scan, orientation-aware zero-coding tables, 5 sign contexts with XOR, 3 magnitude-refinement contexts, run-length cleanup mode, cleanup-only MSB plane with num_passes = 3*nb-2, live significance in sig-prop and pi/VISITED semantics per D.4. One shared Tier1 state machine drives encode and decode through a single MQ code() primitive, so the two directions cannot drift. Self-golden 8x8 vector pinned until the OpenJPEG cross-check lands in step 10. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
WS4 B5: VolumeLoader runs parse, decode, threat sidecar merge, pack_for_gpu, and center_of_mass on a worker thread; the render thread only performs wgpu uploads from the precomputed buffer. Spinner + disabled Open buttons while in flight, load failures surfaced in the sidebar. Layer switching keeps the synchronous pack fallback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
WS4 B6: render_composite parallelizes by row via composite_pixel with a pixel-identity test against a sequential reference; the multiplicative opacity chain (density curve x band alpha x global x gradient boost) is documented on TransferFunction and cross-referenced in raycast.wgsl. ColorBand.alpha retained as material-selective opacity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Steps 7+8: B.10 packet header codec (FF-stuffed bit I/O, inclusion + zero-bitplane tag trees, Table B.4 pass counts -- corrected from the plan's prose, which was non-conformant -- Lblock length signaling) and the DWT-to-EBCOT-to-LRCP-packet tile pipeline with Mb from a QCD closure. Legacy raw-DWT tile coder frozen verbatim in legacy.rs; the public API still routes through it until step 9. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
L3-L5: regular-mode scan per A.4-A.6 (column-0 Rc border model, A.5.2 special-case mapping, near-lossless reconstruction over RANGE), run mode per A.7/A.19-A.23 (RUNindex persists across lines, EOL lone-bit rule, limited Golomb interruption coding), LSE ID=1 presets honored, DRI/multi-component rejected loudly, precision/NEAR validated. Public default is now Profile::T87; Profile::LegacyGo keeps the frozen Go bitstream, fixtures byte-identical. Also fixes a trailing-0xFF stuff byte omission found during bring-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Step 9: encode() emits real T.800 (DC level shift, COD from options,
E-10 QCD exponents with G=2, verified Psot); decode() walks markers
structurally with a full support-matrix (specific Unsupported per
legal-but-unsupported feature) and routes through the tier-2 pipeline.
LegacyPolicy{Auto,StandardOnly,LegacyOnly} behind default-on
legacy-decode feature; Auto discriminates on the all-zero-QCD anomaly
plus payload fingerprint. Legacy fixtures decode under Auto and are
refused under StandardOnly. Also documents that the 1.0.0 writer
emitted a wrong SOD code (FFD3); both accepted at the legacy offset.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
WS3 A3+A4: GrayImage fields private behind accessors; Dataset::iter yields &Element (+ tags()); rows/columns/bits_allocated/ pixel_representation return Option (number_of_frames keeps its DICOM-defined default of 1); root re-exports for the common surface. Registry adapters collapse into codec_adapter! with typed Backend errors and honest DimensionMismatch; decode_frame no longer sniffs (explicit decode_frame_sniffed instead); sniff_codec drops the RLE guess and JP2 magic and walks JPEG markers structurally; jpeg2k decodes pinned to LegacyPolicy::StandardOnly for untrusted data. roxel distinguishes MissingAttribute from zero-valued Rows/Columns. MIGRATION.md started. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
WS4 B2 (pure code motion): Volume + GPU packing stay in mod.rs with re-exports preserving every crate::volume::* path; DICOS extraction and value parsing move to dataset.rs; ThreatBox parsing/coloring and sidecar loading move to threats.rs. Tests split along the same seams; no consumer edits needed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
WS3 A5: reader tests move to reader/{tests,test_util}.rs with reusable
wire-format builders; the luggage generator becomes an example and
imports UIDs from the new dicos::uid module instead of local dupes;
write/parse fixed-point proptests (4 properties x 100 cases); cargo-fuzz
parse_dataset target under fuzz/ (workspace-excluded); criterion
removed from workspace deps; codec_adapter! gated to codec features.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds Truncated/InvalidParameter variants matching jpegls, checked width*height with a 2^28-pixel cap before allocation in both decoders, structured truncation errors in RLE segment handling, and removes the unreachable MAX_SEGMENTS encode check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jpegli: PASSES two-direction interop with libjpeg-turbo 3.1.4 (our streams decode under djpeg; cjpeg -lossless psv/Pt/restart fixtures decode exactly). jpegls: env-gated charls harness (fixtures populated where charls tooling exists). jpeg2k: OpenJPEG fixtures + two-direction harness checked in but IGNORED — decoding real opj_compress output exposed a tier-1/MQ conformance defect that self-round-trips masked; root-cause fix to follow before the ignores come off. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two shared encoder/decoder deviations from T.800 that self-round-trips masked and OpenJPEG interop exposed: the MQ coder placed the MPS sub-interval at the interval base (producing bitwise-complement codewords vs Annex C), and the forward 5/3 DWT ran rows-then-columns (observable through floor rounding). Both fixed to the normative behavior; our code-block bytes are now byte-identical to OpenJPEG's for the cross-check vector, and the two-direction opj interop tests are un-ignored and green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mutation tests (truncation at every prefix, bit flips, chopped trailing markers, reordered RSTn) across all four codecs; cargo-fuzz decode targets per codec crate (jpegls covers both profiles); CI gains an interop job (OpenJPEG both directions; jpegli fixture-decode only on ubuntu's turbo 2.x) and a weekly fuzz-smoke job. The jpegli mutation suite found a real bug -- unchecked SSSS > 16 from a corrupted DHT panicked in debug and silently mis-read in release -- fixed with a range check in the scan loop and the test un-ignored. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
WS3 A6: workspace and inter-crate versions to 2.0.0; READMEs and lib docs rewritten to the verified conformance claims (OpenJPEG and libjpeg-turbo interop noted); MIGRATION.md completed; PUBLISHING.md updated; testdata/ (37MB) and roxel-lrg.gif (12MB) untracked forward- only with CI generating synthetic fixtures via the example; README GIF points at the v2.0.0 release asset (upload on release). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The workspace /target/ ignore did not match crates/*/fuzz/target, so ~213 MB of fuzz build outputs (856 files) were committed. Untrack them and extend .gitignore to cover every nested fuzz target/corpus/artifacts dir.
Round-2 hardening from the final Codex review: crafted QCD exponents + guard bits could drive num_bitplanes >= 32 and shift past 31 bits (debug panic / release corruption) -- rejected at QCD parse plus a defensive guard before the tier-1 shift loop; multi-precinct images (dimension > 32768) were silently packed as one packet per resolution -- now rejected loudly as Unsupported in both encode and decode, making the README's 'rejected loudly' guarantee true; CodecError and Jpeg2kOptions gain #[non_exhaustive] for 2.x semver headroom. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Round-2 hardening from the final Codex review: the Basic Offset Table length drove an unbounded reserve() before any bytes were read (a tiny file could request ~1B entries) -- entries are now pushed in the read loop with a bounded speculative reservation; encapsulated fragments are assembled into frames using the BOT offsets per PS3.5 A.4 instead of exposing every fragment as its own frame; native 8-bit pixel data respects BitsAllocated (one sample per byte) instead of always pairing into u16, with a ParseWarning when BitsAllocated is absent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Round-2 hardening from the final Codex review: SOF55 rejects zero and oversized dimensions (2^28-pixel cap) before the pixel-buffer alloc (was an ~8 GiB bomb); LSE ID=1 presets are validated per T.87 C.2.4.1.1 (MAXVAL within precision, NEAR < T1 <= T2 <= T3 <= MAXVAL, RESET >= 3) instead of feeding context arithmetic unchecked. The frozen-fixture regenerator is demoted from an #[ignore] test to a reference-only function so the suite has zero ignored tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Round-2 hardening from the final Codex review: oversubscribed DHT tables (Kraft check), SOS DC selector >= 4, and zero SOF3 dimensions (also a DRI divide-by-zero) all now return errors instead of panicking; SSSS category 16 is rejected below full precision per T.81 H.1.2.2; truncated entropy now returns Truncated instead of zero-padding to a full-size wrong Ok. Mutation suite strengthened to assert Err on mid-entropy truncation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… enums Follow-through on the final Codex review: Value and PixelData gain #[non_exhaustive] (dicosctl grows a wildcard arm); README corrects the SV1 conflation (Process 14 lossless, default SV1, predictors 1-7 selectable), the CharLS claim (harness present, self-skips until fixtures supplied), and refreshes test counts to 536; MIGRATION describes the actual LegacyPolicy::Auto QCD-fingerprint flow and the new non_exhaustive data enums. The jpeg2k multi-precinct rejection and non_exhaustive error enum make the previously-inaccurate 'rejected loudly' and 'every codec error enum' claims true. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The A6 forward-only cleanup untracked all of testdata/, but bag_ct.dcs is a curated sample the reader and luggage integration tests assert on and the example generator cannot reproduce -- so those tests silently skipped on a fresh checkout. Re-track the curated samples (bag_ct.dcs plus the two threat sample dirs); only the CI-generated testdata/synthetic/ stays gitignored. README/MIGRATION/CI reconciled. Note: the two threat sample directories (~32 MB) are not referenced by any automated test -- they are roxel demo data -- and can be dropped later if repo weight matters. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
dicos.rs 2.0.0 — standards-conformant codecs, hardened parsing, honest docs
A four-way critical review found the just-published 1.0.0 crates overstated conformance — most severely,
pure_jpeg2kemitted raw DWT coefficients, not JPEG 2000 at all. This release makes all five crates genuinely conformant within an explicit, documented scope, hardens every decode path against untrusted input, and corrects the docs to match what the code actually does.Planned and adversarially reviewed with Codex (gpt-5.6-sol, high effort) twice — once on the plan, once on the finished branch.
Codecs — real conformance, verified against reference implementations
LegacyPolicy.Profile::LegacyGopreserves the frozen 1.0.0 / Go-compatible bytes.Untrusted-input hardening (second review pass)
Fixed a class of panics and allocation bombs on attacker-controlled bytes across every decoder: DHT oversubscription, SOS selector bounds, zero dimensions, QCD shift overflow, the DICOM Basic Offset Table allocation bomb, SOF pixel-count caps, strict truncation handling, and multi-precinct rejection. Multi-fragment DICOM frame assembly and native 8-bit pixel handling corrected. Mutation-test suites, per-codec cargo-fuzz targets, and a two-direction interop CI job added.
dicos core API 2.0 (breaking)
Typed
#[non_exhaustive]errors, truncation semantics,ParseWarning, privateGrayImagefields with accessors,Option-returning attribute accessors,Dataset::iter() -> &Element. See MIGRATION.md for the full 1.x → 2.0 guide.roxel viewer
Background volume loading off the render thread, state consolidation, module splits, rayon-parallel slice compositing, GPU error dialogs.
Verification
cargo clippy --workspace --all-features --all-targetsclean;cargo package --workspace --lockedsucceeds.🤖 Generated with Claude Code