Skip to content

Add support for custom validators to GatewayStruct - #315

Merged
timkpaine merged 4 commits into
mainfrom
ac/structs
Aug 13, 2026
Merged

Add support for custom validators to GatewayStruct#315
timkpaine merged 4 commits into
mainfrom
ac/structs

Conversation

@arhamchopra

@arhamchopra arhamchopra commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Description

Adds a lightweight, dynamic validator registry to GatewayStruct. Application code (or other modules/structs) can attach validation and normalization/enrichment logic to a struct type at runtime — using arbitrary callables (lambdas, bound methods) — without subclassing or hand-writing pydantic validators. Validators run automatically during pydantic validation.

Following the pydantic model, a validator both validates and may transform: it receives a value, returns the (possibly transformed) value, and raises to reject. There is a single concept (no separate "transformer") with before/after modes.

New APIs on GatewayStruct:

  • add_validator(fn, *, mode="after") — register fn(value) -> value. The validator returns the (possibly transformed) value and raises (e.g. ValueError) to reject the input (surfaced by the REST API as a 422). mode="before" (alias "pre") runs on the raw input (e.g. a dict) prior to construction — useful for accepting legacy/aliased shapes; mode="after" (alias "post", the default) runs on the constructed struct. Returns fn, so it works as a bare decorator (@S.add_validator) or a decorator factory (@S.add_validator(mode="before")).
  • clear_validators(*, mode=None) — remove registrations; mode=None clears both, otherwise just before/pre or after/post (teardown / idempotency when registering dynamically at graph-build time).

Behavior:

  • Registrations are aggregated across the MRO, so base-class and subclass validators both run.
  • They execute inside the wrap validator (_validate_gateway_struct), which is the robust funnel: subclasses frequently override _validate_gateway_struct_after without calling super(), which would bypass an after-hook-based registry — running here avoids that.
  • Ordering per validation: before validators → construct → after validators → the struct's _validate_gateway_struct_after hook. Because after validators run before the hook, they may normalize data that the hook then checks.
  • Fires on any pydantic path (REST /send, model_validate, TypeAdapter, JSON snapshot replay) and for nested structs when a containing struct is validated. It does not run on native (non-pydantic) construction.

Safety / guards:

  • A validator returning None raises a clear error (a validator must return the value; otherwise pydantic would silently yield None).
  • mode and callable-ness are validated at registration (fail-fast).
  • Fully backward compatible — a no-op for any struct with no registrations.

Type of Change

  • Bug fix
  • New feature
  • Documentation update
  • Refactor / code cleanup
  • CI / build configuration
  • Other (describe below)

Checklist

  • Linting passes (make lint) — ruff check + format clean; PR only touches Python
  • Tests pass (make test)
  • New tests added for new functionality — tests/utils/struct/test_validators_registry.py (~30 cases: before/after transform + raise-to-reject, pre/post aliases, MRO aggregation, sibling isolation, decorator (bare + factory), bound-method validator, nested firing, None-guard, override-without-super() regression, after-validators-run-before-after-hook, clear_validators (all + by mode), invalid-mode / non-callable rejection, manual _run_validators on the CSP path)
  • Documentation updated (if applicable) — docstrings on all new methods + a "Custom Struct Validators" section in docs/wiki/Develop.md
  • Changelog / version bump (if applicable)

@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Test Results

746 tests  +51   738 ✅ +51   7m 34s ⏱️ -1s
  1 suites ± 0     8 💤 ± 0 
  1 files   ± 0     0 ❌ ± 0 

Results for commit c33b113. ± Comparison against base commit 960bd4f.

♻️ This comment has been updated with latest results.

@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.17081% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.34%. Comparing base (960bd4f) to head (c33b113).

Files with missing lines Patch % Lines
...way/tests/utils/struct/test_validators_registry.py 99.20% 4 Missing ⚠️
csp_gateway/utils/struct/base.py 98.88% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #315      +/-   ##
==========================================
+ Coverage   87.92%   88.34%   +0.41%     
==========================================
  Files         143      144       +1     
  Lines       15061    15658     +597     
  Branches     1446     1475      +29     
==========================================
+ Hits        13243    13833     +590     
- Misses       1818     1825       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@arhamchopra arhamchopra changed the title Add support for custom validators/transformers to GatewayStruct Add support for custom validators to GatewayStruct Aug 11, 2026
Comment thread csp_gateway/tests/utils/struct/test_validators_registry.py Outdated
Signed-off-by: Arham Chopra <arham.chopra@cubistsystematic.com>
Signed-off-by: Arham Chopra <arham.chopra@cubistsystematic.com>
Signed-off-by: Arham Chopra <arham.chopra@cubistsystematic.com>
Dispatch the _validate_gateway_struct_after hook on the concrete struct type so a
subclass instance in a base-annotated field cannot bypass its own rules, matching
the dispatch already used for registered validators.

Require an "after" validator to return an instance of the validated class, and
only reject a None return when the incoming value was not itself None so a null
input reports a type error rather than blaming the validator.

Cache the MRO-resolved validators as a tuple and sample the registry version
before the walk, so the resolved list cannot be mutated through the returned
reference and a racing registration cannot publish a stale list stamped current.

Factor the id/timestamp scrub into _scrub_identity and apply it both before and
after the "before" validators, so one cannot reinstate a caller-supplied id.
Honor force_new_id and force_new_timestamp for instance input by rebuilding the
struct, since scrubbing in place would mutate the caller's object and strand it
in the lookup registry under its old id.

Document that only ValueError and AssertionError reach the client as a 422, that
registering on a shared base instruments every struct in the process, and correct
the validation entry points named in the docs.

Signed-off-by: Tim Paine <3105306+timkpaine@users.noreply.github.com>
@timkpaine
timkpaine merged commit 22a1587 into main Aug 13, 2026
11 checks passed
@timkpaine
timkpaine deleted the ac/structs branch August 13, 2026 22:59
timkpaine added a commit that referenced this pull request Aug 14, 2026
Brings in the GatewayStruct validator registry (#315) along with the two copier
update rounds and the state API work.

Resolved a conflict in .github/workflows/build.yaml: took main's re-indented test
job, which corrects steps that sat at four spaces while their neighbours used
six, then re-applied v3's junit upload fix from f757918 so the test job keeps
path/files at junit.xml rather than reverting to the '**/junit.xml' glob. The
build job's junit steps are new from main and are left as main has them.

Signed-off-by: Tim Paine <3105306+timkpaine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants