Generic JSON input

[Stranger6667]

At the moment, jsonschema supports only serde_json, which is convenient, but implies significant overhead in some cases. For example, in Python bindings, I got to convert Python structures to serde_json::Value first and this conversion takes up to 85% of the overall validation time. It will be much faster if those Python objects will be used directly without conversion.

It could be done with a generic Json trait that will implement some API on the input data (likely similar to serde_json). In this case, implementing Json on a newtype wrapper around PyAny will eliminate the need for conversion. Additionally, it will reduce the memory footprint as no copying will be needed.

For existing serde_json data it should be optimized away as the wrapper will just delegate the calls to serde_json::Value.

[Adityakumar37]

Hi @Stranger6667 👋

I've been reading through the codebase and I'm interested in working on this. Before I start, I want to confirm my understanding.

I traced the overhead to crates/jsonschema-py/src/ser.rs — specifically the to_value() function which fully converts PyAny serde_json::Value before validation begins. Every entry point (is_valid, validate, iter_errors, evaluate) calls this unconditionally.

The fix as I understand it:

1. Define a generic Json trait in the core jsonschema crate that mirrors serde_json::Value's API
2. Make the Validate trait in validator.rs generic over T: Json instead of hardcoded to &Value
3. Implement Json directly for *mut PyObject in the Python bindings so ser::to_value() is bypassed and Python objects are walked lazily during validation

My main concern is the ripple effect across all ~40 keyword files in crates/jsonschema/src/keywords/ which all currently take &Value directly.

I also noticed this would unblock #945 (recursive objects) as a nice side effect, since the crash there also happens inside ser.rs during conversionbut that's a separate concern and my focus here is purely on #239.

Does this approach sound right?

[Stranger6667]

@Adityakumar37

I think it would be way too big jump right now and I'd advice against it.

There are multiple attempts already by me in stale branches in this repo and also jsonlike that I wrote ~2.5 years ago for this purpose.

Instead, I'd suggest taking a look at #851 and #883. It is a prerequisite for this issue and has a smaller scope. The core idea is that schema compilation does not need anything owned (except for small bits for precise error messages) and all dependencies to it should be provided by references.

For example, Registry owns all the resources, but it is not really necessary except for fetched resources. So the compilation flow would be:

• User provides borrowed resources that are at hand
• Registry builds internal navigation structures for reference resolution and fetches all reachable external resources to its internal storage (technically it could be a registry that has only borrowed data and it produces "resolution context" that borrows from registry + contains this fetched data - it avoids self-references and splits concerns)
• Compiler borrows this registry as a way to resolve references

Then we can move to validation. Technically it could be the other way around, but I think it is better to resolve this part first and then implement generic JSON specifically for schema compilation.

So, I think we need to conceptually simplify the compilation flow first without adding generics because it could make the flow less visible & approachable.

Another thing is that your comment strongly reminds me what LLMs gave me when I asked them to provide me an overview of this problem area. In my experience they work better when the scope is smaller.

After that it should be way more clear about possible obstacles for introducing generics to compilation & validation.

[Adityakumar37]

@Stranger6667
Thanks for the direction! I'll look into #851 and #883. I've already been going through the codebase so I'll take some time to understand these properly.

[Stranger6667]

@Adityakumar37 FYI. Over the weekend I took a look at the Registry split issue and right now I have a working prototype - will push in a few days once I am sure about a few performance aspects.

[Adityakumar37]

Thanks for the update! I went through the dd/rework-registry branch. I can see Registry now only stores documents via Cow<'doc, Value>Borrowed for user-provided schemas, Owned for fetched ones and ResolutionContext takes over computing resources and anchors by walking all documents eagerly in new() I also noticed that build_validator in compiler.rs calls registry.context() on every validation call, which means resources and anchors are rebuilt from scratch each time
In the meantime, is there another open issue you'd suggest I look at?

[corentinguilloteau]

Hi,

I’m currently building a Rust equivalent of pajv, aiming to use JSON Schema as a validation layer for various configuration formats. It’s conceptually similar to jsonlike that you mentioned earlier, although the end goal is somewhat different.

My current implementation introduces a clone operation from input format struct T to serde_json::Value, which I then use as a proxy to validate T. This adds noticeable overhead, and from what I understand, the improvement discussed here could significantly reduce it.

I’d be very interested in seeing this move forward and would be happy to help — whether that’s testing or contributing to implementation.

Thanks again for your work!