perf(schema_engine): cut per-node bookkeeping from compiled validation - #126
Conversation
Validation spent most of its time on bookkeeping around each schema node
rather than on assertions. No change to what is accepted or rejected.
- Keep the reference-cycle guard on a stack compared by identity instead
of a Set that hashed the mutable EvaluationPath via objectid per node.
- Pass the CompiledNode down instead of re-finding it by NodeId, which
has no hash method, for every child and $ref.
- Resolve static $ref targets and properties children once per compiled
graph into a JumpTable indexed by node; subschema/select share it.
- Record a shared sentinel issue under branches read only for validity
(anyOf/oneOf alternatives, not, if, contains) instead of rendering a
path per failure; counts and max_issues are unchanged.
- Check keyword presence before keyword_applies.
- Runtime: cache the subschema view per descriptor under graph_lock, and
decode Union{Absent, Nothing, X} directly when it has one real variant.
|
We found significant slowdowns in the serde path without these changes. |
|
@tanmaykm is there anything else needed to merge this PR? The impact on our serde is pretty significant due some of the nesting in the data model. |
|
@quinnj is this PR more on your camp to review? |
tanmaykm
left a comment
There was a problem hiding this comment.
Thanks for this. I read through all six changes, traced each shortcut against the general path it bypasses, and reproduced the speedup independently on Julia 1.12.7 and 1.13.0. One small change requested below, with a tested patch.
Verification
Synthetic workload: 20k nested branch objects through $ref'd models with nullable anyOf properties.
| Case | main 1.12 | PR 1.12 | main 1.13 | PR 1.13 |
|---|---|---|---|---|
isvalid, valid document |
1653 ms / 909 MiB | 407 ms / 262 MiB | 1498 ms / 913 MiB | 385 ms / 284 MiB |
validate collect-all, valid |
1656 ms | 430 ms | 1477 ms | 406 ms |
validate fail-fast, one deep failure |
1552 ms | 415 ms | 1405 ms | 384 ms |
validate collect-all, one deep failure |
1580 ms | 435 ms | 1431 ms | 428 ms |
Issue paths and reasons are identical on both branches. A $ref to an embedded resource with its own $id (the canonical-lookup branch of the JumpTable) resolves and reports identically too.
Correctness notes from the read-through, all fine:
- The cycle-guard stack does the same comparisons the
Setdid (isequalon the tuple fell back to identity for the mutable path) and is only touched whentracks_cyclesis set. node.idequals the old_compiled_node(schema, compiled.id).idsince the node map stores each compiled node under its own id.- The
$refshortcut reproduces_reference_targetfor the static case exactly, and every unresolved slot falls back to the old path including its errors. Indices are 1-based and contiguous so the vector sizing is right. - Invalid speculative children are never absorbed (
anyOf/oneOfalternatives,not,if,contains), so the sentinel cannot leak into returned issues, and counts are preserved formax_issues. - The single-variant
_decode_unionpath is equivalent in bothoneofmodes and produces the same message.
One change requested
The max_issues error loses its path under a discarded branch. When the limit is hit inside a speculative branch in collect-all mode, the EvaluationError names the instance root instead of the real location, because _SPECULATIVE_ISSUE carries an empty path and both throw sites read .path off it. Same on 1.12 and 1.13.
s = SchemaEngine.CompiledSchema(Dict("$schema" => SchemaEngine.DRAFT202012.uri,
"properties" => Dict("p" => Dict("anyOf" => Any[Dict("allOf" => Any[false, false, false])]))))
SchemaEngine.validate(s, Dict("p" => 1); fail_fast = false, max_issues = 2)
# main: ... for instance "/p": the issue limit was reached
# PR: ... for the instance root: the issue limit was reachedAcceptance is unchanged, but it is a diagnostic regression. The patch below keeps the EvaluationPath object unrendered in the sentinel's val field and renders it only at the two throw sites, so the hot path still never builds a path string. With it applied the reproducer reports /p again, the compiled-schema and runtime testsets pass on 1.12.7 and 1.13.0, and the benchmark is unchanged within noise. A test pinning the path in that message would be good too.
--- a/src/schema_engine/compiled_validation.jl
+++ b/src/schema_engine/compiled_validation.jl
@@ -122,9 +122,16 @@ struct _LazyIssue
value::Any
end
-# Stands in for every issue raised under a speculative branch. Counted like a real one, so
-# `max_issues` behaves the same, but never built: building one renders the instance path.
-const _SPECULATIVE_ISSUE = SingleIssue(nothing, "", "speculative", nothing)
+# Stands in for an issue raised under a speculative branch. Counted like a real one, so
+# `max_issues` behaves the same, but the instance path is kept unrendered in `val` and only
+# rendered if the issue limit is reached and the path has to appear in the error.
+_speculative_issue(path::EvaluationPath) = SingleIssue(nothing, "", "speculative", path)
+
+function _issue_path(issue::SingleIssue)
+ issue.reason == "speculative" && issue.val isa EvaluationPath &&
+ return _path_string(issue.val::EvaluationPath)
+ return issue.path
+end
function _invalidate!(
result::EvaluationResult,
@@ -133,7 +140,7 @@ function _invalidate!(
)
!context.collect_all && !result.valid && return result
if context.speculative > 0
- return _invalidate!(result, context, _SPECULATIVE_ISSUE)
+ return _invalidate!(result, context, _speculative_issue(lazy.path))
end
return _invalidate!(
result,
@@ -162,7 +169,7 @@ function _invalidate!(
issue_count < context.max_issues || throw(
EvaluationError(
context.schema.root,
- issue.path,
+ _issue_path(issue),
"the issue limit was reached",
),
)
@@ -219,7 +226,7 @@ function _absorb!(
issue_count + length(child_issues) <= context.max_issues || throw(
EvaluationError(
context.schema.root,
- isempty(child_issues) ? "" : first(child_issues).path,
+ isempty(child_issues) ? "" : _issue_path(first(child_issues)),
"the issue limit was reached",
),
)Non-blocking
- Lock on the hot path.
_schema_atnow takesgraph_locktwice per model validation, where the old path was lock-free once the graph existed. Negligible single-threaded, but a serialization point for multi-threaded decoding. Fine as a follow-up: an immutable cache swapped atomically, or precomputing views for all known descriptors when the graph is built. _NO_SCOPEis a shared mutable global. It is only compared, never mutated, so safe today. A one-line comment saying it must never be pushed to would protect the next reader.
With the error-path fix in, this is good to merge.
…ulative branches The speculative issue sentinel carried an empty path, so hitting max_issues inside an anyOf/oneOf/not/if/contains branch reported the instance root. Keep the unrendered EvaluationPath in the sentinel and render it only at the two throw sites.
|
@tanmaykm I applied the fixes and opened an issue about the lock to track later and explore |
Validating generated models spends most of its time on bookkeeping around each schema node
rather than on the assertions themselves. On a 44,856-row document (ACTIVSg10k power system,
PowerOpenAPIModels), decoding with
validate = truedrops from 5.72 s to 1.57 s; thenon-validating decode drops from 0.82 s to 0.40 s. No change to what is accepted or rejected.
Changes
EvaluationContext.activewas aSet{Tuple{Int,EvaluationPath,Tuple{Vararg{ResourceId}}}}, touched three times per node(
in,push!,delete!). Hashing the mutableEvaluationPathgoes throughobjectid,and building the scope key converted a
Vectorto aTupleeach time. It is now a stackscanned with
===on the path: entries are pushed and popped in call order, so the keyand the detection are the same.
CompiledNodedown._evaluate_schemaand its helpers received aNodeIdand looked the compiled node up again (
evaluation_nodes[node]) for every child and$ref.NodeIdhas nohashmethod, so each lookup hashed its strings recursively.They now receive the
CompiledNodethat_evaluate_compiled_nodealready holds.JumpTableonCompiledSchema. Static$reftargets andpropertieschildren areresolved once per compiled graph into vectors indexed by
CompiledNode.index, replacinga
(NodeId, keyword)and an(Int, Tuple{String,String})lookup per reference and perproperty.
subschema/selectshare their template's table. Anything unresolved atcompile time (
$dynamicRef,$recursiveRef, a missing transition) takes the existingpath unchanged.
anyOf/oneOfalternatives,not, anifcondition and
containscandidates are only read for validity, but every failing onebuilt a
SingleIssue, rendering its instance path. Under those branches a sharedsentinel issue is recorded instead, so issue counts and
max_issuesbehave identically.keyword_applies. It walks a chain of stringcomparisons and was evaluated before checking the schema had the keyword at all.
_schema_atcaches the subschema view per(direction, resource, pointer)under the existinggraph_lock(it was rebuilt, URI parse included, on everyvalidation);
_decode_uniondecodes aUnion{Absent, Nothing, X}directly when there isone real variant.
Tests
New direct coverage:
Compiled evaluation shortcuts(issue paths and reasons through$ref'dproperties, a fully failinganyOfreporting only itself, collect-all issue order followingdeclared property order,
if/notstill deciding validity) andruntime validation shortcuts(the single-variant union path and its error, the cached subschema view). The existing
Compiled evaluation safetyset already pins cycle detection andmax_issuesunderdiscarded branches; both pass unchanged.
OPENAPI_SCHEMA_SUITE=all: all testsets pass; official JSON-Schema-Test-Suite9884/9884 across drafts 4–2020-12 in both
fail_fastmodes.OPENAPI_CORPUS_TESTS=smalland=all: pass.git diff --check: clean.Downstream, PowerSystems.jl's suite (which round-trips every model through these
validators) gives the same result on this branch as on its base.
🤖 Generated with Claude Code