Skip to content

perf(schema_engine): cut per-node bookkeeping from compiled validation - #126

Merged
tanmaykm merged 2 commits into
JuliaComputing:mainfrom
QXT-Energy:jd/schema-engine-hot-path
Sep 25, 2026
Merged

tanmaykm merged 2 commits into
JuliaComputing:mainfrom
QXT-Energy:jd/schema-engine-hot-path

Conversation

@jd-lara

@jd-lara jd-lara commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Validating generated models spends most of its time on bookkeeping around each schema node
rather than on the assertions themselves. On a 44,856-row document (ACTIVSg10k power system,
PowerOpenAPIModels), decoding with validate = true drops from 5.72 s to 1.57 s; the
non-validating decode drops from 0.82 s to 0.40 s. No change to what is accepted or rejected.

Changes

  1. Reference-cycle guard without hashing. EvaluationContext.active was a
    Set{Tuple{Int,EvaluationPath,Tuple{Vararg{ResourceId}}}}, touched three times per node
    (in, push!, delete!). Hashing the mutable EvaluationPath goes through objectid,
    and building the scope key converted a Vector to a Tuple each time. It is now a stack
    scanned with === on the path: entries are pushed and popped in call order, so the key
    and the detection are the same.
  2. Carry the CompiledNode down. _evaluate_schema and its helpers received a NodeId
    and looked the compiled node up again (evaluation_nodes[node]) for every child and
    $ref. NodeId has no hash method, so each lookup hashed its strings recursively.
    They now receive the CompiledNode that _evaluate_compiled_node already holds.
  3. JumpTable on CompiledSchema. Static $ref targets and properties children are
    resolved once per compiled graph into vectors indexed by CompiledNode.index, replacing
    a (NodeId, keyword) and an (Int, Tuple{String,String}) lookup per reference and per
    property. subschema/select share their template's table. Anything unresolved at
    compile time ($dynamicRef, $recursiveRef, a missing transition) takes the existing
    path unchanged.
  4. No issues built for discarded branches. anyOf/oneOf alternatives, not, an if
    condition and contains candidates are only read for validity, but every failing one
    built a SingleIssue, rendering its instance path. Under those branches a shared
    sentinel issue is recorded instead, so issue counts and max_issues behave identically.
  5. Test keyword presence before keyword_applies. It walks a chain of string
    comparisons and was evaluated before checking the schema had the keyword at all.
  6. Runtime: _schema_at caches the subschema view per (direction, resource, pointer) under the existing graph_lock (it was rebuilt, URI parse included, on every
    validation); _decode_union decodes a Union{Absent, Nothing, X} directly when there is
    one real variant.

Tests

New direct coverage: Compiled evaluation shortcuts (issue paths and reasons through $ref'd
properties, a fully failing anyOf reporting only itself, collect-all issue order following
declared property order, if/not still deciding validity) and runtime validation shortcuts
(the single-variant union path and its error, the cached subschema view). The existing
Compiled evaluation safety set already pins cycle detection and max_issues under
discarded branches; both pass unchanged.

  • Julia 1.12, OPENAPI_SCHEMA_SUITE=all: all testsets pass; official JSON-Schema-Test-Suite
    9884/9884 across drafts 4–2020-12 in both fail_fast modes.
  • Julia 1.10: all testsets pass.
  • OPENAPI_CORPUS_TESTS=small and =all: pass.
  • git diff --check: clean.

Downstream, PowerSystems.jl's suite (which round-trips every model through these
validators) gives the same result on this branch as on its base.

🤖 Generated with Claude Code

Validation spent most of its time on bookkeeping around each schema node
rather than on assertions. No change to what is accepted or rejected.

- Keep the reference-cycle guard on a stack compared by identity instead
  of a Set that hashed the mutable EvaluationPath via objectid per node.
- Pass the CompiledNode down instead of re-finding it by NodeId, which
  has no hash method, for every child and $ref.
- Resolve static $ref targets and properties children once per compiled
  graph into a JumpTable indexed by node; subschema/select share it.
- Record a shared sentinel issue under branches read only for validity
  (anyOf/oneOf alternatives, not, if, contains) instead of rendering a
  path per failure; counts and max_issues are unchanged.
- Check keyword presence before keyword_applies.
- Runtime: cache the subschema view per descriptor under graph_lock, and
  decode Union{Absent, Nothing, X} directly when it has one real variant.
@jd-lara

jd-lara commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

We found significant slowdowns in the serde path without these changes.

@jd-lara

jd-lara commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

@tanmaykm is there anything else needed to merge this PR? The impact on our serde is pretty significant due some of the nesting in the data model.

@jd-lara

jd-lara commented Sep 25, 2026

Copy link
Copy Markdown
Contributor Author

@quinnj is this PR more on your camp to review?

@tanmaykm tanmaykm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. I read through all six changes, traced each shortcut against the general path it bypasses, and reproduced the speedup independently on Julia 1.12.7 and 1.13.0. One small change requested below, with a tested patch.

Verification

Synthetic workload: 20k nested branch objects through $ref'd models with nullable anyOf properties.

Case main 1.12 PR 1.12 main 1.13 PR 1.13
isvalid, valid document 1653 ms / 909 MiB 407 ms / 262 MiB 1498 ms / 913 MiB 385 ms / 284 MiB
validate collect-all, valid 1656 ms 430 ms 1477 ms 406 ms
validate fail-fast, one deep failure 1552 ms 415 ms 1405 ms 384 ms
validate collect-all, one deep failure 1580 ms 435 ms 1431 ms 428 ms

Issue paths and reasons are identical on both branches. A $ref to an embedded resource with its own $id (the canonical-lookup branch of the JumpTable) resolves and reports identically too.

Correctness notes from the read-through, all fine:

  • The cycle-guard stack does the same comparisons the Set did (isequal on the tuple fell back to identity for the mutable path) and is only touched when tracks_cycles is set.
  • node.id equals the old _compiled_node(schema, compiled.id).id since the node map stores each compiled node under its own id.
  • The $ref shortcut reproduces _reference_target for the static case exactly, and every unresolved slot falls back to the old path including its errors. Indices are 1-based and contiguous so the vector sizing is right.
  • Invalid speculative children are never absorbed (anyOf/oneOf alternatives, not, if, contains), so the sentinel cannot leak into returned issues, and counts are preserved for max_issues.
  • The single-variant _decode_union path is equivalent in both oneof modes and produces the same message.

One change requested

The max_issues error loses its path under a discarded branch. When the limit is hit inside a speculative branch in collect-all mode, the EvaluationError names the instance root instead of the real location, because _SPECULATIVE_ISSUE carries an empty path and both throw sites read .path off it. Same on 1.12 and 1.13.

s = SchemaEngine.CompiledSchema(Dict("$schema" => SchemaEngine.DRAFT202012.uri,
    "properties" => Dict("p" => Dict("anyOf" => Any[Dict("allOf" => Any[false, false, false])]))))
SchemaEngine.validate(s, Dict("p" => 1); fail_fast = false, max_issues = 2)
# main: ... for instance "/p": the issue limit was reached
# PR:   ... for the instance root: the issue limit was reached

Acceptance is unchanged, but it is a diagnostic regression. The patch below keeps the EvaluationPath object unrendered in the sentinel's val field and renders it only at the two throw sites, so the hot path still never builds a path string. With it applied the reproducer reports /p again, the compiled-schema and runtime testsets pass on 1.12.7 and 1.13.0, and the benchmark is unchanged within noise. A test pinning the path in that message would be good too.

--- a/src/schema_engine/compiled_validation.jl
+++ b/src/schema_engine/compiled_validation.jl
@@ -122,9 +122,16 @@ struct _LazyIssue
     value::Any
 end
 
-# Stands in for every issue raised under a speculative branch. Counted like a real one, so
-# `max_issues` behaves the same, but never built: building one renders the instance path.
-const _SPECULATIVE_ISSUE = SingleIssue(nothing, "", "speculative", nothing)
+# Stands in for an issue raised under a speculative branch. Counted like a real one, so
+# `max_issues` behaves the same, but the instance path is kept unrendered in `val` and only
+# rendered if the issue limit is reached and the path has to appear in the error.
+_speculative_issue(path::EvaluationPath) = SingleIssue(nothing, "", "speculative", path)
+
+function _issue_path(issue::SingleIssue)
+    issue.reason == "speculative" && issue.val isa EvaluationPath &&
+        return _path_string(issue.val::EvaluationPath)
+    return issue.path
+end
 
 function _invalidate!(
     result::EvaluationResult,
@@ -133,7 +140,7 @@ function _invalidate!(
 )
     !context.collect_all && !result.valid && return result
     if context.speculative > 0
-        return _invalidate!(result, context, _SPECULATIVE_ISSUE)
+        return _invalidate!(result, context, _speculative_issue(lazy.path))
     end
     return _invalidate!(
         result,
@@ -162,7 +169,7 @@ function _invalidate!(
         issue_count < context.max_issues || throw(
             EvaluationError(
                 context.schema.root,
-                issue.path,
+                _issue_path(issue),
                 "the issue limit was reached",
             ),
         )
@@ -219,7 +226,7 @@ function _absorb!(
             issue_count + length(child_issues) <= context.max_issues || throw(
                 EvaluationError(
                     context.schema.root,
-                    isempty(child_issues) ? "" : first(child_issues).path,
+                    isempty(child_issues) ? "" : _issue_path(first(child_issues)),
                     "the issue limit was reached",
                 ),
             )

Non-blocking

  • Lock on the hot path. _schema_at now takes graph_lock twice per model validation, where the old path was lock-free once the graph existed. Negligible single-threaded, but a serialization point for multi-threaded decoding. Fine as a follow-up: an immutable cache swapped atomically, or precomputing views for all known descriptors when the graph is built.
  • _NO_SCOPE is a shared mutable global. It is only compared, never mutated, so safe today. A one-line comment saying it must never be pushed to would protect the next reader.

With the error-path fix in, this is good to merge.

…ulative branches

The speculative issue sentinel carried an empty path, so hitting max_issues inside an
anyOf/oneOf/not/if/contains branch reported the instance root. Keep the unrendered
EvaluationPath in the sentinel and render it only at the two throw sites.
@jd-lara

jd-lara commented Sep 25, 2026

Copy link
Copy Markdown
Contributor Author

@tanmaykm I applied the fixes and opened an issue about the lock to track later and explore

@tanmaykm
tanmaykm dismissed their stale review September 25, 2026 20:46

Requested change applied in 57a0168.

@tanmaykm tanmaykm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick turnaround. The fix in 57a0168 is exactly what was asked for, and the test covering both throw sites is a nice addition. Verified locally on Julia 1.12.7 and 1.13.0 alongside the green CI. Approving.

@tanmaykm
tanmaykm merged commit f38c958 into JuliaComputing:main Sep 25, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants