A reference implementation of CRUS-T score: a normalized, programmatically computable runtime reliability score for agentic AI, from
Satyaki Ghosh Dastidar, Proportionate Governance in Practice: An IDEA–CRUS-T Framework for Risk-Tiered, Runtime-Governed AI Systems in Financial Services. Preprint, 2026.
Maintained by AI Risk Consulting.
CRUS-T scores an AI capability or execution trajectory on four dimensions — Consistency, Robustness, Uncertainty awareness, and Safety — each built from three standard, auditable sub-metrics, aggregated with a collapse-sensitive weighted geometric mean, and gated on a conservative statistical bound rather than a point estimate. Safety additionally carries a discontinuous hard gate: a single critical policy violation zeroes the governed safety score regardless of the continuous score, so it can never be averaged away.
This library implements the score itself (Section 5 of the paper) plus a thin tier-gating layer on top of it (Section 5.6–5.7) that decides continue/escalate/contain against a threshold vector you supply. It does not implement the paper's separate five-dimension risk-tiering method (Section 4 — Authority/Impact/Agency/Exposure/Recoverability → control tier); that assigns a design-time control tier, whereas this library is about the runtime reliability score that verifies a tier's assumptions continue to hold in production.
Early (0.1.0). The math is unit-tested against the paper's equations and
against the worked example in Appendix C. The weights, tolerances, and
thresholds you configure are governance parameters, not statistical
defaults — see Governance parameters below
before using this for anything with real consequence.
pip install -e . # from a local checkout
pip install -e ".[dev]" # + pytest, for running the test suiteRequires Python 3.10+. No required runtime dependencies.
| Paper section | Module |
|---|---|
| 5.1 Design principles and normalization primitives | crus_t.primitives |
| 5.2 Consistency (C1/C2/C3) | crus_t.metrics.consistency |
| 5.3 Robustness (R1/R2/R3) | crus_t.metrics.robustness |
| 5.4 Uncertainty awareness (U1/U2/U3) | crus_t.metrics.uncertainty |
| 5.5 Safety (S1/S2/S3) + critical-violation hard gate | crus_t.metrics.safety |
| Eqs. 8/12/16/20-21 — dimension aggregation | crus_t.dimensions |
| Eq. 22 — composite score | crus_t.score |
| Eq. 24 — bootstrap lower-confidence-bound gating | crus_t.bootstrap |
| 5.6-5.7 — tier gate, runtime zones (continue/escalate/contain) | crus_t.gating |
| Per-run scoring | crus_t.run |
| Rolling many runs up into a batch/portfolio view | crus_t.aggregate |
| Governance-owned weights/tolerances | crus_t.config |
Every public function's docstring cites the specific equation it implements, so you can go from a number in the paper straight to the code that produces it (and back).
Several CRUS-T sub-metrics depend on a learned component the paper
deliberately keeps out of the governed math: an entailment model for
groundedness (S1), an embedding or entailment model for semantic
similarity (C1/C2), a task-quality function Q(.) for Robustness. This
library takes the outputs of those components (an entailment boolean, a
similarity score, a quality score) as plain function arguments — it does
not call any model itself. Section 5.5 of the paper is explicit about why:
"a probabilistic verifier operating in the same semantic space as the
system it polices is architecturally blind to fluent-but-wrong outputs,"
so whatever model you plug in needs to be validated and governed
separately, on its own precision/recall for the violation class it's
detecting.
from crus_t import (
compute_consistency, compute_robustness, compute_uncertainty,
governed_safety, PolicyObservation, safety_to_dimension_score,
ReliabilityVector, composite,
ThresholdVector, evaluate_gate,
make_run_score, aggregate_run_scores,
)
# 1. Compute each dimension from its three sub-metrics (already observed
# from your evaluation harness / production telemetry).
c = compute_consistency(c1=0.94, c2=0.88, c3=0.97)
r = compute_robustness(r1=0.85, r2=0.96, r3=0.80)
u = compute_uncertainty(u1=0.89, u2=0.81, u3=0.85)
safety_result = governed_safety(
entailed_claims=98, total_claims=100,
policy_observations=[PolicyObservation("no_stale_citation", severity=1.0, violation_count=0)],
n_trajectories=100,
authorized_calls=10, total_external_calls=10,
critical_violation_observed=False,
tau_safety=0.10,
)
s = safety_to_dimension_score(safety_result)
vector = ReliabilityVector(c, r, u, s)
print(vector.as_dict()) # {"C": 0.93, "R": 0.87, "U": 0.85, "S": ...}
print(composite(vector)) # single reporting number, Eq. (22)
# 2. Package it as a run and gate it against a tier's thresholds. Threshold
# values are always your own governance input -- this library has no
# built-in per-tier table.
thresholds = ThresholdVector(consistency=0.80, robustness=0.75, uncertainty=0.78, safety=0.90)
gate = evaluate_gate(vector.as_dict(), thresholds)
run = make_run_score(run_id="run-001", capability_id="drafting", vector=vector, gate=gate)
print(gate.zone) # RuntimeZone.CONTINUE / ESCALATE / CONTAIN
# 3. Later, roll many runs up into a batch view.
agg = aggregate_run_scores([run, ...])
print(agg.escalation_rate, agg.containment_rate, agg.axis_summaries["safety"].mean)For a regulatory-grade decision, gate on a bootstrap lower bound instead of the point estimate (Eq. 24):
from crus_t import bootstrap_lower_bound
# `statistic` recomputes a dimension score from a resampled item list --
# wire this to your own evaluation-set records.
result = bootstrap_lower_bound(evaluation_items, statistic=my_dimension_statistic, seed=42)
gate = evaluate_gate({"C": result.lower_bound, ...}, thresholds)See examples/worked_example.py for a runnable version of Appendix C's
Drafting-capability example, and examples/quickstart.py for the snippet
above.
Every tolerance (tau), every sub-metric weight, every threshold vector,
and the bootstrap confidence level are, per Section 5.1 of the paper,
governance parameters: "set by the second line of defense from pooled
validation evidence, versioned, and subject to challenge — not free
parameters tuned to produce a desired outcome." crus_t.config.GovernanceConfig
gives them one explicit, serializable home; ThresholdVector in
crus_t.gating is a plain, explicit input with no built-in defaults. This
library will not stop you from tuning a threshold until a report looks
good — that governance discipline lives with your process, not the code.
pip install -e ".[dev]"
pytest # run the test suite
pytest --cov=crus_t # with coverageIf you use CRUS-T in published work or an internal governance standard:
@misc{ghoshdastidar2026crust,
author = {Ghosh Dastidar, Satyaki},
title = {Proportionate Governance in Practice: An {IDEA}--{CRUS-T}
Framework for Risk-Tiered, Runtime-Governed {AI} Systems
in Financial Services},
year = {2026},
note = {Preprint},
howpublished = {\url{https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7075038}}
}CRUS-T is developed and maintained by Satyaki Ghosh Dastidar at AI Risk Consulting, which works with banks, fintechs and other regulated institutions on AI and model risk governance.
The score is also under review as a submission to the NIST AI Metrology Center.
Questions about applying this in a regulated environment: info@airisk-consulting.com
MIT. See LICENSE.