Eve library

ISIS v2 Benchmark Regression Assertions

Guide

Revision 1

On this page

Generated At: 2026-03-06T11:41:55.313Z Source: scripts/isis/validate-v2-benchmark-regression-gate.mjs

Assertions#

Assertion Metric Actual Rule Threshold Result
Capability workflow pass rate non-regression capability.workflowCapabilityPassRate 1 >= 0.777778 PASS
Capability node-class pass rate non-regression capability.nodeTargetPassRate 0.916667 >= 0.833333 PASS
Missing required workflow capabilities must not increase capability.missingRequiredCapabilitiesCount 0 <= 4 PASS
Failed required node targets must not increase capability.failedRequiredNodeTargetsCount 1 <= 2 PASS
UX required capability pass rate non-regression ux.requiredCapabilityPassRate 1 >= 1 PASS
UX workflow pass rate non-regression ux.workflowPassRate 1 >= 1 PASS
UX passed workflow count non-regression ux.workflowsPassed 48 >= 48 PASS
Missing required UX capabilities must not increase ux.missingRequiredCapabilityCount 0 <= 0 PASS
Missing UX capability bucket count must not increase ux.missingCapabilityBuckets 0 <= 0 PASS
Taxonomy category coverage non-regression ux.taxonomyCategoryCoverage 1 >= 1 PASS
Control-mode average quality score non-regression control.averageQualityScore 0.972039 >= 0.251944 PASS
Failing control-mode count must not increase control.failingControlCount 7 <= 27 PASS
High-severity control failures must not increase control.highSeverityFailureCount 0 <= 27 PASS
Control modes with no workflow coverage must not increase control.noWorkflowCoverageCount 0 <= 5 PASS

Gate Result#

  • Status: PASS