Eve library

ISIS v2 Control-Mode Quality Benchmark

Guide

Revision 1

On this page

Generated At: 2026-03-06T11:07:55.343Z Source: scripts/isis/run-v2-control-mode-quality-benchmark.mjs

Summary#

  • Controls analyzed: 38
  • Required controls analyzed: 26
  • Passed controls: 31
  • Failing controls: 7
  • Overall control pass rate: 81.58%
  • Required control pass rate: 84.62%
  • Average quality score: 0.9720
  • High-severity control gaps: 0

Failure Taxonomy#

Class Severity Description Impacted Controls
no-workflow-coverage critical Control modality is defined in taxonomy but not used by any workflow. 0
ux-capability-gap high Control-specific UX capability mappings are missing or failing benchmark probes. 0
workflow-quality-gap high One or more workflows using this control fail required UX quality coverage checks. 0
modality-coverage-gap medium Observed workflow modality coverage for the control does not match declared supported modalities. 7
benchmark-score-gap medium Composite benchmark score is below pass threshold (0.75). 0

Per-Control Benchmark#

Control Required Workflows Workflow Pass Capability Pass Modality Coverage Score Result Failures
controlnet-inpaint no 2 100.00% 100.00% 33.33% 0.9000 FAIL modality-coverage-gap
faceid yes 2 100.00% 100.00% 33.33% 0.9000 FAIL modality-coverage-gap
instantid yes 1 100.00% 100.00% 33.33% 0.9000 FAIL modality-coverage-gap
ipadapter-composition no 1 100.00% 100.00% 33.33% 0.9000 FAIL modality-coverage-gap
ipadapter-face yes 5 100.00% 100.00% 33.33% 0.9000 FAIL modality-coverage-gap
ipadapter-style yes 2 100.00% 100.00% 33.33% 0.9000 FAIL modality-coverage-gap
lens-profile-lock no 4 100.00% 100.00% 50.00% 0.9250 FAIL modality-coverage-gap
controlnet-canny yes 1 100.00% 100.00% 66.67% 0.9500 PASS -
controlnet-lineart yes 1 100.00% 100.00% 66.67% 0.9500 PASS -
controlnet-scribble no 1 100.00% 100.00% 66.67% 0.9500 PASS -
controlnet-softedge no 1 100.00% 100.00% 66.67% 0.9500 PASS -
controlnet-tile no 3 100.00% 100.00% 66.67% 0.9500 PASS -
keyframe-anchor-lock no 7 100.00% 100.00% 66.67% 0.9500 PASS -
openpose-sequence yes 1 100.00% 100.00% 66.67% 0.9500 PASS -
identity-lock yes 7 100.00% 100.00% 75.00% 0.9625 PASS -
camera-lock yes 26 100.00% 100.00% 100.00% 1.0000 PASS -
camera-path-guide yes 7 100.00% 100.00% 100.00% 1.0000 PASS -
color-script-lock no 42 100.00% 100.00% 100.00% 1.0000 PASS -
controlnet-depth yes 6 100.00% 100.00% 100.00% 1.0000 PASS -
controlnet-normal yes 5 100.00% 100.00% 100.00% 1.0000 PASS -
controlnet-pose yes 2 100.00% 100.00% 100.00% 1.0000 PASS -
controlnet-segmentation yes 3 100.00% 100.00% 100.00% 1.0000 PASS -
depth-constraint yes 26 100.00% 100.00% 100.00% 1.0000 PASS -
identity-adapter yes 26 100.00% 100.00% 100.00% 1.0000 PASS -
latent-reuse yes 26 100.00% 100.00% 100.00% 1.0000 PASS -
model-lock no 48 100.00% 100.00% 100.00% 1.0000 PASS -
motion-vector-lock no 7 100.00% 100.00% 100.00% 1.0000 PASS -
palette-lock yes 26 100.00% 100.00% 100.00% 1.0000 PASS -
pose-constraint yes 26 100.00% 100.00% 100.00% 1.0000 PASS -
prompt-template-lock no 46 100.00% 100.00% 100.00% 1.0000 PASS -
reference-ensemble yes 26 100.00% 100.00% 100.00% 1.0000 PASS -
sampler-scheduler-lock no 48 100.00% 100.00% 100.00% 1.0000 PASS -
scene-lock yes 26 100.00% 100.00% 100.00% 1.0000 PASS -
seed-lock yes 48 100.00% 100.00% 100.00% 1.0000 PASS -
style-anchor yes 26 100.00% 100.00% 100.00% 1.0000 PASS -
style-lock yes 46 100.00% 100.00% 100.00% 1.0000 PASS -
temporal-lock yes 27 100.00% 100.00% 100.00% 1.0000 PASS -
wardrobe-lock yes 26 100.00% 100.00% 100.00% 1.0000 PASS -

Top Failing Controls#

  • controlnet-inpaint: score 0.9000, failures modality-coverage-gap, missing capabilities -
  • faceid: score 0.9000, failures modality-coverage-gap, missing capabilities -
  • instantid: score 0.9000, failures modality-coverage-gap, missing capabilities -
  • ipadapter-composition: score 0.9000, failures modality-coverage-gap, missing capabilities -
  • ipadapter-face: score 0.9000, failures modality-coverage-gap, missing capabilities -
  • ipadapter-style: score 0.9000, failures modality-coverage-gap, missing capabilities -
  • lens-profile-lock: score 0.9250, failures modality-coverage-gap, missing capabilities -

Traceability#

  • Checklist item: WS05-009
  • Inputs: ISIS_V2_CONTROL_TAXONOMY.json, workflow-taxonomy-registry.json, ISIS_V2_UX_INTERACTION_BENCHMARK.json, ISIS_V2_UX_INTERACTION_MATRIX.json