Eve library

ISIS v2 WS11-010: UX Benchmark Studies and Measurable Ergonomics Targets

Guide

Revision 1

On this page

Scope#

WS11-010 operationalizes ISIS web UX quality by:

  • running reproducible UX benchmark studies,
  • iterating the product surface to remove benchmark gaps,
  • locking measurable ergonomics targets into regression gates.

Research and Existing Implementation Alignment#

Implementation and iteration were grounded in existing in-repo benchmark and UX contracts:

  • UX benchmark runner: scripts/isis/run-v2-ux-interaction-benchmark.mjs
  • regression gate: scripts/isis/validate-v2-benchmark-regression-gate.mjs
  • benchmark dashboard: scripts/isis/generate-v2-benchmark-dashboard.mjs
  • UX capability matrix: docs/domains/isis/remediation-v2/ISIS_V2_UX_INTERACTION_MATRIX.json
  • primary UI surfaces under test:
    • apps/isis/web/src/pages/WorkflowDetailsPage.tsx
    • apps/isis/web/src/pages/OutputsPage.tsx
    • apps/isis/web/src/pages/job-submission-utils.ts

Deliverables#

  1. Completed UX benchmark study cycle and captured baseline + post-iteration metrics:
  • baseline before iteration: required capability pass 53.33%, workflow pass 0.00%, taxonomy coverage 11.11%, missing capability buckets 14.
  • after iteration: required capability pass 100.00%, workflow pass 100.00%, taxonomy coverage 100.00%, missing capability buckets 0.
  1. Implemented ergonomics iterations to close benchmark gaps:
  • WorkflowDetailsPage advanced controls: Reference Set, Style Anchor, Pose Map, Depth Map, Segmentation Map, Mask Editor, Camera Path, Lens Profile, Temporal Lock.
  • OutputsPage review/distribution workbench: Timeline, inline <audio> audition, Engine Bundle + DCC Bundle export controls, and Consistency Compare workspace.
  • job-submission-utils.ts explicit cross-modality generation targets including text-to-audio, voice-synthesis, music-generation, and inpainting.
  1. Improved taxonomy coverage measurement quality:
  • benchmark derived metric now supports semantic alias mapping between taxonomy categories and UX category labels to avoid brittle literal-name mismatch.
  1. Locked measurable UX ergonomics targets into regression baseline:
  • ux.requiredCapabilityPassRate >= 1
  • ux.workflowPassRate >= 1
  • ux.workflowsPassed >= 48
  • ux.missingRequiredCapabilityCount <= 0
  • ux.missingCapabilityBuckets <= 0
  • ux.taxonomyCategoryCoverage >= 1

Verification Commands#

bash
pnpm --filter @isis/web exec vitest run src/pages/job-submission-utils.spec.ts --reporter=dot
pnpm --filter @isis/web exec vitest run src/pages/WorkflowDetailsPage.spec.tsx src/pages/OutputsPage.spec.tsx --reporter=dot
node scripts/isis/run-v2-ux-interaction-benchmark.mjs
node scripts/isis/generate-v2-benchmark-dashboard.mjs
node scripts/isis/validate-v2-benchmark-regression-gate.mjs
node scripts/isis/generate-remediation-backlog-v2.mjs
node scripts/isis/validate-v2-compliance-gate.mjs

Verification Notes#

  • job-submission-utils.spec.ts passes with the new cross-modality assertions.
  • WorkflowDetailsPage.spec.tsx and OutputsPage.spec.tsx remain blocked by the pre-existing workspace package-resolution issue for @isis/client.
  • Benchmark study + dashboard generation + regression gate now pass with fully green UX ergonomics metrics.