Eve library

ISIS Storyboard Frame Set Regression Tests and Golden Datasets Runbook

Runbook

Revision 1

On this page

Last Updated: 2026-03-02
Scope: MXA-0448 regression-test and golden-dataset coverage for storyboard-frame-set workflows.

Purpose#

Generate and validate the canonical artifact that captures deterministic golden storyboard-frame-set datasets, regression drift scenarios, and suite-level acceptance thresholds for workflow quality and packaging stability.

Artifacts#

  • docs/domains/isis/remediation-v2/ISIS_V2_STORYBOARD_FRAME_SET_REGRESSION_TESTS_GOLDEN_DATASETS.json
  • docs/domains/isis/remediation-v2/ISIS_V2_STORYBOARD_FRAME_SET_REGRESSION_TESTS_GOLDEN_DATASETS.md

Generator#

bash
node scripts/isis/generate-v2-storyboard-frame-set-regression-tests-golden-datasets.mjs

Options:

  • --report-date=YYYY-MM-DD (default: current UTC date)

Validator#

bash
node scripts/isis/validate-v2-storyboard-frame-set-regression-tests-golden-datasets.mjs

Source Inputs#

  • docs/domains/isis/remediation-v2/ISIS_REMEDIATION_V2_TICKETS.json
  • docs/domains/isis/remediation-v2/ISIS_V2_STORYBOARD_FRAME_SET_REQUIREMENTS_BRIEF.json
  • docs/domains/isis/remediation-v2/ISIS_V2_STORYBOARD_FRAME_SET_QUALITY_EVALUATOR_ACCEPTANCE_GATE.json
  • apps/isis/generation-api/src/routes/storyboard-frame-set-regression-golden-dataset.ts
  • apps/isis/generation-api/src/routes/storyboard-frame-set-regression-golden-dataset.spec.ts
  • apps/isis/generation-api/src/routes/storyboard-frame-set-quality-gate-utils.ts
  • apps/isis/generation-api/src/schemas/storyboard-frame-set.schema.ts

Operational Cadence#

  1. Regenerate when workflow parameters, quality thresholds, or output-package contract requirements change.
  2. Validate before release-candidate sign-off for storyboard-frame-set quality stability.
  3. Keep golden-case coverage and regression-case diagnostics synchronized with quality-gate and output-package contract policies.
  • apps/isis/generation-api/src/routes/storyboard-frame-set-regression-golden-dataset.ts
  • apps/isis/generation-api/src/routes/storyboard-frame-set-quality-gate-utils.ts
  • apps/isis/generation-api/src/schemas/storyboard-frame-set.schema.ts
  • docs/domains/isis/remediation-v2/ISIS_REMEDIATION_V2_BACKLOG.md