Eve library

ISIS Prop Pack General Regression Tests and Golden Datasets Runbook

Runbook

Revision 1

On this page

Last Updated: 2026-02-28
Scope: MXA-0232 regression-test and golden-dataset coverage for prop-pack-general workflows.

Purpose#

Generate and validate the canonical artifact that captures deterministic golden datasets, regression drift scenarios, and suite-level acceptance thresholds for prop-pack-general workflow quality and packaging stability.

Artifacts#

  • docs/domains/isis/remediation-v2/ISIS_V2_PROP_PACK_GENERAL_REGRESSION_TESTS_GOLDEN_DATASETS.json
  • docs/domains/isis/remediation-v2/ISIS_V2_PROP_PACK_GENERAL_REGRESSION_TESTS_GOLDEN_DATASETS.md

Generator#

bash
node scripts/isis/generate-v2-prop-pack-general-regression-tests-golden-datasets.mjs

Options:

  • --report-date=YYYY-MM-DD (default: current UTC date)

Validator#

bash
node scripts/isis/validate-v2-prop-pack-general-regression-tests-golden-datasets.mjs

Source Inputs#

  • docs/domains/isis/remediation-v2/ISIS_REMEDIATION_V2_TICKETS.json
  • docs/domains/isis/remediation-v2/ISIS_V2_PROP_PACK_GENERAL_REQUIREMENTS_BRIEF.json
  • docs/domains/isis/remediation-v2/ISIS_V2_PROP_PACK_GENERAL_QUALITY_EVALUATOR_ACCEPTANCE_GATE.json
  • apps/isis/generation-api/src/routes/prop-pack-general-regression-golden-dataset.ts
  • apps/isis/generation-api/src/routes/prop-pack-general-regression-golden-dataset.spec.ts
  • apps/isis/generation-api/src/routes/prop-pack-general-quality-gate-utils.ts
  • apps/isis/generation-api/src/schemas/prop-pack-general.schema.ts

Operational Cadence#

  1. Regenerate when workflow parameters, quality thresholds, or output-package contract requirements change.
  2. Validate before release-candidate sign-off for prop-pack-general quality stability.
  3. Keep golden-case coverage and regression-case diagnostics synchronized with quality-gate and output-package contract policies.
  • apps/isis/generation-api/src/routes/prop-pack-general-regression-golden-dataset.ts
  • apps/isis/generation-api/src/routes/prop-pack-general-quality-gate-utils.ts
  • apps/isis/generation-api/src/schemas/prop-pack-general.schema.ts
  • docs/domains/isis/remediation-v2/ISIS_REMEDIATION_V2_BACKLOG.md