Eve library

ISIS NPC Crowd Variation Pack Regression Tests and Golden Datasets Runbook

Runbook

Revision 1

On this page

Last Updated: 2026-02-27
Scope: MXA-0106 regression-test and golden-dataset coverage for npc-crowd-variation-pack workflows.

Purpose#

Generate and validate the canonical artifact that captures deterministic golden datasets, regression drift scenarios, and suite-level acceptance thresholds for npc-crowd-variation-pack workflow quality and packaging stability.

Artifacts#

  • docs/domains/isis/remediation-v2/ISIS_V2_NPC_CROWD_VARIATION_PACK_REGRESSION_TESTS_GOLDEN_DATASETS.json
  • docs/domains/isis/remediation-v2/ISIS_V2_NPC_CROWD_VARIATION_PACK_REGRESSION_TESTS_GOLDEN_DATASETS.md

Generator#

bash
node scripts/isis/generate-v2-npc-crowd-variation-pack-regression-tests-golden-datasets.mjs

Options:

  • --report-date=YYYY-MM-DD (default: current UTC date)

Validator#

bash
node scripts/isis/validate-v2-npc-crowd-variation-pack-regression-tests-golden-datasets.mjs

Source Inputs#

  • docs/domains/isis/remediation-v2/ISIS_REMEDIATION_V2_TICKETS.json
  • docs/domains/isis/remediation-v2/ISIS_V2_NPC_CROWD_VARIATION_PACK_REQUIREMENTS_BRIEF.json
  • docs/domains/isis/remediation-v2/ISIS_V2_NPC_CROWD_VARIATION_PACK_QUALITY_EVALUATOR_ACCEPTANCE_GATE.json
  • apps/isis/generation-api/src/routes/npc-crowd-variation-pack-regression-golden-dataset.ts
  • apps/isis/generation-api/src/routes/npc-crowd-variation-pack-regression-golden-dataset.spec.ts
  • apps/isis/generation-api/src/routes/npc-crowd-variation-pack-quality-gate-utils.ts
  • apps/isis/generation-api/src/schemas/npc-crowd-variation-pack.schema.ts

Operational Cadence#

  1. Regenerate when workflow parameters, quality thresholds, or output-package contract requirements change.
  2. Validate before release-candidate sign-off for npc-crowd-variation-pack quality stability.
  3. Keep golden-case coverage and regression-case diagnostics synchronized with quality-gate and output-package contract policies.
  • apps/isis/generation-api/src/routes/npc-crowd-variation-pack-regression-golden-dataset.ts
  • apps/isis/generation-api/src/routes/npc-crowd-variation-pack-quality-gate-utils.ts
  • apps/isis/generation-api/src/schemas/npc-crowd-variation-pack.schema.ts
  • docs/domains/isis/remediation-v2/ISIS_REMEDIATION_V2_BACKLOG.md