Scope#
WS06-010 adds benchmark suites for:
- Geometry quality validation outcomes.
- Texture fidelity validation outcomes.
- Downstream engine ingest success validation outcomes.
Research and Reuse#
Implemented by composing existing validated primitives already in ISIS:
- Geometry checks:
validateMeshQuality(mesh-quality-validator.ts) - Texture checks:
validatePBRTextureCoverage+validateUVAndTexelDensity - Engine ingest checks:
createEngineExportBundles+validateEngineExportBundles
This approach avoids duplicate logic and keeps benchmark scoring coupled to production validators.
Deliverables#
- Added a new benchmark module:
libs/isis/3d-generation/src/processing/three-d-benchmark-suite.ts
- Added benchmark coverage tests:
libs/isis/3d-generation/src/processing/three-d-benchmark-suite.spec.ts
- Exported benchmark APIs through processing/root package exports for reuse:
libs/isis/3d-generation/src/processing/index.tslibs/isis/3d-generation/src/index.ts
Benchmark Suite Capabilities#
- Runs deterministic benchmark datasets across geometry, texture, and engine ingest domains.
- Tracks per-case:
- Validation result
- Expected result
- Expectation accuracy
- Error/warning counts
- Weighted score
- Produces aggregate report with:
- Suite-level expectation accuracy
- Suite-level average score
- Engine ingest success rate
- Cross-suite overall score and threshold pass/fail
Verification Commands#
bash
pnpm --filter @isis/3d-generation exec vitest run src/processing/three-d-benchmark-suite.spec.ts src/processing/three-d-provenance.spec.ts src/processing/mesh-quality-validator.spec.ts src/processing/pbr-texture-validator.spec.ts src/processing/uv-texel-quality-validator.spec.ts src/processing/engine-export-bundles.spec.ts src/processing/dcc-export-bundles.spec.ts src/processing/lod-pipeline-validator.spec.ts src/processing/rig-readiness-validator.spec.ts
pnpm --filter @isis/3d-generation exec tsc -p tsconfig.json --noEmit
pnpm --filter @isis/workflows exec vitest run src/workflows.spec.ts
pnpm --filter @isis/workflows exec tsc -p tsconfig.json --noEmit
pnpm --filter @isis/generation-api exec vitest run src/schemas/job.schema.spec.ts src/workflows/executor.spec.ts
pnpm --filter @isis/output-registry exec vitest run src/schemas/output.schema.spec.ts
pnpm --filter @isis/generation-api exec tsc -p tsconfig.json --noEmit
pnpm --filter @isis/output-registry exec tsc -p tsconfig.json --noEmit
node scripts/isis/validate-v2-compliance-gate.mjs