Eve library

The two pinned QTI 3 validators (M10.5.d / M1.11.g)

Other

Revision 1

On this page

Neither validator is ours, and neither is committed here. What is committed is enough to install exactly the ones M1.11's contract pins, and to fail loudly if what arrives is not them.

Validator What it is Pinned as
1edtech-qti3-xsd 1EdTech's own QTI 3.0.1 XSDs, run through xmlschema xsd-lock.json — 21 URLs with a SHA-256 each
longsight-qti3-cli @longsightgroup/qti3-cli, an independent QTI 3 consumer package.json + package-lock.json at 0.10.3

The versions and integrity pins in evidence/v1-workbenches/metis-qti-package.json (profile.validatorToolchain) are the authority; a receipt that does not match them fails tests/test_qti3_conformance.py.

Install#

sh
cd apps/metis/service
python3 -m venv tools/qti3/.venv
tools/qti3/.venv/bin/pip install -r tools/qti3/requirements.txt
tools/qti3/.venv/bin/python tools/qti3/fetch_xsd.py     # ~31 MB into tools/qti3/cache/
npm --prefix tools/qti3 install

cache/, node_modules/ and .venv/ are gitignored. fetch_xsd.py keeps a downloaded schema only if its SHA-256 is the one xsd-lock.json names, so a schema that changed under its URL stops the run instead of quietly changing what "valid" means.

Produce the receipts#

sh
PYTHONPATH=src:tests .venv/bin/python scripts/export_qti_conformance.py

One package per assessment kind the runtime exports, both validators over each, their output retained verbatim under evidence/v1-workbenches/metis-conformance/<validator>/.

Two things worth knowing before trusting a green run#

The consumer can exit 0 without running. Its entry module guards on import.meta.url === `file://${process.argv[1]}` , so invoked through node_modules/.bin/qti3 — or through any symlinked path — it does nothing, prints nothing, and exits 0. That silence is indistinguishable from a pass at a glance. The runner invokes the resolved dist/index.js and refuses to write a receipt for a report that carries no JSON verdict (measured 2026-09-11, after it fooled the first run).

xmlschema will fetch a missing schema over the network. With a permissive URI mapper, a schema outside the cache is silently downloaded, so the closure can look complete when it is not — xlink.xsd was reached that way for weeks, and only surfaced when the mapper was made to refuse. validate_xsd.py now raises on any uncached remote URI, which makes the lockfile the whole truth about what the documents were validated against.