Date: 2026-02-22
Checklist Item: MIG-003
Ticket: ISIS-0172
Goal#
Backfill missing provenance records for historical completed outputs so every
eligible output has a valid outputs.provenance_id reference and a
corresponding row in provenance.
Scope#
Eligible outputs are records in outputs with status in:
availablearchivedexpired
A record is considered missing provenance when either:
outputs.provenance_idis null, oroutputs.provenance_idpoints to a non-existent provenance row.
Backfill Strategy#
For each eligible output, the backfill pipeline performs:
- Existing provenance reconciliation:
- If a provenance row already exists by
provenance.output_id, relinkoutputs.provenance_idto that row. - If multiple provenance rows exist for one output, mark unresolved for manual review.
- If a provenance row already exists by
- Provenance reconstruction when no row exists:
- Build
RegisterProvenanceRequestfrom output metadata and canonical fields. - Insert provenance and update
outputs.provenance_idtransactionally.
- Build
Source Resolution Rules#
Reconstruction uses deterministic source resolution:
metadata.provenance(highest fidelity)- Output-level workflow/model fields and metadata aliases
- Parameter maps (
metadata.parameters,inputParameters,generationParameters) - Input reconstruction from
metadata.inputs,metadata.inputUrl, andoutput.parentIds - Fallback defaults from output record (
createdAt,jobId,workflowId)
Operational Interfaces#
API (admin)#
POST /api/v1/outputs/admin/provenance/backfill
Request body:
json
{
"limit": 2000,
"dryRun": false
}
Access requirements:
- Bearer auth role:
admin,ops, orsre - API key permission:
outputs:admin
CLI#
isis outputs backfill-provenance --dry-run --limit 5000
Result Contract#
Backfill response includes:
scannedmissingProvenancebackfilledunresolvedfailedbackfilledOutputIdsunresolvedOutputIdsfailedOutputIdssourceCountscheckedAt
Safety Controls#
- Dry-run mode for impact estimation without writes.
- Bounded scan with hard max limit (
50000) and page batching. - Ambiguous provenance conflicts are not auto-resolved; flagged as unresolved.
- Audit actor attribution is preserved through
actorIdfrom authenticated principal.