Date: 2026-02-22
Checklist Item: OPS-001
Ticket: ISIS-0236
Scope#
Operational resilience coverage for Isis data persistence:
- Backup procedures (automated + manual + logical).
- Restore procedures (PITR + snapshot).
- Explicit recovery objectives (RTO/RPO) for Isis data stores.
Evidence Matrix#
| Requirement | Evidence | Status |
|---|---|---|
| Backup runbook exists with executable steps | docs/domains/isis/runbooks/database-backup-restore.md defines daily backup verification, on-demand manual snapshots, logical backup commands, and backup health gates. |
Pass |
| Restore runbook exists with deterministic flow | Same runbook defines restore strategy decision tree, isolated restore target procedure, validation, and cutover sequence for Isis APIs. | Pass |
| Recovery objectives are explicit and measurable | Same runbook includes source-of-truth RPO/RTO targets for Isis PostgreSQL metadata and artifact stores, plus measurement policy (incident declaration -> ready validation). |
Pass |
| Restore validation automation exists | scripts/isis/verify_database_restore.sh generates machine-readable restore verification report with table-count thresholds and integrity checks, supports strict mode for CI/drill gating. |
Pass |
| Post-restore reporting is codified | Runbook includes RTO/RPO reporting checklist and required incident fields (restore_start_time, restore_validation_pass_time, rto_minutes, estimated_rpo_minutes, objective_met). |
Pass |
Reproduction Commands#
bash
RESTORE_DATABASE_URL="postgres://..." STRICT=true scripts/isis/verify_database_restore.sh