Checklist: YSD-22058 (rollback) · drill kind rollback Decision owner
role: the service owner for svc-study-workspace Machinery:
apps/yemaya/svc-study-workspace/src/persistence/migrate.ts
The schema half of backing a change out is already held: every migration says
how to undo its own schema change in its header, and
check-release-integrity.mjs fails the build when one does not. The build
half was not written down at all — whether the previous build can still read
what the new one wrote, how it is put back, and who decides.
That order matters more than it looks, and it is why this is a document rather than a command.
Read this before you undo anything#
The schema and the build come back in opposite orders.
A rollback is safe when the previous build can read the current schema. That is the normal case, because a forward-only migration discipline adds before it removes. So:
- Put the previous build back first. If it reads the current schema, you are done, and you should stop here. Most rollbacks end at this line.
- Only then consider undoing the schema, and only if step 1 failed — because the previous build cannot read what the new migration made. Undoing a migration destroys whatever the new build wrote into the new shape.
⚠️ Reversing that order takes the service down twice. Undo the schema while the new build is still serving and every write it makes fails; the outage is then yours as well as the one you were fixing.
The steps#
| # | Action | Owner role | Expected |
|---|---|---|---|
| 1 | Halt the rollout; stop moving traffic to the new build | on-call engineer | 2 min |
| 2 | Return traffic to the previous build; confirm readiness and error rate | on-call engineer | 10 min |
| 3 | Decide whether the schema must be undone, using its migration header | service owner decides | 10 min |
| 4 | If so, apply the header's undo and confirm the previous build reads it | service owner | 20 min |
Rollback time objective: 55 minutes, from halting the rollout to the
previous build serving reads it used to serve. Set 2026-08-15, and derived
rather than chosen: the four steps are budgeted at 2 + 10 + 10 + 20 = 42, and
the objective is that plus ten minutes of headroom, rounded up to the next five
— the same rule yemaya-study-rollout.md and yemaya-study-recovery.md use.
Step 4 is conditional, and the objective covers the case where it runs. A rollback that step 3 decides needs no schema undo finishes well inside it; that is the easy path, and an objective sized for the easy path is one that fails every time the hard path happens.
Detection#
The same three signals as yemaya-study-rollout.md, plus the one specific to
this direction: the previous build erroring on reads it used to serve, which
means step 2 did not succeed and step 3 is now not optional.
What no rollback returns#
Anything the new build wrote into a shape the old one has no column for. It is not lost — it is in the database — but it is unreadable to the build now serving, and undoing the migration deletes it. Say so before step 4 rather than discovering it after.
Communication#
| Audience | Within |
|---|---|
| The on-call channel — rolling back, and why | immediately |
| Affected tenants, if any write was rejected during the window | 30 min |
Done means#
- The previous build serves, readiness passes, error rate is back to baseline.
study_schema_migrationsmatches what that build expects.- Recorded in the procedure exercise register, with what was lost at step 4 if anything was.
Known shortfall#
migrations-rollout.integration.spec.ts drives the transactional half — a real
migration aborted mid-flight against live PostgreSQL, leaving no record behind.
Nothing exercises putting a previous build back, or the read-compatibility
question at step 1, which is the part of a rollback that actually goes wrong.