# Runbook — Watermark Verification Failure

> Watermarking signature or recovery against a generated asset failed
> verification. Use to block asset promotion, run watermark recovery, and route
> the exception to rights and provenance review.

## Scope

Watermarking is applied to Isis-generated assets (image, video, audio, 3D) and
to selected high-trust Sophia evidence captures. The watermark is distinct from
the C2PA / content-credentials manifest (covered by `provenance-failure.md`) —
the watermark proves the asset bytes themselves were emitted by Oshun, even when
stripped of metadata.

Failure modes:

- Encoder failed to embed the watermark in a generated asset.
- Verifier cannot extract a watermark from an asset that should have one.
- Watermark extracted but does not match the expected payload (signature
  mismatch, payload tampering).
- Watermark survives a transformation it should not (false positive) or fails a
  transformation it should survive (false negative on a known benchmark
  transform).
- Recovery pipeline (against scraped/redistributed media) fails to detect a
  known emitted watermark.

## Severity classification

| Sev  | Condition                                                                                                                      |
| ---- | ------------------------------------------------------------------------------------------------------------------------------ |
| sev1 | Watermark encoder is silently no-op — assets are emitted unwatermarked while telemetry reports success.                        |
| sev1 | Verifier accepts a forged or tampered payload as valid (signature compromise).                                                 |
| sev2 | Encoder failure rate > 1% sustained; recovery pipeline false-negative rate > 5%; verifier false-positive rate above threshold. |
| sev3 | Single asset rejected; no promotion has occurred.                                                                              |

Source events: `isis.watermark.encoded`, `isis.watermark.encode_failed`,
`isis.watermark.verified`, `isis.watermark.verification_failed`,
`isis.watermark.recovered`.

## Detection signals

- Alerts: watermark-pipeline panels on `dashboards-service-health.ts` — encode
  success rate, verify success rate, recovery hit/miss against the scheduled
  audit set.
- Benchmark: scheduled `evaluation-manifest.ts` watermark scope runs daily
  against a fixture set covering compression, crop, rotation, color shift, audio
  re-encode, and video re-encode. A regression past threshold fires a sev2.
- Customer signals: spike in `media.share.watermark_dispute_filed` (a third
  party challenges the asset's origin).
- Operator signals: rights review queue flags a watermark dispute.

## Initial triage (first 15 minutes)

1. Acknowledge `oshun-isis-oncall` (and `oshun-compliance-oncall` for sev1
   signature compromise).
2. **Block asset promotion immediately**: enable the promotion-freeze flag for
   the affected asset class. The freeze emits `studio.asset.promotion_frozen`.
   Already-promoted assets are not recalled; new promotions are paused.
3. Determine failure boundary:
   - Encoder fault → suspect the most recent Isis workflow promotion; delegate
     to `model-workflow-rollback.md`.
   - Verifier fault → suspect verifier-version drift or signing-key drift.
   - Recovery fault → suspect detector drift or fixture drift.
   - Forged payload accepted → escalate to security and treat as potential
     signing-key compromise.
4. Page rights lead if the affected assets are subject to a rights agreement
   (e.g., licensed sources).

## Diagnosis

1. **Encoder fault**: re-run the encode for 20 sample assets and verify the
   watermark extract round-trip. If the encoder emits a payload that the
   verifier rejects, the encoder/verifier pair is misaligned — verify both are
   on the same certified version.
2. **Verifier fault**: pull 50 known-good emitted assets (last 7 days) and run
   them through the verifier. A drop in extract rate implicates the verifier.
   Compare verifier configuration (keys, parameters) to the release-taxonomy
   certified row.
3. **Recovery fault**: run the recovery pipeline against the daily fixture set.
   If the per-transform recovery rate is below threshold, the detector
   regressed. Compare against the last certified detector version in
   release-taxonomy.
4. **Forged payload**: confirm the payload's claimed signing key versus the
   active key set. If the signature validates against a key not in the active
   set, the key is compromised or the rotation history is corrupted — treat as
   security incident.
5. **Customer dispute**: cross-reference the disputed asset's payload against
   the Oshun emission registry. If no emission record exists, the asset is not
   Oshun-origin and the dispute is dismissed with a public response template; if
   a record exists, follow the recovery-and-attribute path.

## Mitigation

1. **Block asset promotion**: already done in triage.
2. **Watermark recovery**: enqueue `isis.watermark.recovery.requested` for the
   affected asset cohort — the recovery pipeline re-extracts payloads from the
   asset bytes and compares against the emission registry. Mismatches are
   quarantined.
3. **Encoder rollback**: if the encoder is the failure boundary, follow
   `model-workflow-rollback.md` for the encoder workflow target. Already-emitted
   assets remain emitted; their inner watermark is verified against the encoder
   version that emitted them.
4. **Verifier rollback or forward-fix**: verifier regressions should be
   forward-fixed where possible (an updated verifier sees more transforms, not
   fewer). If the regression is a misaligned key set, re-publish the certified
   key set to all verifier nodes; do not rotate keys during the incident.
5. **Signing-key compromise (sev1)**: do not roll back the rotation — issue a
   new signing key and publish the revocation of the compromised key. Coordinate
   with security; treat the compromised window's assets as not-attestable until
   re-attested under the new key.
6. **Customer-visible mode**: while frozen, customer "verify origin" tooling
   renders `media.share.verification_unavailable_shown` instead of a hard error.
   Recovery against externally-redistributed media continues to operate against
   the recovered emission registry.

## Communication cadence

| Phase          | Template ID                                 | Audience          | Cadence                              |
| -------------- | ------------------------------------------- | ----------------- | ------------------------------------ |
| Internal       | `incident.internal.slack.investigating`     | internal          | within 5 min                         |
| Identified     | `incident.customer.status_page.identified`  | status_page       | for customer-visible blocks          |
| Rights/partner | `incident.partner.email.identified`         | partner           | if external rights affected          |
| Regulator      | `incident.regulator.email.resolved` (adapt) | regulator         | only for sev1 signing-key compromise |
| Resolved       | `incident.customer.status_page.resolved`    | status_page       | within 60 min of unblock             |
| Postmortem     | `incident.customer.status_page.postmortem`  | status_page/legal | within 5 biz day                     |

Customer dispute responses use the rights/legal-co-authored template; do not
improvise.

## Escalation

- Isis lead is incident commander for encoder/verifier/recovery failures.
- Security lead is incident commander for signing-key compromise.
- For sev1 forged-payload acceptance, page security, compliance, and legal at
  minute 0.
- Standard 5/5/15/30/60 escalation otherwise.

## Recovery verification

- Encoder success rate ≥ 99.9% for 60 min.
- Verifier extract rate ≥ 99% over a fresh 50-asset known-good sample.
- Recovery pipeline passes the daily benchmark set above threshold.
- Promotion-freeze lifted after a 50-asset smoke check.
- For signing-key compromise: revocation propagated, new key published,
  re-attestation queue cleared (or assets explicitly marked not-attestable with
  public-facing copy approved by legal).
- No new `watermark_dispute_filed` for 7 days post-resolution.

## Post-incident

- Postmortem must include: which side regressed (encoder/verifier/recovery/
  key), the affected emission window, the recovery outcomes, the asset cohort
  with explicit attribution outcomes, and any rights/legal notifications.
- Update the watermark benchmark fixtures to include the failure transform or
  attack vector.
- If a key compromise occurred, file a key-rotation policy review and a HSM
  access-audit review.
- Update the customer-facing "verify origin" copy if the incident revealed a
  copy gap.