# Log Retention Policy

Owner: platform team Last reviewed: 2026-05-28

This doc specifies how long logs are kept by tier and by domain, and what
triggers a purge. It exists because:

- Compliance audits want to know retention windows up front.
- On-call needs to know how far back they can grep when triaging long-tail
  issues.
- DBAs need to know what storage to provision for log indexes.

## Tiers

| Tier                               | Retention                  | Storage                                    | Notes                                             |
| ---------------------------------- | -------------------------- | ------------------------------------------ | ------------------------------------------------- |
| Application (info+)                | 7 days                     | CloudWatch / Loki hot                      | Service-level events, healthy operation           |
| Application (debug)                | 24 hours                   | CloudWatch / Loki hot                      | Only on when a debug flag is toggled              |
| Error                              | 30 days                    | CloudWatch / Loki hot + S3 cold copy at 7d | Stack traces, 5xx responses, unhandled exceptions |
| Audit (security)                   | 90 days hot, 7 years cold  | Elasticsearch / S3 (Glacier)               | Tamper-evident; never editable in place           |
| Access (request)                   | 14 days                    | CloudWatch / Loki hot                      | HTTP request line, status, duration               |
| Audit (admin)                      | 365 days hot, 7 years cold | Elasticsearch / S3 (Glacier)               | Privileged-user actions, immutable                |
| Privacy events (consent, deletion) | 7 years                    | Elasticsearch / S3 (Glacier)               | GDPR Article 30 record-of-processing              |

## Per-domain retention overrides

Only document overrides here; the tier above is the default.

| Domain                    | Override                                          | Why                                                           |
| ------------------------- | ------------------------------------------------- | ------------------------------------------------------------- |
| aphrodite/payment         | Audit logs at **10 years hot/cold**               | PCI scope, financial records                                  |
| lilith/svc-auth           | Login audit kept **180 days hot**                 | Investigations on suspicious-login chains often span >90 days |
| iris/api memory snapshots | Per-user-controlled retention via consent records | Users opt-in to a retention window between 24h and 12mo       |

## Purge mechanism

- **CloudWatch / Loki hot tier** — automatic by log-group retention setting. Set
  via Terraform; never via console.
- **S3 cold tier** — lifecycle rule per bucket: transition to Glacier at the
  retention boundary; delete at the audit boundary.
- **Elasticsearch** — ILM policy per index alias, deletes the closed index when
  older than retention.
- **Postgres `audit_log` / `audit_logs` tables** (bellona, hathor, isis, sophia,
  yemaya) — the sweeper at
  [`scripts/retention/audit-log-retention.ts`](../../scripts/retention/audit-log-retention.ts)
  deletes rows older than the configured retention window. Defaults to the
  "Audit (security)" 90-day tier above; per-domain overrides go through
  `<DOMAIN>_AUDIT_RETENTION_DAYS` (e.g. `LILITH_AUDIT_RETENTION_DAYS=180` for
  the svc-auth override). Cron schedule:
  `15 3 * * *  npx tsx scripts/retention/audit-log-retention.ts --apply`. The
  script defaults to `--dry-run`; the cron expression must include `--apply`
  explicitly. It batches DELETEs (10_000 rows / batch with a 100ms pause) so it
  never holds a long table lock, and exits non-zero if any configured domain hit
  a SQL error so the cron alert routing fires.
- **Oshun BFF `admin_audit_event`** — retained for the canonical 2,555-day
  `audit_event` compliance window. The BFF evaluates expiry on startup and at
  least daily, and defers deletion while a referenced actor, target, subject,
  tenant, or owner has an active legal hold or retention exception. Successful
  reads require `admin:*`, write and durably flush an `audit.access.*` event,
  and are HMAC-SHA-256 chained with a production key held outside PostgreSQL.
  The operator-readable contract is `GET /v1/admin/audit-log/policy`; the
  machine-reviewed control record is
  [`docs/audits/eve-sota-retention-access-policy/2026-09-15.json`](../audits/eve-sota-retention-access-policy/2026-09-15.json).
- **Tombstone audit events** (`oshun_v1_tombstone_audit_events`) — these are
  governed by the privacy-event tier (7 years) and pruned by the privacy-team
  workflow, not by the retention cron above. Consent records are never deleted
  by the sweeper.

## What is NOT a "log" under this policy

- Database row-level audit columns (`created_at`, `updated_at`, `deleted_at`) —
  these are part of the schema retention policy, not log retention.
- User-generated content (posts, recordings, profile fields) — governed by the
  per-domain data-retention rules in `docs/compliance/data-retention.md` (TBD).
- Backup files — governed by `docs/runbooks/database-backups.md` (Postgres) and
  `docs/runbooks/object-storage-backups.md` (TBD, task #108).

## Adding a new log stream

1. Tag every event with the tier name (`app.info`, `audit.security`, etc.) at
   emit time so the routing layer can apply the right retention.
2. If the stream needs an override, add it to the table above in the same PR
   that introduces the stream.
3. If the override is more permissive than the default (longer retention),
   confirm with the platform team — the budget impact is real.