Documentation Reorganization Plan

Status: proposed Last updated: 2026-05-08 Owner: docs-platform Tooling: tools/docs/reorg.mjs

Hard constraints#

• Never touch V1/ at the repo root. That directory holds the V1 feature inventory and is owned outside this reorg. Excluded by hard prefix in tools/docs/reorg.mjs (see FORBIDDEN_PREFIXES).
• Never touch .claude/worktrees/. CLAUDE.md mandate.
• Every move uses git mv so history follows the file.
• Each phase is a single commit: script invocation + moves + link rewrites + verifier passes.
• Scope changes do not bypass pnpm-lock.yaml, lint, or the stub scanner; if a hook fails, fix the cause, do not skip.

Diagnosis#

Three orthogonal axes are jumbled in docs/: genre (reference, audit, runbook, ADR, proposal, spec), domain (lilith, metis, isis, nyx, ...), and time (evergreen, V1, P2, archived). Concretely:

Pristine = each axis lives at exactly one level: genre → release-or-domain → file.

Target structure#

docs/
├── README.md                         # Concise nav (replaces 67KB current)
├── CONTRIBUTING.md
│
├── reference/                        # Cross-cutting EVERGREEN
│   ├── architecture.md
│   ├── api-contracts.md
│   ├── coding-standards.md
│   ├── cross-domain-integration.md
│   ├── data-ownership-matrix.md
│   ├── dependency-management.md
│   ├── documentation-structure.md
│   ├── getting-started.md
│   ├── monitoring.md
│   ├── rollback-procedures.md
│   ├── security.md
│   ├── shared-libraries.md
│   ├── testing.md
│   ├── ci-cd.md
│   ├── faq.md
│   ├── inventories/                  # auto-generated artifacts (regen cmd documented)
│   └── runbooks/                     # cross-cutting only (cost, incident, scaling)
│
├── domains/                          # SINGLE source of truth per domain
│   ├── README.md                     # Coverage index
│   ├── _template/                    # The 8 required files (per existing standard)
│   └── <domain>/
│       ├── README.md
│       ├── architecture.md
│       ├── features.md
│       ├── developer-guide.md
│       ├── database.md
│       ├── service-catalog.md
│       ├── user-stories.md
│       ├── api/
│       ├── adr/                      # Domain-scoped ADRs
│       ├── audits/                   # Domain audits (point-in-time)
│       ├── runbooks/                 # Domain ops (isis lands ~1001 files here)
│       ├── proposals/
│       └── extras/                   # One-offs absorbed from docs/<domain>/*
│
├── adr/                              # GLOBAL ADRs only (single numbering)
│
├── proposals/                        # Pre-decision proposals
│
├── releases/                         # Release-cycle artifacts (decay together)
│   ├── v1/
│   │   ├── README.md
│   │   ├── scope/
│   │   ├── specs/
│   │   ├── design/
│   │   ├── runbooks/
│   │   ├── reports/
│   │   ├── domains/                  # V1 inventory snapshot
│   │   └── audits/
│   │       ├── <domain>/             # NISABA, ARETE, METIS, TARA, ...
│   │       └── <surface>/            # MOBILE, WEB, SHELL, CUSTOMER
│   ├── p2/
│   └── _archive/
│
├── guides/
├── compliance/   conventions/   infrastructure/
├── integrations/ migration/     performance/
├── research/     training/

What dissolves#

Naming conventions (applied to everything moved)#

• Directory and file names: lowercase-kebab.md. SCREAMING_SNAKE_CASE is dropped.
• ADRs: NNNN-short-title.md with single global numbering.
• Audits: OSHUN_V1_NISABA_STUDY_PLANS_AUDIT.md → releases/v1/audits/nisaba/study-plans.md.
• Every directory has a README.md (1-2 paragraphs: what's here, how to find things).

Phase plan#

Each phase is one commit. Run order matters: Phase N assumes Phase N-1 landed.

Phase 0 — Tooling and plan (this commit)#

No file moves. Lands tools/docs/reorg.mjs and this plan. Updates docs/reference/documentation-structure.md (formerly DOCUMENTATION_STRUCTURE_STANDARD.md) to describe the full taxonomy (later phase finishes that rename).

Phase 1 — Domain orphans (source-only)#

Move six hand-maintained top-level docs/<domain>/ dirs into docs/domains/<domain>/extras/: yemaya, iris, metis, hathor, bellona, veritas. ~125 files, almost all markdown. The few ../ links that escape the moved tree are fixed in the same commit.

docs/lilith/ and docs/kalika/ are NOT in this phase. They are mixed-mode (hand + generated) and fully-generated respectively, and need different handling — see Phase 1.5 and Phase 1.6 below.

Rewriter regex. The link rewriter uses (?<![/\w-])docs/<d>(?![\w-]) so it does NOT false-match libs/openapi/docs/<d>/, libs/<d>/core/docs/, /studio/docs/<d>/ URL slugs, or docs/<d>-orchestration/. The naive \b form caused 235+ false positives in a Phase 1 attempt and was discarded.

Phase 1.5 — docs/lilith/ source / generated split#

docs/lilith/ (1106 files) mixes hand-maintained markdown (~440 files across sync/, observability/, schemas/, metaverse/, reports/, code-samples/, examples/, performance/, scripts/, guides/, database/, curricula/, notes/) with generated content (generated/, api-reference/, plus 540 HTML files from TypeDoc / Stoplight).

The split:

• Hand-maintained *.md and source files → docs/domains/lilith/extras/
• docs/lilith/generated/, docs/lilith/api-reference/ and any other generator output → either .gitignore + relocate to libs/lilith/sdk/api-docs/ (typical TypeDoc target), or stay tracked under docs/domains/lilith/api-docs/ with a top-of-tree note that the content is generated.

The choice between gitignore and tracked-with-note is Phase 1.5's deliverable; not yet decided.

Phase 1.6 — docs/kalika/ (gitignore the generated tree)#

docs/kalika/ is 100% generated content (6452 HTML, 180 JS, 72 CSS, 36 SVG, 36 .nojekyll) plus 15 markdown sources. All commits to it are authored by Codex (the AI). Generators are explicit:

• pnpm kalika:api-reference — produces docs/kalika/api-reference/
• pnpm kalika:tutorial-notebooks — produces docs/kalika/tutorial-notebooks/
• pnpm kalika:domain-guide:check — verifies hand-maintained domain guide source matches expected output
• pnpm kalika:migration-guides:check — same for migration guides

Decision: gitignore the generated subtrees so the repo no longer tracks 6800 build artifacts. Add .gitignore entries for:

docs/kalika/api-reference/generated/
docs/kalika/tutorial-notebooks/generated/
docs/kalika/**/*.html
docs/kalika/**/*.js
docs/kalika/**/*.css
docs/kalika/**/.nojekyll

Then git rm -r --cached the matching tracked files, leaving them on disk but untracked. CI continues to regenerate them via the existing *:check scripts.

The 15 markdown sources stay tracked. Move them to docs/domains/kalika/ as part of bringing kalika into the standard domain shape — Phase 1.6 lands the gitignore + cleanup; the markdown relocation happens in Phase 6 (V1 inventory + outliers) so it batches with the rest of the kalika-related cleanup.

Phase 2 — Runbooks de-aggregation#

Move docs/runbooks/isis-*.md (1001 files) into docs/domains/isis/runbooks/. Move 3 cross-cutting files (cost-management.md, incident-response.md, scaling.md) to docs/reference/runbooks/.

Before executing: locate any generator script that writes into docs/runbooks/ and update its output target. If the runbooks are regenerated from manifests, the regen command is captured in docs/domains/isis/runbooks/README.md.

Phase 3 — ADR consolidation#

Resolve the docs/adr/ ↔ docs/oshun/adr/ numbering collision by renumbering oshun/adr/ entries into a single global sequence under docs/adr/. Domain-scoped ADRs (i.e., ADRs whose scope is a single domain rather than the platform) move to docs/domains/<domain>/adr/.

The renumbering map is preserved in docs/adr/_renumbering-map.md so old references can be traced.

Phase 4 — V1 audit corral (the big one, 306 files)#

Bucket docs/oshun/OSHUN_V1_<TOKEN>_*_AUDIT.md by <TOKEN>:

• Domain tokens (NISABA, ARETE, METIS, TARA, VERITAS, NYX, ...) → docs/releases/v1/audits/<domain>/<subject>.md
• Surface tokens (MOBILE, WEB, SHELL, CUSTOMER) → docs/releases/v1/audits/<surface>/<subject>.md
• The remainder (LIBRARY, DOMAIN, ACTIVITY, ASSISTANT, HOME, EXPLORE, PROFILE, ASSESSMENT, ...) → docs/releases/v1/audits/cross-cutting/<token-lowercased>/<subject>.md

Same commit rewrites the 150 internal cross-links and any external inbound links the verifier finds.

Phase 5 — V1 specs / scope / design#

The 248 non-AUDIT OSHUN_V1_*.md files split by content type:

The phase-5 path map is hand-curated, not regex-derived — content genre cannot be inferred from name alone.

Phase 6 — V1 inventory + P2 + outlier audits#

• docs/oshun/domains/ → docs/releases/v1/domains/ Update scripts/docs/generate-oshun-domain-reference.mjs to write to the new location, or retire it if docs/domains/<d>/ is now sufficient.
• docs/oshun/p2/ → docs/releases/p2/
• docs/oshun/audits/ (5 files) and the outlier docs/oshun/ASSET_AUDIT.md (no OSHUN_V1_ prefix) → distributed to docs/releases/v1/audits/<bucket>/
• docs/oshun/admin/ (14 redesign specs, V1_AWEB_*) → docs/releases/v1/specs/admin/

Phase 7 — Cross-cutting reports / analysis / audits#

Split the top-level docs/reports/, docs/analysis/, docs/audits/:

• V1-related → docs/releases/v1/reports/
• Evergreen schema/dependency inventories → docs/reference/inventories/ (each with a one-line regen command in its frontmatter)

Phase 8 — Top-level cleanup and nav#

• Move docs/architecture.md, docs/api-contracts.md, docs/coding-standards.md, etc. into docs/reference/.
• Replace 67KB docs/README.md with a concise nav (~200 lines max).
• Delete now-empty docs/oshun/.
• Verify zero references to docs/oshun/, docs/runbooks/isis-, or any SCREAMING_SNAKE_CASE doc path remain outside docs/releases/_archive/.

Execution model#

# Plan a phase (no writes):
node tools/docs/reorg.mjs --phase=1 --dry-run

# Execute a phase (git mv + link rewrites):
node tools/docs/reorg.mjs --phase=1 --execute

# Verify post-phase invariants:
node tools/docs/reorg.mjs --phase=1 --verify

# Verify ALL phases up to and including a phase:
node tools/docs/reorg.mjs --through=1 --verify

Safety rails#

• Forbidden prefixes are checked at the top of every run. V1/, .claude/worktrees/, node_modules/, .git/ are rejected as from or to for any move.
• Tombstones for high-traffic paths. For ~150 cross-linked V1 audit files, the executor leaves a 1-line redirect at the old path (Moved to /docs/releases/v1/audits/<bucket>/<subject>.md). Tombstones are removed in a follow-up commit two weeks after each phase (calendar reminder, not automated).
• Verifier fails the build on any banned reference outside docs/releases/_archive/. Banned references are derived from each phase's bannedAfter list.
• Generated content is identified before its source is moved. Each generator's output target is updated in the same commit as the move.
• Atomic per-phase commits so review is feasible. Reviewers read the script + the path map; they do not read every file diff.

Open questions#

• Does scripts/docs/generate-oshun-domain-reference.mjs keep generating docs/oshun/domains/, or is the generator retired in favor of docs/domains/<d>/? Decided in Phase 6.
• Once V1 ships, is docs/releases/v1/audits/ archived wholesale to docs/releases/_archive/v1/audits/, or kept hot indefinitely? Default: archive 6 months after launch.
• External links (Slack, Notion, public docs, CI dashboards) outside the repo cannot be rewritten by the script. Inventory them before Phase 4 lands.