# Study Data, Evidence, Learning, and Operations This page is the technical and governance companion to the [Yemaya Study Workspace overview](./yemaya-study-workspace.md). It follows a study record from source identity through anchors, evidence, notebooks, analysis, learning, export, retention, deletion, and recovery. ```mermaid erDiagram TENANT ||--o{ STUDY_PROJECT : owns STUDY_PROJECT ||--o{ SOURCE_WORK : contains SOURCE_WORK ||--o{ SOURCE_EDITION : versions SOURCE_EDITION ||--o{ TRACK : exposes TRACK ||--o{ ANCHOR : locates ANCHOR ||--o{ ANCHOR_REVISION : versions ANCHOR_REVISION ||--o{ ANNOTATION : supports ANNOTATION }o--o{ EVIDENCE_RECORD : cites STUDY_PROJECT ||--o{ NOTEBOOK_ITEM : organizes NOTEBOOK_ITEM }o--o{ EVIDENCE_RECORD : links SOURCE_EDITION ||--o{ ANALYSIS_RUN : inputs ANALYSIS_RUN ||--o{ ANALYSIS_OUTPUT : produces ANALYSIS_OUTPUT }o--o{ ANNOTATION : suggests STUDY_PROJECT ||--o{ LEARNING_ACTIVITY : assigns LEARNING_ACTIVITY }o--o{ NOTEBOOK_ITEM : evaluates STUDY_PROJECT ||--o{ EXPORT_CASE : exports STUDY_PROJECT ||--o{ DELETION_CASE : governs RIGHTS_RECORD ||--o{ SOURCE_EDITION : permits RIGHTS_RECORD ||--o{ ANALYSIS_RUN : permits ``` The model keeps source/version, coordinates, assertions, notebook synthesis, automated analysis, learning, rights, export, and deletion identities distinct so each can be reviewed, invalidated, retained, or reclaimed honestly. ## Identity model The workspace must not collapse distinct identities into a convenient string. At minimum, preserve separate identifiers and versions for: | Identity | Why it is separate | | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | | Tenant, project, membership, role | Authorization and isolation are evaluated in context; a user id alone is insufficient. | | Source work, edition/cut/build, source object, ingest version | The same work can have materially different frames, timing, content, rights, hashes, and provenance. | | Media/document/game track | Audio, video, transcript, document, input, camera, telemetry, or state tracks use different coordinate systems and availability. | | Anchor and anchor revision | A region/time range is versioned and may be reprojected, invalidated, superseded, or disputed. | | Annotation, claim/observation, relation | Content, evidence status, authorship, review, and disagreement evolve independently of the anchor. | | Notebook item, collection, question, hypothesis, decision | Learning and synthesis objects have their own history and links; they are not comments on a blob. | | Analysis run, method/model/provider, output | Reproducibility and withdrawal require exact input, parameters, code/model version, provider, and result identity. | | Export and deletion case | External disclosure and erasure are governed workflows with manifests, signatures, receipts, and audit. | The [schema inventory](../../../docs/proposals/yemaya-study-workspace/schema-inventory.md) maps existing media, annotation, notebook, evidence, rights, search, graph, learning, replay, telemetry, signing, and audit concepts and records identity conflicts that adapters must not paper over. ## Source intake and rights Every source begins with a rights posture, not only a file upload: 1. Identify tenant/project, uploader, origin, declared ownership/license, permitted purposes, territory/time constraints, and consent where people are depicted or recorded. 2. Validate type, size, container, extension, magic bytes, archive structure, URL policy, and malware/quarantine status before a parser or previewer sees the payload. 3. Hash and register the immutable source object; preserve the logical work and edition/build identity separately. 4. Extract only permitted metadata and derived representations. Record tool, version, parameters, and failure/partial state. 5. Propagate access and rights constraints to previews, thumbnails, transcripts, embeddings, annotations, model inputs, exports, caches, and backups. Rights **expiry** and retention **deletion** are separate boundaries. At expiry, use, processing, display, sharing, or export may stop while a justified retention period still preserves restricted records. At retention end, the reclamation workflow removes eligible source and derived material and proves what remains. The operational sequence is in the [rights-expiry runbook](../../../docs/runbooks/yemaya-study-expiry.md). ## Anchors and evidence An evidence record binds a statement to inspectable context: ```text tenant/project + source edition/build + track + anchor coordinates/basis/revision + observation or claim + author + status + method/model/provider/version + uncertainty + supporting/counter evidence + relations + review/adjudication + history + rights ``` Coordinate bases include timecode, frames, samples, pages, text spans, image regions, object/pose tracks, game ticks, events, world positions, and declared normalized coordinates. Conversion is explicit and versioned. A player or renderer may present convenient time, but stored evidence cannot rely on an unstated frame rate or a mutable media URL. Evidence status distinguishes observation, hypothesis, automated suggestion, accepted annotation, reviewed/adjudicated result, disputed result, superseded record, unavailable input, and withheld output. Confidence never replaces the status or evidence list. ## Analysis runs and provider/model lineage An analysis request records authorized input references, purpose, requested capability, parameters, expected output schema, budget/capacity class, and the requesting actor. The execution record adds selected adapter, tool/model/ provider versions, environment, timestamps, retry/fencing state, raw result identity, normalized projection identity, validation, and final disposition. Provider output is untrusted input. It passes schema and bounds validation, rights and policy checks, prompt-injection containment, provenance attachment, and review rules before becoming a visible suggestion. Failure to obtain a result produces a failure or withheld state; it must not reuse a stale result without labeling it. Model withdrawal is a lineage problem, not a config edit. Operators must find affected outputs, determine whether the old model can be reproduced, rerun where permitted, compare or invalidate projections, communicate differences, and preserve history. See the [model-replacement runbook](../../../docs/runbooks/yemaya-study-model-replacement.md). ## Search and graph Search indexes permission-filtered projections, never raw cross-tenant storage. Index rows retain tenant/project, source/anchor, rights state, schema/version, and deletion lineage. Query filters apply before aggregation and ranking so counts, facets, suggestions, and timing do not leak inaccessible sources. Semantic/vector retrieval states what representation was embedded, which model and version produced it, which rights allowed it, and what “similar” means for that feature. Re-embedding creates a versioned index and controlled cutover; it does not silently change saved-query results. The relation graph supports evidence, comparison, learning, and creative lineage. Edges are typed and attributable. Derived graph projections can be rebuilt from source records; they are not a second ungoverned system of record. ## Notebook, questions, and original decisions Notebook objects hold notes, excerpts, questions, collections, hypotheses, comparisons, practice outcomes, and original creative decisions. Each object has revision history, authorship, access, source backlinks, and unresolved-link behavior. Copying an excerpt into a notebook does not remove its source rights. A useful creative-decision record answers: - What problem or intention is ours? - Which observations or comparisons informed it? - Which elements are rejected or deliberately changed? - What constraint, experiment, or exercise will test it? - What output or review resulted? - Which sources can be shared with the decision, and which remain private? ## Learning and evaluation governance Metis learning objects can reference Study Workspace sources, questions, exercises, submissions, rubrics, and evidence. The adapter preserves both domains' identities and versions. Progress and assessment results do not turn a subjective annotation into objective fact. Evaluation corpora and guides are versioned and governed for licensing, consent, privacy, retention, access, publication, deletion, adjudication, and legitimate disagreement. A result names corpus, guide, schema, evaluator, threshold, and software/model versions. Aggregate scores include denominator, withheld/excluded cases, and uncertainty. The maintained source is [Evaluation governance](../../../docs/proposals/yemaya-study-workspace/evaluation-governance.generated.md), with release-lane coverage in the [harness inventory](../../../docs/proposals/yemaya-study-workspace/harness-inventory.md). ## Export and sharing An export is a governed, immutable result with: - requester, tenant/project, scope, purpose, and authorization decision; - included record and artifact versions; - explicit exclusions and reasons, especially restricted source media; - citation and provenance bundle; - schema/tool versions and a content manifest with hashes; - signature and verification instructions where required; - expiry/retention and revocation/takedown handling; - audit event and delivery receipt. Sharing evaluates the recipient and resource at access time. A signed export proves its contents; it does not grant ongoing access to the live project or override source licenses. ## Deletion and tenant closure Deletion follows the reference graph across source objects, previews, transcripts, embeddings, annotations, analysis inputs/outputs, notebooks, search/graph projections, caches, exports, and backups. The workflow is idempotent, retryable, observable, and able to say “pending” or “blocked” with the exact reason. Audit records preserve the fact and authorization of deletion without retaining the prohibited content. External exports or provider copies require tracked receipts or explicit residual-risk reporting. Use the [source-deletion runbook](../../../docs/runbooks/yemaya-study-deletion.md) for a single request and the [tenant-closure runbook](../../../docs/runbooks/yemaya-study-tenant-closure.md) for export-before-close ordering and final verification. ## Persistence and migration posture The service's migration history lives under `apps/yemaya/svc-study-workspace/migrations`; the application routes and domain composition live under `apps/yemaya/svc-study-workspace/src`. Do not infer the live schema from the latest migration filename or generated client alone. Validate migration ordering, schema/contract conformance, backfill behavior, rollback/forward-fix guidance, and the deployed revision. System-of-record tables, derived projections, queues/outboxes, object storage, and caches have different restore and reclamation semantics. Every projection declares its rebuild source and version; a table that no reader consumes is not evidence of a feature. ## Operations and failure modes | Failure | Required behavior | | ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Authorization leakage | Contain access, revoke affected sessions/links, preserve audit, identify affected resources/queries, verify tenant filtering, and communicate scope. | | Malware or parser escape | Quarantine source and derivatives, stop affected workers/provider path, retain safe forensic evidence, rotate exposed credentials, and reprocess only after a trusted fix. | | Parser crash loop | Fence/retry with limits, expose source-specific failure, preserve original input, prevent queue starvation, and offer supported remediation. | | Prompt injection | Treat source text/metadata as data, disable unsafe tools, preserve the attempted instruction in security evidence, and re-evaluate affected outputs. | | Provider compromise | Disable the adapter, rotate secrets, identify inputs/outputs, validate lineage, withhold affected results, and follow replacement/reprocessing policy. | | Projection failure | Keep system-of-record writes intact, mark reads stale/partial, stop invalid cutover, rebuild deterministically, and verify counts/hash/sample semantics. | | Signing failure | Refuse a falsely “verified” export, isolate keys/service, preserve unsigned artifacts as non-deliverable, and reissue with attributable identity. | | Stuck deletion | Keep the case open and visible, retry with fencing, enumerate unreclaimed targets, escalate owners, and never report completion early. | Detailed incident sequences are in [Study Workspace incident runbooks](../../../docs/proposals/yemaya-study-workspace/incident-runbooks.md). ## Rollout, rollback, backup, and restore Rollout uses explicit stages and decision owners; migrations and irreversible side effects require forward-only or compensating plans. A percentage flag is not a safe rollback if new records or external outputs cannot be understood by the previous reader. The [rollout](../../../docs/runbooks/yemaya-study-rollout.md) and [rollback](../../../docs/runbooks/yemaya-study-rollback.md) runbooks define the order and completion signals. Provider changes use the [provider-change runbook](../../../docs/runbooks/yemaya-study-provider-change.md). Backup/restore covers relational state, object storage, and the evidence needed to rebuild projections. Verification checks semantic consistency—not only that files and rows exist—and records known recovery-objective shortfalls. See [backup and restore](../../../docs/runbooks/yemaya-study-backup-restore.md). ## Verification gates A complete change selects applicable lanes from: - contract/schema and generated-client drift; - migration, backfill, persistence, transaction, idempotency, and concurrency; - domain unit and property tests; - service route, auth, tenant, integration, provider, and failure tests; - projection/search/graph reproducibility and deletion propagation; - browser journeys across permissions, success, empty, partial, offline, error, rights expiry, version drift, and recovery; - keyboard, screen reader, contrast, zoom/reflow, reduced motion, captions, and non-pointer alternatives; - performance, scale, capacity, soak, and resource-exhaustion behavior; - security, privacy, malware/parser, SSRF, prompt-injection, export, signing, and tenant-isolation tests; - evaluation corpus/guide/adjudication and release-exit evidence; - rollout, rollback, restore, provider-failover, deletion, and incident drills. Local setup and the authoritative checker list are maintained in [Study Workspace development](../../../docs/proposals/yemaya-study-workspace/DEVELOPMENT.md).