Eve library

OSHUN Studio Experimentation and Feature Flags Journeys and Edge Cases

Spec

Revision 1

On this page

Status: Active
Owner: OSHUN Studio Architecture + OSHUN Web + OSHUN BFF + OSHUN Data/Analytics Engineering + OSHUN Security
Last Updated: 2026-02-23
Related:

  • docs/adr/ADR-0042-oshun-studio-experimentation-and-feature-flags.md
  • docs/releases/v1/specs/studio-experimentation-and-feature-flags-dependency-and-data-contract-map.md

Purpose#

Define primary journeys and deterministic edge-case behavior for Studio experimentation and feature-flag workflows.

This document is the execution artifact for OST-00235.

Sources Used#

  • apps/oshun/web/src/experimentation/feature-flags.ts
  • apps/oshun/web/src/experimentation/hooks.ts
  • docs/adr/ADR-0042-oshun-studio-experimentation-and-feature-flags.md
  • docs/releases/v1/specs/studio-experimentation-and-feature-flags-dependency-and-data-contract-map.md
  • docs/domains/yemaya/features.md
  • docs/domains/isis/features.md
  • docs/domains/hathor/features.md
  • docs/domains/aja/features.md
  • docs/domains/bellona/features.md

Personas#

  • Collaboration Lead
  • Program Operator
  • Domain Integrator
  • Security Reviewer
  • Compliance Auditor

Journey Principles#

  1. Every flag and experiment must remain owner-attributed and expiry-bounded.
  2. Assignment/exposure/outcome events must stay deterministic and replayable.
  3. Policy controls must gate risky production-impacting toggles.
  4. Rollback/kill-switch actions must be immediate, auditable, and idempotent.

1. Primary Journeys#

J1. Define and Publish Feature Flag#

Goal: Create or update canonical flag metadata for cross-domain rollout.

Entry: Experimentation and feature flags workspace

Steps:

  1. Provide flagKey, owner, risk class, and expiry metadata.
  2. Select target domains (Yemaya, Isis, Hathor, Aja, Bellona, OSHUN).
  3. Validate lifecycle constraints (active/deprecated state, expiration bounds).
  4. Publish versioned flag definition to the registry.
  5. Emit flag-definition creation/update audit evidence.

Success State:

  • Flag definition is valid, versioned, and governance-ready.

J2. Configure Targeting and Cohort Assignment#

Goal: Define deterministic targeting predicates and assignment cohorts.

Entry: Published feature flag definition

Steps:

  1. Configure eligibility predicates (workspace, persona, release channel).
  2. Define control and treatment variants with explicit traffic allocation.
  3. Evaluate deterministic bucketing and collision checks.
  4. Persist assignment envelope and targeting snapshot.
  5. Emit assignment generation event with attribution context.

Success State:

  • Assignments are deterministic and auditable for replay/verification.

J3. Execute Progressive Rollout and Exposure Capture#

Goal: Roll out safely while preserving exposure lineage.

Entry: Approved experiment/flag rollout plan

Steps:

  1. Set rollout phase (preview, staging, production) and allocation.
  2. Execute rollout step with policy-aware authorization.
  3. Capture exposure events from participating domain workflows.
  4. Validate exposure completeness and duplication thresholds.
  5. Persist rollout execution + exposure summary evidence.

Success State:

  • Rollout progresses safely with complete exposure attribution.

J4. Evaluate Experiment Outcomes and Release Decision#

Goal: Convert exposure/outcome telemetry into deterministic decisions.

Entry: Exposure and KPI outcome streams available

Steps:

  1. Compute KPI deltas and confidence scores by cohort/variant.
  2. Review statistical confidence, guardrail metrics, and policy constraints.
  3. Compare outcomes against success/failure thresholds.
  4. Decide promote, extend, stop, or rollback.
  5. Emit decision artifact linked to evidence and owners.

Success State:

  • Decision is reproducible, evidence-backed, and policy-compliant.

J5. Trigger Rollback or Kill-Switch Incident Response#

Goal: Contain risk quickly when regression or policy breach is detected.

Entry: Incident trigger from Bellona/observability/policy signals

Steps:

  1. Select rollback or kill-switch action with reason code.
  2. Validate tier + role + approval requirements.
  3. Execute idempotent disable path for affected rollout scope.
  4. Verify propagation to domain runtime evaluators.
  5. Emit incident, rollback, and closure audit evidence.

Success State:

  • Risk is contained immediately with deterministic incident evidence.

2. Critical Edge Cases#

E1. Flag Definition Missing Expiry#

  • Trigger: definition submitted without expiresAt.
  • Required behavior: reject publish and require expiry metadata.

E2. Duplicate Flag Key Collision#

  • Trigger: create/update introduces duplicate flagKey.
  • Required behavior: hard reject and return deterministic conflict error.

E3. Assignment Envelope Missing Cohort#

  • Trigger: generated assignment omits cohortId.
  • Required behavior: quarantine assignment and block rollout advancement.

E4. Variant Allocation Sum Drift#

  • Trigger: control+treatments allocation does not equal 100%.
  • Required behavior: reject targeting spec with explicit correction guidance.

E5. Exposure Event Without Assignment Correlation#

  • Trigger: exposure arrives with no matching assignment identity.
  • Required behavior: quarantine event and raise telemetry integrity alert.

E6. Outcome Event Missing Confidence Metadata#

  • Trigger: KPI outcome has null/invalid confidence score.
  • Required behavior: block promotion decision and classify analysis incomplete.

E7. Policy Tier/Role Mismatch#

  • Trigger: actor selects tier not authorized for current role.
  • Required behavior: deny action and emit policy-denial audit event.

E8. Production Rollout Without CAB Evidence#

  • Trigger: production-impacting rollout action has missing approval ticket.
  • Required behavior: hard deny action and require CAB evidence.

E9. Rollback Request Against Closed Incident#

  • Trigger: rollback action requested after incident marked resolved/closed.
  • Required behavior: deny action and require new incident ticket linkage.

E10. Kill-Switch Propagation Timeout#

  • Trigger: runtime propagation exceeds latency SLO budget.
  • Required behavior: mark execution partial/error and escalate to incident queue.

3. Journey-to-Surface Mapping#

Journey Primary Surface Supporting Surfaces
J1 Define and Publish Feature Flag Experimentation/feature flags workspace Flag registry + governance metadata services
J2 Configure Targeting and Assignment Targeting/assignment panel BFF evaluator + cohort bucketing service
J3 Execute Progressive Rollout Rollout control panel Runtime evaluators + exposure ingestion pipelines
J4 Evaluate Outcomes and Decide Experiment analysis panel KPI calculators + observability readiness indicators
J5 Trigger Rollback or Kill-Switch Incident response panel Policy engine + kill-switch execution control path

4. Instrumentation Requirements#

Required events:

  • experimentation_feature_flags.journey.started
  • experimentation_feature_flags.journey.step.completed
  • experimentation_feature_flags.journey.blocked
  • experimentation_feature_flags.journey.recovered
  • experimentation_feature_flags.journey.completed

Required dimensions:

  • experimentKey, flagKey, cohortId, variant, sourceDomain
  • actorId, actorRole, workspaceId, permissionTier, releaseChannel
  • approvalTicketRef, incidentId, outcome, latencyMs, errorCode
  • kpiId, delta, confidenceScore, rollbackId, killSwitchExecuted

5. Acceptance Criteria (OST-00235)#

OST-00235 is complete only when:

  1. At least five experimentation/feature-flag journeys are documented.
  2. At least ten deterministic edge cases are documented.
  3. Journey-to-surface mapping is explicit.
  4. Instrumentation requirements and dimensions are documented.