Eve library

OSHUN V1 Domain Service Health, Readiness, and Circuit-Breaker Conventions

Spec

Revision 1

On this page

Status: Accepted v1.0
Date: March 23, 2026
Owners: OSHUN Engineering, OSHUN Platform, Domain Leads
Related:

  • docs/releases/v1/specs/todos.md
  • docs/releases/v1/scope/domain-service-routing-and-discovery-rules.md
  • apps/oshun/bff/src/routes/health.ts
  • apps/oshun/bff/src/adapters/domain-service-adapters.ts
  • apps/oshun/bff/src/health/domain-service-health.ts
  • apps/metis/api-gateway/src/proxy/backend-proxy.ts

1. Purpose#

This document defines the shared health-check, readiness-probe, and circuit-breaker conventions that the OSHUN shell and BFF use to detect domain-service degradation uniformly.

It closes V1-ARCH-088.

The repo already contained real health endpoints across Tara, Veritas, Nyx, Nisaba, Iris, and Metis, but they did not share one path shape or one status model. The BFF also hardcoded /healthz for every active adapter even though the upstreams do not actually expose one shared probe contract.

The result was that the shell could detect only a subset of failures correctly, and readiness behavior depended on domain-specific quirks instead of one normalized rule set.

2. Repo-Informed Current State#

Observed health surfaces in the repository are:

Service runtime Liveness-like surface Readiness-like surface Detailed surface Notes
Tara API /health /ready /health /health reports healthy or degraded; /ready reports ready: boolean
Veritas API /healthz, /health /readyz /health/detailed /readyz already reports ok, degraded, or unhealthy
Nyx API /health none /health only liveness-style health exists today
Iris API /health/live, /health /health/ready /health/detailed readiness exists but Iris is not yet an active BFF adapter
Nisaba API /healthz /readyz /health/detailed follows Veritas-style normalized readiness status
Metis API gateway /healthz, /health /readyz, /ready /health/detailed gateway already includes executable circuit-breaker semantics
Arete adapter API /v1/arete/healthz none none shell-facing adapter contract already points here

That means one shell-wide rule cannot be "always call /healthz" and cannot be "treat any non-200 as the only degradation signal."

3. Canonical Conventions#

3.1 Probe Semantics#

Every shell-serving domain service should expose three conceptual probe types:

  1. liveness Verifies the process is alive. It should not fail merely because a downstream dependency is degraded.
  2. readiness Verifies the service is ready to serve shell traffic with acceptable dependency state.
  3. detailed Returns dependency-level detail for dashboards, incident review, and debugging.

The BFF uses the strongest available probe in this priority order:

  1. readiness
  2. liveness

Detailed health is informational and not the primary shell-routing probe.

3.2 Normalized Status Model#

The BFF normalizes all upstream probe payloads into one shell-facing status model:

Normalized status Meaning
ok service is healthy enough for normal shell traffic
degraded service can still serve traffic, but a dependency or capability is impaired
down service should be treated as unavailable for shell traffic

Canonical input mapping rules:

  • ok, ready, healthy, and alive normalize to ok
  • degraded, partial, warning, and not_initialized normalize to degraded
  • down, unhealthy, not_ready, error, and failed normalize to down
  • ready: false normalizes to down
  • HTTP 5xx from the selected probe normalizes to down
  • network and timeout failures normalize to down
  • dependency booleans normalize to ok or down
  • dependency objects normalize independently so the shell can inspect which dependency degraded

3.3 BFF Readiness Semantics#

GET /healthz on the BFF remains a pure liveness probe.

GET /readyz on the BFF now evaluates the active domain adapters and returns:

BFF status HTTP Meaning
ready 200 all active domain adapters are ok
degraded 200 at least one active domain adapter is degraded or down, but at least one remains available
not_ready 503 every active domain adapter is down

This rule is deliberate. OSHUN is a multi-domain shell with partial-failure behavior. One degraded domain should not eject the whole BFF from rotation when the shell can still serve other domains correctly.

4. Active BFF Adapter Contract#

For the current active adapter set, the canonical probe map is:

Adapter Preferred probe Probe path Liveness path Detailed path
Tara readiness /ready /health /health
Veritas readiness /readyz /healthz /health/detailed
Nyx liveness /health /health /health
Arete liveness /v1/arete/healthz /v1/arete/healthz none
Nisaba readiness /readyz /healthz /health/detailed
Metis readiness /readyz /healthz /health/detailed

Implications:

  • Tara and Veritas must be checked through readiness, not their simpler liveness-style routes, because the shell needs dependency-aware degradation signals.
  • Nyx currently lacks a readiness surface, so its liveness route is the best available shell-facing signal until Nyx adds readiness.
  • Arete remains on its existing adapter-specific health route until a fuller shell-facing readiness contract exists.

Future BFF adapters for Iris and Lilith must register the same convention metadata before they are considered production-ready shell dependencies.

5. Shared Circuit-Breaker and Retry Policy#

The BFF now standardizes the health-probe resilience policy across active adapters using the same baseline numbers already present in the Metis API gateway:

  • health probe timeout: 2000ms
  • max retries for retryable probe failures: 1
  • retryable probe status codes: 429, 502, 503, 504
  • retry delay: 250ms
  • circuit-breaker failure threshold: 5
  • circuit-breaker reset timeout: 30000ms
  • half-open success budget before closing: 3

Conventions for breaker behavior:

  • degraded responses do not open the breaker by themselves
  • down, timeout, and network failures count as breaker failures
  • when the breaker is open, the BFF reports the service as down immediately without waiting on another upstream timeout

6. Executable BFF Contract#

The BFF now implements the following executable behavior:

  • domain adapter getHealth() calls use per-service probe-path conventions instead of a hardcoded /healthz
  • each normalized domain health result includes:
    • service id
    • normalized status
    • check timestamp
    • probe type used
    • source path used
    • raw upstream status when available
    • dependency breakdown when available
    • current probe circuit-breaker state and policy
  • GET /readyz returns the normalized per-domain readiness summary the shell needs for uniform degraded-mode handling

7. Remaining Follow-On Work#

This document and the BFF implementation establish the contract, but they do not remove all domain-specific debt.

Outstanding follow-on implications include:

  • Nyx should add a real readiness endpoint so dependency degradation is visible without relying on liveness only.
  • Arete should publish a more explicit readiness and detailed-health surface if it remains a long-lived shell dependency.
  • future Iris and Lilith shell-facing facades should register their probe metadata before they are promoted into the active BFF adapter set.
  • Shell-to-service integration tests verify end-to-end connectivity and degraded-mode behavior against the adapter registry rather than assuming one generic health path.

8. Acceptance Record#

The OSHUN V1 program accepts the following as the canonical shell-serving health contract:

  • BFF liveness is process-only
  • BFF readiness is domain-aware and partial-failure aware
  • domain service health is normalized into ok | degraded | down
  • probe retry and circuit-breaker posture is shared and typed
  • per-domain probe paths are explicit and registry-backed, not guessed