Eve library

ISIS v2 WS09-006: RunPod Per-Endpoint SLOs

Guide

Revision 1

On this page

Scope#

Implemented per-endpoint RunPod SLO telemetry for latency, success rate, timeout rate, and queue wait, and exposed these SLOs across API, SDK, and web operations surfaces.

Implemented Artifacts#

  • RunPod adapter SLO contracts and runtime aggregation:
    • libs/isis/ai-providers/src/providers/comfy/types.ts
    • libs/isis/ai-providers/src/providers/comfy/index.ts
    • libs/isis/ai-providers/src/providers/comfy/runpod-adapter.ts
    • libs/isis/ai-providers/src/providers/comfy/runpod-adapter.test.ts
  • Generation API SLO surface:
    • apps/isis/generation-api/src/services/comfyui.service.ts
    • apps/isis/generation-api/src/routes/jobs.ts
    • apps/isis/generation-api/src/routes/jobs.spec.ts
  • SDK and web UI exposure:
    • libs/isis/client/src/types.ts
    • libs/isis/client/src/resources/generation.ts
    • libs/isis/client/src/index.ts
    • apps/isis/web/src/pages/RunPodPage.tsx
    • apps/isis/web/src/pages/RunPodPage.spec.tsx

SLO Model#

  • Per-endpoint rolling samples are captured from completed/failed/timeout jobs.
  • SLO objective defaults:
    • latency p95: <= 180000ms
    • success rate: >= 99%
    • timeout rate: <= 1%
    • queue wait p95: <= 30000ms
    • minimum sample size before breach evaluation: 5
  • /api/v1/jobs/runpod/operations now includes endpoint SLO snapshots with:
    • objective thresholds
    • endpoint sample sizes
    • computed p95/percentage metrics
    • breach booleans per SLO dimension
    • endpoint topology enrichment (family, channel)

Verification#

bash
pnpm --filter @isis/ai-providers exec vitest run src/providers/comfy/runpod-adapter.test.ts --reporter=dot
pnpm --filter @isis/generation-api exec vitest run src/routes/jobs.spec.ts src/services/comfyui.service.spec.ts src/middleware/runpod-budget-guardrail.spec.ts --reporter=dot
pnpm --filter @isis/web exec vitest run src/pages/RunPodPage.spec.tsx src/pages/JobSubmissionStudio.spec.tsx --reporter=dot

Outcome#

WS09-006 is now closed with first-class per-endpoint SLO observability across RunPod lanes, enabling objective breach detection for latency, reliability, and queueing behavior in both API and operator UI.