Eve library

ISIS v2 WS08-003: Prompt + Reference Audio Conditioning Support

Guide

Revision 1

On this page

Scope#

WS08-003 implements first-class prompt and reference-audio conditioning contracts for audio workflows, with capability-aware application so conditioning is enforced only when the selected workflow/provider lane supports it.

Research and Existing Implementation Alignment#

Before implementation, existing capability patterns were reviewed in:

  • Internal provider implementations:
    • libs/isis/audio-generation/src/generation/suno-provider.ts
    • libs/isis/audio-generation/src/generation/self-hosted.ts
  • Internal run-time contract normalization patterns:
    • apps/isis/generation-api/src/workflows/executor.ts
    • apps/isis/generation-api/src/schemas/job.schema.ts
  • External Comfy audio references:

These sources informed a contract that supports reference-audio conditioning where backends expose such capabilities, while remaining safe for backends that do not.

Deliverables#

  1. Typed API/SDK conditioning contracts
  • Added audioConditioning / audio_conditioning parameter contracts with:
    • mode and provider capability policy
    • prompt/negative-prompt conditioning hints
    • reference audio URL(s)
    • style reference IDs
    • conditioning strength and optional reference weighting
  1. Validation layer support
  • Added JobAudioConditioningRequestSchema to generation API job schema.
  • Added schema validation for URL constraints, mode enums, and strength bounds.
  • Added schema tests for valid and invalid conditioning payloads.
  1. Workflow capability exposure
  • Added conditioning-capable input contracts to audio workflows:
    • sfx-sound-designer
    • voice-performance-synthesis
    • music-cue-composer
  • Added dedicated conditioning inputs for reference URLs, mode, strength, provider capability policy, and prompt hinting.
  1. Runtime conditioning normalization and application
  • Added canonical audio conditioning normalization in workflow executor.
  • Added workflow-capability-aware input mapping:
    • conditioning only applied when supported input keys exist
    • providerCapabilityMode: "required" blocks application when support is absent
  • Added metadata diagnostics (output.metadata.audioConditioning) describing:
    • requested conditioning payload
    • capability support status
    • applied/used input keys
    • blocked-by-capability outcomes
  1. Execution regression coverage
  • Added executor regression test for simulated music workflow runs asserting:
    • conditioning request normalization
    • provider-capability enforcement behavior
    • emitted audioConditioning diagnostics and applied key set

Verification Commands#

bash
pnpm --filter @isis/workflows exec vitest run src/workflows.spec.ts --reporter=dot
pnpm --filter @isis/generation-api exec vitest run src/schemas/job.schema.spec.ts src/workflows/executor.spec.ts --reporter=dot

Result#

WS08-003 is now implemented with typed, validated, and runtime-enforced audio conditioning support, including provider-capability guards and explicit execution diagnostics.