Scope#
WS08-003 implements first-class prompt and reference-audio conditioning
contracts for audio workflows, with capability-aware application so conditioning
is enforced only when the selected workflow/provider lane supports it.
Research and Existing Implementation Alignment#
Before implementation, existing capability patterns were reviewed in:
- Internal provider implementations:
libs/isis/audio-generation/src/generation/suno-provider.tslibs/isis/audio-generation/src/generation/self-hosted.ts
- Internal run-time contract normalization patterns:
apps/isis/generation-api/src/workflows/executor.tsapps/isis/generation-api/src/schemas/job.schema.ts
- External Comfy audio references:
These sources informed a contract that supports reference-audio conditioning where backends expose such capabilities, while remaining safe for backends that do not.
Deliverables#
- Typed API/SDK conditioning contracts
- Added
audioConditioning/audio_conditioningparameter contracts with:- mode and provider capability policy
- prompt/negative-prompt conditioning hints
- reference audio URL(s)
- style reference IDs
- conditioning strength and optional reference weighting
- Validation layer support
- Added
JobAudioConditioningRequestSchemato generation API job schema. - Added schema validation for URL constraints, mode enums, and strength bounds.
- Added schema tests for valid and invalid conditioning payloads.
- Workflow capability exposure
- Added conditioning-capable input contracts to audio workflows:
sfx-sound-designervoice-performance-synthesismusic-cue-composer
- Added dedicated conditioning inputs for reference URLs, mode, strength, provider capability policy, and prompt hinting.
- Runtime conditioning normalization and application
- Added canonical audio conditioning normalization in workflow executor.
- Added workflow-capability-aware input mapping:
- conditioning only applied when supported input keys exist
providerCapabilityMode: "required"blocks application when support is absent
- Added metadata diagnostics (
output.metadata.audioConditioning) describing:- requested conditioning payload
- capability support status
- applied/used input keys
- blocked-by-capability outcomes
- Execution regression coverage
- Added executor regression test for simulated music workflow runs asserting:
- conditioning request normalization
- provider-capability enforcement behavior
- emitted
audioConditioningdiagnostics and applied key set
Verification Commands#
bash
pnpm --filter @isis/workflows exec vitest run src/workflows.spec.ts --reporter=dot
pnpm --filter @isis/generation-api exec vitest run src/schemas/job.schema.spec.ts src/workflows/executor.spec.ts --reporter=dot
Result#
WS08-003 is now implemented with typed, validated, and runtime-enforced audio
conditioning support, including provider-capability guards and explicit
execution diagnostics.