## Phase 82: Voice-as-First-Class Gameplay Input — Tactical Commands, NPC Dialogue & Squad Control

**Purpose:** Make the player's voice a first-class input mechanism for gameplay
— not just modding (Phase 78). Players can talk to any NPC in natural language,
command tactical squads with real military voice procedures (Rainbow Six / SOCOM
style), negotiate with faction leaders, interrogate prisoners, and direct AI
teammates in real-time combat using voice. This phase combines on-device speech
recognition, dual-path intent classification (fast BERT + fallback SLM), spatial
reasoning from voice and gaze, voice-driven NPC dialogue, tactical squad command
execution, multi-agent formation control, and personality-matched TTS feedback.

**Reference Proposal:** `docs/proposals/LIVING_WORLD_NPC_SOTA_PROPOSAL.md`

**Scope:** Rust crates for ASR inference, intent classification, spatial
resolvers, and formation control. TypeScript libraries for voice dialogue
orchestration, negotiation system, command taxonomy, and TTS pipeline.
Integration with Phase 78 voice infrastructure (shared ASR), Phase 81 cognitive
core (NPC reasoning), Phase 43 behavior trees (action execution), Phase 79.24
personality (personality-modulated responses), and Phase 79.25 relationships
(history-aware dialogue).

**Technology:**

- **Rust crates** (in `libs/maya/engine-core/crates/`): `maya-voice-input`
  (game-side ASR integration, VAD, wake word), `maya-intent` (BERT-FID ONNX
  inference), `maya-spatial-voice` (gaze + entity resolution), `maya-formation`
  (ORCA + Boids tactical formations), `maya-squad-ai` (tactical behavior trees)
- **TypeScript libraries** (in `libs/maya/`): `voice-dialogue` (NPC conversation
  orchestration), `voice-negotiation` (persuasion/intimidation/deception),
  `voice-tactics` (command taxonomy + decomposition), `voice-tts` (personality
  TTS pipeline)
- **External dependencies**: whisper.cpp/CarelessWhisper (ASR), Picovoice
  Porcupine (wake word), ONNX Runtime (BERT-FID), Piper TTS (voice synthesis)
- **Integration targets**: Phase 78 (shared ASR infrastructure), Phase 81
  (cognitive core for NPC reasoning), Phase 43 (behavior trees), Phase 79.24
  (personality/emotions/memory), Phase 79.25 (relationships), Phase 79.31
  (NPC-NPC interaction), Phase 79.34 (information propagation for gossip)

---

### 82.1 On-Device ASR Pipeline for Gameplay (Rust)

Real-time speech recognition optimized for gameplay environments — noisy (combat
SFX, music, ambient), low-latency (<200ms), and streaming (partial results while
speaking). Shares infrastructure with Phase 78 ASR but adds gameplay-specific
noise filtering and context routing.

#### 82.1.1 Core ASR Engine

- [x] **82.1.1.1** Create `maya-voice-input` Rust crate with safe FFI wrapper
      around whisper.cpp (CarelessWhisper variant for streaming, 300ms chunk
      size, causal encoder)
- [x] **82.1.1.2** Implement audio capture integration — receive microphone PCM
      data from platform audio subsystem (WASAPI/CoreAudio/ALSA), 16kHz mono,
      with circular buffer for continuous input
- [x] **82.1.1.3** Create game audio cancellation — subtract game audio output
      (SFX, music, voice chat) from microphone input using adaptive echo
      cancellation to isolate player voice from speakers/headset bleed
- [x] **82.1.1.4** Implement voice activity detection (VAD) — Silero VAD model
      (<2MB, <10ms latency) detecting speech onset within 50ms and speech end
      within 200ms, with hysteresis to prevent false triggers from game audio
      transients (explosions, gunfire)
- [x] **82.1.1.5** Create noise-robust preprocessing — spectral gating to
      suppress game ambient noise, adaptive noise floor estimation that adjusts
      to current game audio environment (quiet exploration vs loud combat)
- [x] **82.1.1.6** Implement streaming transcription — emit partial transcript
      within 150ms of speech onset, update every 100ms as more audio arrives,
      finalize within 200ms of speech end
- [x] **82.1.1.7** Create utterance boundary detection — segment continuous
      speech into complete commands/sentences using pause detection (300ms
      default), prosodic falling intonation, and syntactic completeness
      estimation
- [x] **82.1.1.8** Implement multi-language support — load Whisper multilingual
      model, detect language automatically within first 2 seconds of speech,
      support mid-conversation language switching
- [x] **82.1.1.9** Create per-word confidence scoring — annotate each
      transcribed word with confidence (0-1), flag low-confidence words (<0.6)
      for potential clarification requests
- [x] **82.1.1.10** Implement ASR model configuration — support multiple model
      sizes (tiny.en 75MB for command-only, medium 1.5GB for full conversation)
      with automatic selection based on available resources
- [x] **82.1.1.11** Create ASR performance profiling — measure per-utterance
      latency, word error rate on gameplay audio test set, CPU/GPU usage
- [x] **82.1.1.12** Implement ASR integration tests — test with recorded
      gameplay audio at various noise levels (quiet, combat, music) with ground
      truth transcriptions

---

### 82.2 Tactical Intent Classification — Fast Path (Rust)

BERT-FID-inspired fast intent classifier for structured tactical commands.
Achieves <100ms classification with high accuracy on the 25-category tactical
command taxonomy. High-confidence results execute directly without SLM.

#### 82.2.1 Intent Classifier

- [x] **82.2.1.1** Create `maya-intent` Rust crate with ONNX Runtime integration
      for BERT-class model inference
- [x] **82.2.1.2** Implement tokenizer — BPE tokenizer compatible with
      fine-tuned BERT model, max 64 tokens per utterance (tactical commands are
      short)
- [x] **82.2.1.3** Create 3-layer hierarchical intent taxonomy with 25
      categories: Layer 1 (team selection:
      alpha/bravo/charlie/everyone/nearest), Layer 2 (action:
      move/hold/stack/breach/engage/suppress/cease_fire/flash_clear/
      smoke/cover/overwatch/formation/posture), Layer 3 (target/location
      extraction as named entities)
- [x] **82.2.1.4** Implement named entity recognition — extract team names,
      location references ("that door", "left side", "second floor"), target
      references ("the hostile", "the vehicle"), and formation names ("wedge",
      "column", "diamond") as typed entities
- [x] **82.2.1.5** Create confidence scoring with entropy optimization — compute
      output entropy, classify as high-confidence (<0.3 entropy → execute
      directly) or low-confidence (≥0.3 entropy → route to SLM planner §82.3)
- [x] **82.2.1.6** Implement fast-path execution — high-confidence classified
      intents bypass SLM entirely and map directly to atomic action structs
      submitted to the squad AI (§82.7)
- [x] **82.2.1.7** Create training data generation — use SLM to generate 50,000+
      synthetic tactical command utterances with intent labels, augmented with
      paraphrasing and noise injection
- [x] **82.2.1.8** Implement model fine-tuning pipeline — fine-tune BERT-base
      (110M params) on tactical command dataset, export to ONNX format, quantize
      to INT8 for inference
- [x] **82.2.1.9** Create domain vocabulary injection — add game-specific terms
      (weapon names, location names, callsign names) to tokenizer vocabulary and
      retrain intent classifier when game content changes
- [x] **82.2.1.10** Implement classification latency benchmark — verify <100ms
      end-to-end from text input to intent struct output on target hardware
- [x] **82.2.1.11** Create intent classification evaluation suite — precision,
      recall, F1 per intent category, confusion matrix, with particular focus on
      dangerous misclassifications (e.g., "hold fire" classified as "fire")
- [x] **82.2.1.12** Implement continuous learning — misclassified commands
      (detected when player repeats or corrects) logged for retraining

---

### 82.3 Natural Language Command Decomposition — SLM Path (TypeScript + Rust)

For complex or ambiguous commands that the BERT classifier cannot resolve, a
small language model decomposes natural language into sequences of atomic squad
actions with temporal ordering, dependencies, and spatial resolution.

#### 82.3.1 SLM Command Planner

- [x] **82.3.1.1** Create `libs/maya/voice-tactics/` TypeScript library with
      command decomposition, temporal ordering, and dependency resolution
- [x] **82.3.1.2** Implement command prompt template — system prompt defining
      available atomic actions (move_to, breach, hold, engage, suppress,
      formation, etc.) with parameter schemas, plus current squad state
      (positions, ammo, health, readiness)
- [x] **82.3.1.3** Create atomic action library — 40+ typed actions with
      parameters: `move_to(agent, position)`, `breach(agents[], door, method)`,
      `suppress(agent, area, duration)`, `formation(agents[], type, center)`,
      `hold_position(agent, position, facing)`, etc.
- [x] **82.3.1.4** Implement multi-step decomposition — complex commands like
      "Alpha go around back and breach the kitchen while Bravo holds the front"
      decomposed into: [assign Alpha team, pathfind to rear, hold at kitchen
      door, assign Bravo team, hold at front door, synchronize breach on signal]
- [x] **82.3.1.5** Create temporal dependency resolver — actions tagged with
      prerequisites (breach_after:signal, suppress_until:team_in_position),
      compiled into an execution DAG
- [x] **82.3.1.6** Implement "on my mark" / signal system — commands with "on my
      mark" create a pending action set that executes atomically when player
      says "go" / "execute" / "breach"
- [x] **82.3.1.7** Create conditional commands — "if they surrender, restrain
      them; if they resist, engage" → conditional branches in execution plan
- [x] **82.3.1.8** Implement standing order system — "hold this position until I
      say otherwise" creates a persistent order that remains active until
      explicitly countermanded or the NPC is forced to break it (self-
      preservation)
- [x] **82.3.1.9** Create command validation — verify decomposed plan is
      feasible given current squad state (enough ammo, path exists, agents
      alive), report impossibility to player via TTS if not
- [x] **82.3.1.10** Implement plan preview — before execution, briefly flash
      squad member waypoints and action icons on HUD so player can verify the
      system understood correctly (optional, togglable)

---

### 82.4 Spatial Reasoning from Voice (Rust)

Resolves spatial references in voice commands ("that door", "left flank", "the
red building") to actual world coordinates using player gaze direction,
proximity, entity metadata, and CLIP-based visual similarity.

#### 82.4.1 Spatial Resolver

- [x] **82.4.1.1** Create `maya-spatial-voice` Rust crate with spatial reference
      resolution pipeline
- [x] **82.4.1.2** Implement gaze-anchored reference resolution — "that" +
      entity_type resolves to the entity nearest to the player's crosshair
      raycast intersection, within 50m range, matching the specified type (door,
      window, vehicle, hostile, position)
- [x] **82.4.1.3** Create relative spatial resolution — "left" = 90° left of
      squad facing direction in world space, "right" = 90° right, "behind" =
      180° from movement vector, "ahead" = forward, "above" = +Y
- [x] **82.4.1.4** Implement ordinal spatial references — "second door on the
      left" → enumerate all doors within FOV, sort left-to-right from player
      perspective, select index 2
- [x] **82.4.1.5** Create named landmark resolution — "the church" / "the red
      building" → search entity database for matching name or description within
      200m radius, rank by distance and visibility
- [x] **82.4.1.6** Implement CLIP-based visual reference — "the building with
      the broken window" → run CLIP similarity between description embedding and
      visible entity visual feature embeddings, return highest match above
      threshold
- [x] **82.4.1.7** Create contextual reference resolution — "the same spot" →
      retrieve from squad episodic memory the last referenced position; "where
      we entered" → retrieve entry point from mission memory
- [x] **82.4.1.8** Implement proximity-based disambiguation — when multiple
      entities match a reference, prefer the one closest to the player or
      closest to the referenced team's current position
- [x] **82.4.1.9** Create cover point inference — "take cover" → query navmesh
      cover point graph for nearest cover relative to threat direction, suitable
      for the specified agent
- [x] **82.4.1.10** Implement spatial reference confidence — each resolution
      includes confidence score; low confidence triggers clarification: "Did you
      mean the north door or the south door?" via TTS
- [x] **82.4.1.11** Create spatial resolution debugging — visualize resolved
      positions as markers in the game world, show alternative candidates and
      their scores
- [x] **82.4.1.12** Implement spatial resolution benchmark — test suite with
      100+ reference utterances in various tactical scenarios, measure
      resolution accuracy and disambiguation quality

---

### 82.5 Conversational NPC Interaction — Voice Dialogue Mode (TypeScript)

Any Tier 0 NPC (Phase 81) can engage in natural voice conversation with the
player. The system manages turn-taking, emotional responses, personality-
consistent dialogue, and consequential actions from conversation.

#### 82.5.1 Voice Dialogue System

- [x] **82.5.1.1** Create `libs/maya/voice-dialogue/` TypeScript library with
      dialogue session management, turn-taking, and consequence engine
- [x] **82.5.1.2** Implement conversation initiation — detect when player is
      addressing an NPC via: proximity (<3m) + gaze (looking at NPC) + speech
      (VAD active) + optional wake word ("Hey [name]"). Activate dialogue mode
      for that NPC.
- [x] **82.5.1.3** Create dialogue session state — track conversation history,
      current topic, NPC emotional trajectory, and player relationship changes
      accumulating during this conversation
- [x] **82.5.1.4** Implement NPC response generation — player utterance fed to
      NPC's SLM (Tier 0 via Phase 81) with full context: personality, current
      emotion, relationship with player, memories of player, current activity,
      location context, knowledge/rumors, and conversation history
- [x] **82.5.1.5** Create personality-consistent dialogue — SLM system prompt
      enforces personality-driven speech patterns: high Intelligence → complex
      vocabulary, high Humor → jokes/wit, low Education → simpler language, high
      Formality → respectful forms of address
- [x] **82.5.1.6** Implement emotion-driven responses — NPC emotional state
      (Phase 79.24.3) modulates SLM response: frightened NPC speaks haltingly,
      angry NPC is curt and confrontational, happy NPC is warm and generous
- [x] **82.5.1.7** Create knowledge-bounded responses — NPC only knows what
      they've experienced (memory stream) and what they've heard (gossip via
      Phase 79.34). NPC cannot reveal information they don't have.
- [x] **82.5.1.8** Implement turn-taking management — detect when player stops
      speaking (VAD + pause), begin NPC response generation immediately, handle
      player interruption gracefully (NPC pauses or adjusts response)
- [x] **82.5.1.9** Create conversation consequences — actions NPC decides to
      take based on conversation: give item, reveal information, change opinion
      of player, agree to quest, refuse request, offer trade, attack, flee.
      Consequences dispatch to game systems.
- [x] **82.5.1.10** Implement conversation memory — entire exchange stored as
      episodic memories for both the NPC and the player profile, affecting
      future interactions
- [x] **82.5.1.11** Create conversation ending — NPC naturally concludes
      conversation when topic exhausted, they need to resume activity, or
      emotional state makes continued conversation unlikely (anger → walks away,
      fear → runs)
- [x] **82.5.1.12** Implement eavesdropping — nearby NPCs (Tier 0/1) can
      "overhear" player-NPC conversations, storing overheard information in
      their memory streams, potentially spreading it as gossip (Phase 79.34)
- [x] **82.5.1.13** Create multi-NPC conversation — player can address a group
      of NPCs, each responds in turn based on personality (dominant
      personalities speak first, shy personalities wait), with NPCs reacting to
      each other's responses
- [x] **82.5.1.14** Implement conversation logging — full transcripts stored for
      debugging, with NPC internal state (emotion, opinion changes) annotated at
      each turn
- [x] **82.5.1.15** Create dialogue integration tests — scripted conversation
      scenarios verifying personality consistency, knowledge boundaries, and
      correct consequence dispatch

---

### 82.6 Voice Negotiation & Persuasion System (TypeScript)

Players can negotiate, persuade, intimidate, and deceive NPCs through natural
conversation. Outcomes depend on NPC personality, relationship, evidence, and
the quality of the player's argument.

#### 82.6.1 Negotiation Engine

- [x] **82.6.1.1** Create `libs/maya/voice-negotiation/` TypeScript library with
      negotiation state machine, persuasion model, and outcome resolver
- [x] **82.6.1.2** Implement negotiation detection — identify when a
      conversation transitions from chat to negotiation (player makes a request,
      offer, or demand) and activate the negotiation engine
- [x] **82.6.1.3** Create persuasion scoring model — SLM evaluates player
      argument quality considering: logical coherence, emotional appeal,
      evidence cited, reputation leverage, and alignment with NPC values
- [x] **82.6.1.4** Implement personality-modulated resistance — NPC personality
      determines negotiation difficulty: high Stubbornness increases resistance,
      high Agreeableness decreases it, high Greed makes financial arguments more
      effective, high Honor makes duty arguments more effective
- [x] **82.6.1.5** Create trade negotiation — "Can you do 30 gold instead of
      50?" → NPC evaluates: item value, own need for gold, player relationship,
      surplus stock, personality (greedy vs generous). Returns counter-offer,
      acceptance, or firm refusal.
- [x] **82.6.1.6** Implement diplomatic persuasion — "We need your militia
      against the bandits" → NPC evaluates: threat assessment, bravery,
      community values, self-interest, military strength. May impose conditions
      ("Only if you deal with the river crossing first").
- [x] **82.6.1.7** Create intimidation mechanics — "You don't want to make an
      enemy of me" → NPC evaluates: player's reputation and apparent power vs
      own courage, available backup, consequences of compliance. Cowardly NPCs
      yield, brave NPCs call bluff.
- [x] **82.6.1.8** Implement deception detection — "I'm a merchant guild
      inspector" → NPC evaluates: player appearance, known facts, NPC suspicion
      trait, whether claim is verifiable. High-Intelligence NPCs are harder to
      deceive.
- [x] **82.6.1.9** Create multi-round negotiation — negotiation can span
      multiple exchanges with escalating or de-escalating tension, NPC patience
      tracking (eventually walks away if negotiation drags)
- [x] **82.6.1.10** Implement witness effects — NPCs nearby who witness
      intimidation or deception may change their opinion of the player, or
      report behavior to authorities (Phase 79.29 justice system)
- [x] **82.6.1.11** Create faction-level negotiation — player negotiating with a
      faction leader affects faction-wide disposition, potentially triggering
      Phase 79.36 political consequences
- [x] **82.6.1.12** Implement negotiation memory — NPCs remember past
      negotiations, broken promises increase resistance in future dealings, kept
      promises increase trust

---

### 82.7 Squad Tactical Command Execution (Rust + TypeScript)

Bridges voice commands to squad NPC behavior trees. Handles command queuing,
synchronization points, and team assignment for tactical operations.

#### 82.7.1 Command Execution Engine

- [x] **82.7.1.1** Create `maya-squad-ai` Rust crate with tactical behavior tree
      extensions, command queue, and synchronization primitives
- [x] **82.7.1.2** Implement command queue — per-team FIFO queue of pending
      commands. New voice commands append to queue. "Cancel" clears queue.
      "Override" replaces current execution.
- [x] **82.7.1.3** Create team assignment system — maintain named teams (Alpha,
      Bravo, Charlie) with persistent membership. "You" targets nearest team
      member. "Everyone" targets all. Team composition changeable via voice.
- [x] **82.7.1.4** Implement breach execution — coordinated multi-step: stack
      formation at breach point → breach method (kick/explosive/flashbang) →
      entry sequence (first/second/third person) → room clear pattern (left/
      right split, center dominate, cross-entry)
- [x] **82.7.1.5** Create hold and synchronize — "Breach on my mark" puts team
      in ready state. All teams reach ready → player says "Go" → synchronized
      execution with <100ms delta between first and last agent action.
- [x] **82.7.1.6** Implement movement execution — pathfind to target position
      using Phase 43 navmesh, with tactical movement (slice the pie around
      corners, check doorways, cover leapfrog) instead of direct-line movement
- [x] **82.7.1.7** Create fire control execution — "Weapons free" sets ROE to
      engage all identified hostiles. "Weapons hold" restricts to self-defense.
      "Suppress [area]" directs sustained fire at area without requiring
      identified targets.
- [x] **82.7.1.8** Implement posture transitions — "Go quiet" activates stealth
      behavior tree: suppressed weapons, CQC preference, whispered comms, slower
      movement. "Go loud" reverses: full auto, shout callouts, sprint.
- [x] **82.7.1.9** Create non-lethal mode — "Non-lethal" restricts squad to
      tasers, batons, restraints. Hostage rescue scenarios require this.
- [x] **82.7.1.10** Implement command acknowledgment dispatch — when command is
      received, immediately dispatch acknowledgment to TTS (§82.10): "Copy,
      moving to position" or "Negative, path blocked"
- [x] **82.7.1.11** Create command failure reporting — if execution fails mid-
      way (path destroyed, target eliminated, agent down), report failure via
      TTS and await new orders or fall back to autonomous behavior
- [x] **82.7.1.12** Implement command history — player can say "Again" or
      "Repeat last" to re-execute the most recent command for a team
- [x] **82.7.1.13** Create tactical command integration tests — scripted breach
      scenarios verifying correct sequencing, synchronization, and failure
      handling

---

### 82.8 Multi-Agent Formation Control (Rust)

ORCA-based collision avoidance with Boids-based formation maintenance. Squads
maintain tactical formations while navigating through environments, dynamically
adjusting spacing for indoor/outdoor transitions.

#### 82.8.1 Formation System

- [x] **82.8.1.1** Create `maya-formation` Rust crate with formation templates,
      ORCA integration, and dynamic spacing
- [x] **82.8.1.2** Implement formation template library — wedge (V-shape,
      balanced), column (single-file, narrow spaces), line abreast (horizontal
      spread, wide coverage), diamond (360° protection), echelon left/right,
      staggered column
- [x] **82.8.1.3** Create per-agent formation slot assignment — each team member
      assigned a relative offset from formation center. Slot assignment
      considers role (point man at front, medic at center, heavy weapons at
      rear).
- [x] **82.8.1.4** Implement formation center tracking — formation center
      follows player (when leading) or designated waypoint (when directed), with
      smooth acceleration/deceleration matching player speed
- [x] **82.8.1.5** Create ORCA collision avoidance — each squad member runs ORCA
      velocity obstacle computation against all nearby agents (teammates +
      civilians + enemies) to prevent collisions while maintaining formation
- [x] **82.8.1.6** Implement adaptive spacing — automatically tighten formation
      in narrow spaces (hallways, doorways) and expand in open areas, using
      navmesh width queries to determine available space
- [x] **82.8.1.7** Create bounding overwatch movement — team splits into fire
      team and maneuver team. Fire team holds and covers. Maneuver team advances
      to next cover. Roles swap. Alternating advance with mutual support.
- [x] **82.8.1.8** Implement formation transition — smooth interpolation between
      formation types over 1-2 seconds when player gives formation change
      command, agents pathfind to new relative positions
- [x] **82.8.1.9** Create formation break and reform — when formation is broken
      by combat or obstacles, agents attempt to reform at earliest opportunity.
      If reformation is impossible (separated by walls), maintain local subgroup
      coherence.
- [x] **82.8.1.10** Implement fire sector auto-assignment — based on formation
      type and threat direction, each agent automatically assigns a coverage
      sector (no two agents cover the same sector, full 360° coverage in
      diamond, forward 180° in wedge)
- [x] **82.8.1.11** Create formation performance benchmark — verify 16-agent
      formation with ORCA collision avoidance at <0.5ms per frame
- [x] **82.8.1.12** Implement formation visualization — debug overlay showing
      formation template, agent slots, ORCA velocity obstacles, fire sectors

---

### 82.9 Combat AI with Voice Override Priority (Rust)

Autonomous tactical combat behavior that squad NPCs execute between voice
commands. NPCs act like competent soldiers — seeking cover, calling contacts,
managing ammo — but always yield to player voice commands.

#### 82.9.1 Autonomous Combat Behavior

- [x] **82.9.1.1** Implement threat assessment — continuously scan for hostile
      entities, rate threat level (distance × weapon × stance × visibility),
      prioritize engagement order (nearest immediate threat first)
- [x] **82.9.1.2** Create cover-seeking behavior — maintain proximity to cover
      objects (walls, crates, vehicles). When engaged, immediately move to
      nearest viable cover relative to threat direction. Evaluate cover quality
      (thickness, height, flanking exposure).
- [x] **82.9.1.3** Implement peek and fire — from cover, lean/peek to acquire
      target, fire burst, return to cover. Vary peek direction and timing to
      avoid predictability.
- [x] **82.9.1.4** Create suppressive fire behavior — when teammate is moving,
      provide covering fire at threat position. Sustain fire for movement
      duration. Call "Covering!" via TTS.
- [x] **82.9.1.5** Implement ammo management — track magazine state, call "Mag
      change!" when reloading, seek cover during reload. Call "Running low!" at
      25% ammo remaining.
- [x] **82.9.1.6** Create casualty response — when teammate goes down, nearest
      agent calls "Man down!", moves to downed teammate under covering fire,
      drags to cover, applies first aid. Player can override with "Leave him,
      push forward."
- [x] **82.9.1.7** Implement autonomous callouts — "Contact! Three hostiles,
      ground floor, east side!" using actual spatial relationship to player and
      compass bearing. Call frequency modulated by personality (taciturn NPCs
      call less frequently).
- [x] **82.9.1.8** Create room clearing protocol — autonomous CQB: slice the
      pie, buttonhook or cross-entry at doorways, clear corners, announce
      "Clear!" or "Hostile down!" per room section
- [x] **82.9.1.9** Implement retreat behavior — when outgunned (threat level
      exceeds squad capability by 2:1+), propose retreat: "We're outgunned,
      suggest we pull back!" Player can confirm or override.
- [x] **82.9.1.10** Create noise discipline — in stealth posture, avoid firing
      unless directly threatened, prefer CQC takedowns, whisper callouts, move
      slowly

#### 82.9.2 Personality-Driven Combat Variation

- [x] **82.9.2.1** Implement bravery-driven aggression — high Bravery NPCs push
      aggressively, volunteer for point position, advance under fire. Low
      Bravery NPCs stay further back, hesitate before advancing, prefer
      long-range engagement.
- [x] **82.9.2.2** Create cautiousness-driven thoroughness — high Cautiousness
      NPCs take extra time to clear corners, double-check rooms, move slowly
      through doorways. Low Cautiousness NPCs clear faster but may miss threats.
- [x] **82.9.2.3** Implement veteran vs rookie skill progression — rookie NPCs
      have jerkier movement between cover, slower weapon transitions, wider aim
      spread. Veterans move fluidly, transition quickly, and have tighter aim.
      NPCs improve with combat experience over time.
- [x] **82.9.2.4** Create loyalty-driven protective behavior — high Loyalty NPCs
      prioritize player safety, interpose between player and threats, call
      warnings about flanking enemies targeting the player. Low Loyalty NPCs
      prioritize self-preservation.
- [x] **82.9.2.5** Implement discipline-driven command compliance — high
      Discipline NPCs comply immediately and precisely with voice commands. Low
      Discipline NPCs may hesitate, comply imperfectly, or question orders ("You
      sure about that?"). Extremely low Discipline + low Loyalty may refuse
      suicidal orders.
- [x] **82.9.2.6** Create aggression-driven engagement style — high Aggression
      NPCs prefer close engagement, push toward enemies, use shotguns/SMGs. Low
      Aggression NPCs prefer distance, use rifles, maintain defensible
      positions.
- [x] **82.9.2.7** Implement stress-degraded performance — under sustained
      combat, NPCs accumulate stress. High Neuroticism NPCs degrade faster: aim
      worsens, callouts become panicked, movement becomes erratic. High
      emotional stability NPCs maintain composure under prolonged combat.
- [x] **82.9.2.8** Create personality-driven combat integration tests — verify
      that personality facets produce visibly different combat behavior across a
      squad of 4-8 NPCs with diverse personalities

#### 82.9.3 Voice Override Priority System

- [x] **82.9.3.1** Implement 6-level priority hierarchy — P5: player voice
      command, P4: self-preservation (under direct fire), P3: team safety
      (rescue downed), P2: standing order (persistent command), P1: autonomous
      tactical, P0: idle/formation
- [x] **82.9.3.2** Create priority arbitration — when voice command (P5)
      arrives, current behavior tree node is suspended (not terminated), voice
      command node injected, executes to completion, then suspended behavior
      resumes or is replaced depending on world state
- [x] **82.9.3.3** Implement self-preservation exception — NPC in immediate
      lethal danger (grenade at feet, melee attack) may delay voice command
      execution by 0.5-1s to dodge/evade first, then comply. Reports: "Taking
      cover! Moving to position after!"
- [x] **82.9.3.4** Create compliance personality modulation — high Discipline
      NPCs comply immediately and precisely. Low Discipline NPCs may hesitate,
      comply imperfectly, or question orders ("You sure about that?"). Extremely
      low Discipline + low Loyalty may refuse suicidal orders.
- [x] **82.9.3.5** Implement standing order persistence — "Hold this position"
      remains active as P2 until countermanded. If NPC is forced to leave
      position (grenade, fire), returns ASAP and reports "Returning to
      position."
- [x] **82.9.3.6** Create command queue vs override behavior — sequential
      commands queue (execute in order). "Override" or urgent tone command
      replaces queue entirely. System distinguishes by utterance analysis.
- [x] **82.9.3.7** Implement priority system debugging — visualize current
      priority stack per NPC, active behavior tree node, queued commands

---

### 82.10 TTS Feedback & Personality-Matched Voice (TypeScript + Rust)

NPCs respond vocally to commands and events using text-to-speech synthesis that
matches their personality, emotional state, and tactical situation.

#### 82.10.1 Voice Synthesis Pipeline

- [x] **82.10.1.1** Create `libs/maya/voice-tts/` TypeScript library with voice
      profile management, emotion modulation, and TTS provider abstraction
- [x] **82.10.1.2** Implement on-device TTS using Piper TTS (Rust integration) —
      lightweight neural TTS (<100MB model), <100ms latency for short utterances
      (callouts, acknowledgments)
- [x] **82.10.1.3** Create voice profile system — each NPC assigned a voice
      profile specifying: base voice model, pitch range, speaking rate,
      accent/dialect, vocabulary complexity. Derived from personality facets.
- [x] **82.10.1.4** Implement emotion modulation — current emotional state
      (Phase 79.24.3) modulates TTS parameters: fear → higher pitch, tremor,
      faster pace; anger → lower pitch, louder, clipped words; calm → steady
      pace, neutral pitch; confidence → strong projection
- [x] **82.10.1.5** Create tactical brevity — in combat, NPC voice responses are
      short and clipped ("Copy!", "Moving!", "Contact left!"). In dialogue mode,
      full sentences. Context determines verbosity level.
- [x] **82.10.1.6** Implement spatial audio integration — NPC voice responses
      spatialized in 3D using Phase 79.38 audio propagation: NPC at distance
      sounds distant, NPC behind wall sounds muffled, NPC on radio sounds
      filtered
- [x] **82.10.1.7** Create voice priority queue — multiple NPCs speaking
      simultaneously are serialized with priority (command acknowledgment >
      contact callout > status update > ambient chatter) to prevent audio
      overload
- [x] **82.10.1.8** Implement lip sync integration — TTS audio output drives
      Phase 79.17 viseme generation for lip-synchronized facial animation
- [x] **82.10.1.9** Create voice variety system — ensure squad members have
      distinct voices so player can identify who is speaking without seeing them
- [x] **82.10.1.10** Implement TTS caching — frequently used callouts ("Copy",
      "Moving", "Contact!", "Clear!") pre-generated and cached at session start
      for instant playback without synthesis latency
- [x] **82.10.1.11** Create radio filter effect — when NPC is out of visual
      range, apply radio comm filter (bandwidth limiting, static noise) to TTS
      output to simulate radio communication
- [x] **82.10.1.12** Implement TTS quality benchmark — verify <150ms latency for
      callouts, <500ms for full sentences, natural prosody, personality
      distinguishability between NPC voices

---

### 82.11 Wake Word & Voice Activation Modes (Rust)

Three voice input modes for different gameplay situations, ensuring voice
control feels natural and never interferes with gameplay when not intended.

#### 82.11.1 Activation System

- [x] **82.11.1.1** Implement push-to-talk mode — designated key/button
      activates microphone. Default mode. Zero false activation risk. Released
      key = end of utterance.
- [x] **82.11.1.2** Create wake word mode using Picovoice Porcupine — custom
      wake words: "Hey [NPC name]" for conversation, "Squad" or "Team" for
      tactical commands. <10KB per keyword, <30ms detection, fully on-device.
- [x] **82.11.1.3** Implement proximity auto-listen — when player is within 2m
      of an NPC in dialogue mode, microphone stays active (with clear on-screen
      indicator). Deactivates when player moves away or conversation ends.
- [x] **82.11.1.4** Create mode switching — player can switch between modes via
      settings or in-game command. Modes can be assigned per context (PTT for
      combat, wake word for exploration, auto-listen for dialogue).
- [x] **82.11.1.5** Implement false activation prevention — wake word + VAD
      confirmation required (must detect actual speech after wake word, not just
      ambient noise matching)
- [x] **82.11.1.6** Create voice input indicator UI — persistent but unobtrusive
      HUD element showing current mode (PTT/wake/auto), microphone state
      (listening/processing/recognized), and confidence level
- [x] **82.11.1.7** Implement voice input routing — based on game context, route
      transcribed speech to appropriate handler: tactical command → §82.2/82.3,
      NPC dialogue → §82.5, negotiation → §82.6, general → context analysis
- [x] **82.11.1.8** Create custom wake word training — players can assign custom
      names to squad members, Porcupine trains custom keyword model on first use
- [x] **82.11.1.9** Implement voice input accessibility — configurable
      sensitivity for different speech volumes, support for speech impediments
      via adjusted recognition thresholds, visual command input as
      always-available fallback
- [x] **82.11.1.10** Create voice activation integration tests — test all three
      modes in gameplay scenarios: PTT during combat, wake word during
      exploration, auto-listen during NPC dialogue

---

### Phase 82 Summary — Voice-as-First-Class Gameplay Input

**New Systems:**

| System Area                  | Section | Tasks   |
| ---------------------------- | ------- | ------- |
| Core ASR Engine              | 82.1.1  | 12      |
| Intent Classifier            | 82.2.1  | 12      |
| SLM Command Planner          | 82.3.1  | 10      |
| Spatial Resolver             | 82.4.1  | 12      |
| Voice Dialogue System        | 82.5.1  | 15      |
| Negotiation Engine           | 82.6.1  | 12      |
| Command Execution            | 82.7.1  | 13      |
| Formation System             | 82.8.1  | 12      |
| Autonomous Combat Behavior   | 82.9.1  | 10      |
| Personality Combat Variation | 82.9.2  | 8       |
| Voice Override Priority      | 82.9.3  | 7       |
| Voice Synthesis Pipeline     | 82.10.1 | 12      |
| Activation System            | 82.11.1 | 10      |
| **TOTAL Phase 82**           |         | **145** |

---

### Voice Capabilities After Phase 82

| Capability          | Before Phase 82                   | After Phase 82                                        |
| ------------------- | --------------------------------- | ----------------------------------------------------- |
| Player voice input  | Modding companion only (Phase 78) | Full gameplay voice control                           |
| NPC conversation    | Text/scripted dialogue            | Natural voice conversation with any NPC               |
| Squad commands      | Menu/button-based                 | Natural language tactical voice commands              |
| Negotiation         | Dialogue tree options             | Free-form voice persuasion/intimidation/deception     |
| Command latency     | N/A                               | <650ms end-to-end (matching F.A.C.U.L.)               |
| Spatial references  | N/A                               | Gaze + CLIP + entity DB resolution                    |
| Squad formations    | Manual positioning                | Voice-commanded tactical formations                   |
| Combat AI           | Basic behavior trees              | Full autonomous tactical behavior with voice override |
| NPC voice responses | Pre-recorded barks                | Personality-matched TTS with emotion                  |
| Activation modes    | N/A                               | PTT, wake word, proximity auto-listen                 |

**Combined with Phase 81's cognitive core and Phase 79's NPC life simulation,
Phase 82 makes every NPC in the Maya world someone the player can talk to — and
every squad member someone the player can command with their voice in real-time
tactical combat.**
