# V2 Bot Authoring Guide

This guide covers `TODOS.phase-72.72.19.1.15`: authoring custom test scenarios
for the V2 playtesting bot framework. It defines the scenario contract, review
workflow, validation commands, and reporting requirements used by the headless
`V2BotHarness` and the nightly `v2-nightly-bot-suite`.

## Source Of Truth

- Authoring guide contract:
  [BotAuthoringGuide_V2_Contract.json](../../ue/Content/V2/QA/BotAuthoringGuide_V2_Contract.json)
- Bot harness contract:
  [BotHarness_V2_Contract.json](../../ue/Content/V2/QA/BotHarness_V2_Contract.json)
- Random exploration bot contract:
  [RandomExplorationBot_V2_Contract.json](../../ue/Content/V2/QA/RandomExplorationBot_V2_Contract.json)
- Coverage mapping contract:
  [BotCoverageMapping_V2_Contract.json](../../ue/Content/V2/QA/BotCoverageMapping_V2_Contract.json)
- Navigation stress-test bot contract:
  [NavigationStressTestBot_V2_Contract.json](../../ue/Content/V2/QA/NavigationStressTestBot_V2_Contract.json)
- Combat bot contract:
  [CombatBot_V2_Contract.json](../../ue/Content/V2/QA/CombatBot_V2_Contract.json)
- Puzzle-solving bot contract:
  [PuzzleSolvingBot_V2_Contract.json](../../ue/Content/V2/QA/PuzzleSolvingBot_V2_Contract.json)
- Soak test bot contract:
  [SoakTestBot_V2_Contract.json](../../ue/Content/V2/QA/SoakTestBot_V2_Contract.json)
- Load-test bot army contract:
  [LoadTestBotArmy_V2_Contract.json](../../ue/Content/V2/QA/LoadTestBotArmy_V2_Contract.json)
- RL-trained exploration agent contract:
  [RLTrainedExplorationAgent_V2_Contract.json](../../ue/Content/V2/QA/RLTrainedExplorationAgent_V2_Contract.json)
- Quest completion bot contract:
  [QuestCompletionBot_V2_Contract.json](../../ue/Content/V2/QA/QuestCompletionBot_V2_Contract.json)
- Economy bot contract:
  [EconomyBot_V2_Contract.json](../../ue/Content/V2/QA/EconomyBot_V2_Contract.json)
- Performance profiling bot contract:
  [PerformanceProfilingBot_V2_Contract.json](../../ue/Content/V2/QA/PerformanceProfilingBot_V2_Contract.json)
- Bot result reporting contract:
  [BotResultReporting_V2_Contract.json](../../ue/Content/V2/QA/BotResultReporting_V2_Contract.json)
- Nightly CI manifest:
  [v2-bot-run-ci.json](../../ue/Build/QA/v2-bot-run-ci.json)

## Scenario Contract

Every custom test scenario is a deterministic manifest row reviewed by QA and
run through one existing bot profile. The manifest is not free-form prose. It is
the source used to build `FV2BotHarnessAction` scripts, bind
`FV2BotHarnessPolicyBinding` controllers, select automation filters, configure
reporting, and route CI failures.

Use stable ids with the `bot.scenario.*` prefix. Example ids:

- `bot.scenario.market_vendor_loop`
- `bot.scenario.combat_arena_smoke`
- `bot.scenario.quest_marker_regression`
- `bot.scenario.performance_downtown_path`

Each scenario must declare its target map, deterministic seed, frame budget,
selected bot profile, automation filter, setup preconditions, action or policy
inputs, observation schema, assertions, artifacts, and alert policy. The same
scenario id must be used in bot result reports and screenshot artifact paths so
failures can be compared across runs.

## Custom Scenario Fields

| Field               | Authoring rule                                                                                                                        |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| `ScenarioId`        | Stable `bot.scenario.*` id. Do not reuse for a different gameplay purpose.                                                            |
| `Owner`             | QA, gameplay, level, economy, or performance owner responsible for triage.                                                            |
| `Map`               | Cooked or editor map path. Use one map per scenario unless testing travel.                                                            |
| `Seed`              | Deterministic seed. Changing it requires review because baseline output moves.                                                        |
| `FrameBudget`       | Maximum automation frames, plus per-frame action budget when applicable.                                                              |
| `AutomationFilter`  | Existing filter such as `V2.Playtesting.BotHarness`.                                                                                  |
| `BotProfile`        | One of the existing bot profiles: exploration, navigation, combat, puzzle, quest, economy, soak, load, RL, performance, or reporting. |
| `Preconditions`     | Spawn state, inventory, quest state, economy balance, data fixtures, and server mode.                                                 |
| `ActionScript`      | Ordered `FV2BotHarnessAction` sequence or policy input fixture.                                                                       |
| `ObservationSchema` | Required observation fields, hashes, coverage tags, and telemetry counters.                                                           |
| `Assertions`        | Expected pass criteria, disallowed warnings, crash detection, stuck spot detection, and performance budget checks.                    |
| `Artifacts`         | Transcript, report summary, coverage rows, frame-time table, and screenshot artifacts.                                                |
| `AlertPolicy`       | Whether failures page `qa-bot-alerts`, performance owners, or release owners.                                                         |

Keep scenario data compact. Large fixture content should live in data assets or
test fixture files referenced by stable path. Scenario manifests should remain
reviewable in code review and deterministic in CI.

## Bot Selection

Choose the smallest existing bot profile that can prove the scenario's risk:

- Use `V2.Playtesting.BotHarness` for exact scripted player actions or
  `CustomCommand` payloads.
- Use `V2.Playtesting.RandomExplorationBot` when the goal is crash detection,
  stuck spot detection, or noisy item and door interaction pressure.
- Use `V2.Playtesting.BotCoverageMapping` for area, transition, and nav-linked
  coverage requirements.
- Use `V2.Playtesting.NavigationStressTestBot` for pathfinding, fallback
  movement, unreachable target, and stuck-recovery regressions.
- Use `V2.Playtesting.CombatBot` for target selection, ability rotation,
  cooldown use, health/resource checks, and combat outcome assertions.
- Use `V2.Playtesting.PuzzleSolvingBot` for interactable sequences, puzzle
  states, and completion assertions.
- Use `V2.Playtesting.QuestCompletionBot` for objective marker flow, quest
  progress events, and completable quest validation.
- Use `V2.Playtesting.EconomyBot` for vendor loops, trades, inflation probes,
  item duplication probes, and economy exploit detection.
- Use `V2.Playtesting.PerformanceProfilingBot` for standardized paths,
  performance-intensive areas, frame-time capture, and regression comparison.
- Use `V2.Playtesting.BotResultReporting` only when validating report assembly,
  screenshot references, or dashboard-ready output.

Do not create a new bot profile for a one-off content case until the existing
profiles cannot express the risk with `FV2BotHarnessAction`, `CustomCommand`, or
policy input fixtures.

## Action And Observation Authoring

Scripted scenarios use ordered `FV2BotHarnessAction` rows. Each row should state
frame, action kind, axis values, target id, payload JSON, duration, and whether
the action is valid for headless execution. Use `RunBotHarnessScript` to verify
that the harness accepts the action surface and records a transcript.

Policy scenarios use `FV2BotHarnessPolicyBinding`. The binding must name the
controller kind, observation schema, action schema, and whether the controller
can emit any player action. Keep ML or scripted policy fixtures pinned by path
and version so reruns produce the same observation hash.

Observations must include the minimum data needed for a deterministic assertion:
location or area id, health/resource state, quest objective id, economy balance,
inventory state, interactable id, coverage tag, frame-time sample, crash
signature, stuck spot id, and final observation hash as applicable.

## Assertions And Failure Signals

Every scenario must contain at least one pass/fail assertion. Use explicit
checks instead of manual review notes. Common assertion categories are:

- No crash signatures and no harness rejections.
- No stuck spot id after recovery retries.
- Required areas visited and required transitions crossed.
- Expected interactable, combat, puzzle, quest, or economy event observed.
- Economy balance stays within approved source/sink bounds.
- Performance budget holds for p95 frame time, hitch count, and regression
  threshold.
- Final observation hash matches the approved baseline when the scenario is a
  regression fixture.

When a scenario is expected to produce warnings, declare the allowed warning ids
in the manifest. Unexpected warnings must fail CI validation so new regressions
are visible.

## Artifacts And Reporting

Custom scenarios must produce enough artifacts for triage without rerunning the
editor locally. Required artifacts are the action transcript, result summary,
assertion table, final observation hash, and report row. Add coverage rows,
frame-time samples, economy flow tables, or quest progress rows when the bot
profile produces them.

Screenshot artifacts are required for crash, stuck spot, coverage gap,
performance anomaly, puzzle failure, and quest failure reports. Route those
references through `GenerateBotResultReport` so `BotResultReporting` can
aggregate areas visited, crashes found, stuck spots, performance anomalies, and
screenshots in the same report contract used by nightly CI.

## Authoring Workflow

1. Pick the risk being tested and choose the smallest existing bot profile that
   can prove it.
2. Create a stable `ScenarioId`, owner, map, deterministic seed, frame budget,
   automation filter, and alert policy.
3. Author preconditions for spawn state, data fixtures, server mode, inventory,
   quest state, economy state, or performance baseline.
4. Write the `ActionScript` as `FV2BotHarnessAction` rows, or bind a scripted or
   ML controller through `FV2BotHarnessPolicyBinding`.
5. Define the `ObservationSchema` and baseline hashes required for deterministic
   assertions.
6. Add explicit assertions for crash detection, stuck spot detection, gameplay
   outcome, coverage, economy, and performance budget expectations.
7. Define artifacts, including transcript, report summary, screenshot artifacts,
   coverage rows, and frame-time data when relevant.
8. Run the scenario with `RunBotHarnessScript` or the selected bot helper and
   generate a report with `GenerateBotResultReport`.
9. Add the scenario to the nightly bot suite only after the targeted checker,
   result reporting checker, bot-run CI checker, and doc validation pass.
10. Attach command output and artifact paths to review.

## Review Checklist

Before approving a custom test scenario, confirm:

- `ScenarioId` is stable and uses the `bot.scenario.*` namespace.
- Owner, map, deterministic seed, frame budget, automation filter, and bot
  profile are explicit.
- Preconditions do not depend on editor-only state unless the scenario is
  editor-only by design.
- Action rows are valid `FV2BotHarnessAction` data and do not exceed per-frame
  budgets.
- Policy bindings declare their observation schema, action schema, controller
  kind, and versioned fixture paths.
- Assertions fail on new crashes, harness rejections, stuck spots, missing
  coverage, gameplay outcome regressions, economy exploits, or performance
  budget regressions.
- Allowed warnings are named explicitly; all other warnings fail validation.
- Screenshot artifacts are present for crash, stuck, coverage, performance,
  puzzle, and quest findings.
- Bot result reporting includes the scenario id in report rows and artifact
  paths.
- The scenario is covered by CI validation and the nightly
  `v2-nightly-bot-suite` before it is treated as a release signal.

## Validation Commands

Run these commands after changing bot authoring docs, scenario manifests, bot
contracts, result reporting, CI wiring, or Horde gates:

```bash
python3 V2/ue/Tools/check-v2-bot-authoring-guide.py
python3 V2/ue/Tools/check-v2-bot-run-ci.py
python3 V2/ue/Tools/check-v2-bot-result-reporting.py
python3 V2/ue/Tools/check-v2-bot-harness.py
python3 V2/ue/Tools/check-v2-random-exploration-bot.py
python3 V2/ue/Tools/check-v2-bot-coverage-mapping.py
python3 V2/ue/Tools/check-v2-navigation-stress-test-bot.py
python3 V2/ue/Tools/check-v2-combat-bot.py
python3 V2/ue/Tools/check-v2-puzzle-solving-bot.py
python3 V2/ue/Tools/check-v2-soak-test-bot.py
python3 V2/ue/Tools/check-v2-load-test-bot-army.py
python3 V2/ue/Tools/check-v2-rl-trained-exploration-agent.py
python3 V2/ue/Tools/check-v2-quest-completion-bot.py
python3 V2/ue/Tools/check-v2-economy-bot.py
python3 V2/ue/Tools/check-v2-performance-profiling-bot.py
python3 V2/ue/Tools/check-v2-tests-module.py
python3 V2/ue/Tools/check-v2-ci-workflow.py
python3 -m json.tool V2/ue/Content/V2/QA/BotAuthoringGuide_V2_Contract.json
python3 -m json.tool V2/ue/Build/Horde/v2-buildgraph-job.json
python3 V2/tools/validate-v2-docs.py
pnpm exec prettier --check V2/docs/qa/bot-authoring-guide.md V2/ue/Content/V2/QA/BotAuthoringGuide_V2_Contract.json V2/ue/Build/Horde/v2-buildgraph-job.json
```