Historical task source. The Eve task board holds current task state.
This file contains the detailed architecture notes, supporting requirements, and
expanded future backlog for Phase 180. It is intentionally separated from
TODOS/phase-180.md so coding agents can execute the canonical 180.C
checklist in strict top-to-bottom order without confusing this roadmap material
for executable tasks.
Use this file only as reference while completing the current unchecked 180.C
item. Do not check off items here as part of the canonical phase progression.
180.R Supporting Requirements and Detailed Roadmap#
The sections below expand the requirements behind the canonical 180.C
checklist. Coding agents should use them as detail references while executing
the 180.C sequence, not as permission to skip around the build order.
180.0 Coding-Agent Execution Contract and Top-to-Bottom Build Order#
This phase is intentionally ordered for coding agents. Work through it from top to bottom unless a later task is explicitly marked as a future extension. Do not start by building UI, Unreal depth, Browser Use delegation, fleet scheduling, or advanced automation macros before the protocol, policy, gateway, host, audit, and a minimal Blender/browser vertical slice exist. Each task should leave the repo in a better verified state with tests or documented hardware blockers.
180.0.1 Sequential Execution Rules#
- 180.0.1.1 Treat
180.0as the implementation contract for every coding agent before touching180.1+; each later task must inherit these ordering, verification, and safety rules. - 180.0.1.2 Execute discovery tasks before implementation tasks: complete code audits, capability matrices, and dependency decisions before creating new services or adapters.
- 180.0.1.3 Build the system in vertical slices, not disconnected islands: each slice must include protocol, gateway path, host path, adapter behavior, audit entry, tests, and at least one operator or CLI visibility surface.
- 180.0.1.4 Prefer smallest shippable increments: one adapter command, one policy gate, one event type, one UI panel state, or one transport path per task unless the checklist explicitly defines a bundle.
- 180.0.1.5 Do not introduce arbitrary remote shell or arbitrary Python execution as an early convenience; privileged execution comes only after policy, approvals, redaction, and audit are in place.
- 180.0.1.6 Do not expose Blender, Unreal, Chrome CDP, or desktop control ports outside localhost; remote access must go through the host runtime and authenticated gateway.
- 180.0.1.7 Do not rely on Computer Use for flows that already have a deterministic Blender, Unreal, browser, file, or process API path.
- 180.0.1.8 Do not mark an implementation task complete if it only adds types; complete work must include at least one exercising test, fixture, mock, CLI path, or documented hardware-in-the-loop validation.
- 180.0.1.9 Do not bury security exceptions in adapter code; all privileged behavior must pass through shared policy evaluation and emit audit records.
- 180.0.1.10 Keep documentation synchronized with behavior in the same change when commands, permissions, setup steps, or operator-visible workflows change.
180.0.2 Agent Work Item Shape#
- 180.0.2.1 For every coding task, start by naming the exact owning module, affected files, command namespace, permission scope, and expected verification command.
- 180.0.2.2 For every new command, define request schema, response schema, error codes, timeout class, risk class, audit payload, and test fixtures before wiring the command into an agent-facing surface.
- 180.0.2.3 For every new adapter operation, add a mocked adapter test first, then a real-adapter smoke test when the dependency can run in CI or on a hardware test machine.
- 180.0.2.4 For every new gateway route or MCP tool, add contract tests proving schema validation, auth failure, policy denial, successful dispatch, command result, and audit emission.
- 180.0.2.5 For every Remote Host capability, add a local health signal and a diagnostic error that explains missing app, missing permission, unsupported version, or unavailable helper process.
- 180.0.2.6 For every browser capability, include prompt-injection posture, profile isolation behavior, allowed origins, blocked origins, and submission/upload approval behavior.
- 180.0.2.7 For every desktop fallback capability, include screenshot evidence, target-window binding, coordinate normalization, visual verification, and emergency-stop behavior.
- 180.0.2.8 For every UI task, add loading, empty, success, failure, permission-missing, offline, and approval-pending states before treating the UI as complete.
- 180.0.2.9 For every deployment task, include local development, production configuration, secrets handling, health checks, logs, rollback, and smoke-test instructions.
- 180.0.2.10 For every completed task, update the checklist item to checked only after implementation, relevant automated verification, and any hardware/manual evidence notes are committed.
180.0.3 Dependency Ladder#
- 180.0.3.1 Complete product boundary, ownership, code audit, and architecture alignment before creating the protocol package.
- 180.0.3.2 Complete the remote protocol schemas before implementing gateway routes, host commands, MCP tools, Control Room timelines, or adapter command registries.
- 180.0.3.3 Complete device identity, session model, command ledger, and audit data model before implementing mutating Blender, Unreal, browser, or desktop commands.
- 180.0.3.4 Complete pairing, device authentication, and localhost-only adapter transport before enabling any cross-network command path.
- 180.0.3.5 Complete policy evaluation and approval primitives before adding destructive file operations, arbitrary Blender Python, Unreal console commands, shell execution, real browser profile access, or form submission.
- 180.0.3.6 Complete host health, diagnostics, and permission detection before relying on live desktop fallback or stream capture.
- 180.0.3.7 Complete a read-only Blender scene query before any Blender mutation command; complete undo/audit before broad scene-edit macros.
- 180.0.3.8 Complete browser isolated-profile launch and read-only page snapshot before authenticated profile grants, Browser Use delegation, or extension experiments.
- 180.0.3.9 Complete basic Control Room device/session/timeline visibility before advanced stream overlays, replay, fleet views, or mobile observer mode.
- 180.0.3.10 Complete the Blender/browser MVP and gateway reliability gates before starting Unreal depth, because Unreal adapter work depends on the same transport, policy, audit, stream, and operator-control substrate.
180.0.4 Golden Vertical Slices#
- 180.0.4.1 Build Slice 1: protocol-only walking skeleton with schemas, fixtures, command envelope validation, result envelope validation, and audit envelope validation.
- 180.0.4.2 Build Slice 2: gateway-host mocked loop where a fake host connects outbound, receives a read-only diagnostic command, returns a result, and emits an audit entry.
- 180.0.4.3 Build Slice 3: device pairing and revocation loop with a fake host, real database records, auth failure tests, and operator-visible device status.
- 180.0.4.4 Build Slice 4: Remote Host local runtime with mocked adapter supervisor, health checks, permission diagnostics, and reconnect behavior.
- 180.0.4.5 Build Slice 5: Blender read-only scene query routed from MCP/CLI through gateway to host to Blender adapter and back to audit.
- 180.0.4.6 Build Slice 6: Blender safe mutation with approval-aware policy, undo checkpoint, before/after scene evidence, and command result visible in the Control Room timeline.
- 180.0.4.7 Build Slice 7: isolated browser session with CDP navigation, DOM snapshot, screenshot, blocked-domain test, and audit timeline entry.
- 180.0.4.8 Build Slice 8: live stream MVP with WebRTC source selection, fallback screenshot mode, stream health metrics, and pause/stop controls.
- 180.0.4.9 Build Slice 9: desktop fallback MVP with focus window, screenshot, click, visual verification, permission-missing state, and emergency stop.
- 180.0.4.10 Build Slice 10: cloud-agent MVP where a Hetzner/AWS agent uses MCP over the gateway to drive the remote MacBook without inbound ports and the operator can watch/stop the session.
- 180.0.4.11 Build Slice 11: Unreal read-only project/world query through the established gateway-host-adapter pattern.
- 180.0.4.12 Build Slice 12: Unreal safe actor spawn/update with transaction/undo, approval where needed, screenshot verification, and audit replay.
180.0.5 Definition of Done for Each Slice#
- 180.0.5.1 Each slice must include a clear README or docs section explaining how to run it locally and what hardware/software assumptions apply.
- 180.0.5.2 Each slice must include schema or contract tests for every new command, event, permission, and audit field.
- 180.0.5.3 Each slice must include failure-path tests, not only happy path tests.
- 180.0.5.4 Each slice must include observability: logs, metrics or trace IDs, and a way to correlate agent request to gateway command to host result.
- 180.0.5.5 Each slice must include policy coverage for allowed, denied, and approval-required cases when the slice can mutate state or access sensitive data.
- 180.0.5.6 Each slice must include cleanup behavior so sessions, streams, helper processes, browser profiles, temp files, and pending commands are not left dangling after tests or manual runs.
- 180.0.5.7 Each slice must include version/capability reporting so future agents know whether the remote device can perform that slice's operations.
- 180.0.5.8 Each slice must include rollback or recovery notes for partially completed commands.
- 180.0.5.9 Each slice must pass repo-relevant lint, typecheck, unit, contract, integration, and UI/e2e checks for the touched code paths.
- 180.0.5.10 Each slice must end with a committed checklist update marking only completed tasks, never aspirational work.
180.0.6 Task Granularity and Parallelization Rules#
- 180.0.6.1 Split implementation issues so one coding agent can finish the issue in one focused session without needing to understand the entire phase.
- 180.0.6.2 Keep write ownership disjoint when multiple agents work in parallel: one agent owns protocol, another gateway, another host runtime, another adapter, another UI, and another tests only when dependencies are stable.
- 180.0.6.3 Do not parallelize dependent layers until the upstream contract is merged; downstream agents should consume stable schemas, not invent their own.
- 180.0.6.4 Allow read-only research/audit tasks to run in parallel with implementation only when they do not modify shared files.
- 180.0.6.5 Require every parallel worker to list changed files, assumptions, verification run, and remaining blockers in its final note.
- 180.0.6.6 Prefer adding explicit fixtures over relying on live services in unit tests.
- 180.0.6.7 Prefer mocked hardware tests in CI and hardware-in-the-loop smoke tests in nightly/manual workflows for Blender, Unreal, and macOS UI.
- 180.0.6.8 Keep adapters replaceable: no gateway or planner code should import Blender, Unreal, Playwright, or macOS runtime packages directly.
- 180.0.6.9 Keep user-facing Control Room changes behind feature flags until stream, command, approval, and audit states are reliable.
- 180.0.6.10 Use checklists as living execution state: split or add subtasks when implementation reveals missing work instead of hiding that complexity in broad unchecked items.
180.1 Product Boundary, Goals, and Operating Model#
180.1.1 Vision and User Outcomes#
- 180.1.1.1 Define the Remote Creative Control Plane product brief: "remote agents can safely operate local creative software with live human visibility, deep integrations first, desktop fallback second, and full auditability."
- 180.1.1.2 Document the two-MacBook Air baseline topology: one machine running Oshun/orchestration and one machine running Blender, Unreal, and browser sessions on the same LAN.
- 180.1.1.3 Document the cloud-agent topology: agents running on Hetzner, AWS, RunPod, or another server connect through the Oshun Remote Gateway to a MacBook on a private network.
- 180.1.1.4 Define first-class controlled surfaces: Blender, Unreal Editor, Chrome/Chromium, file transfer, terminal/process lifecycle, selected macOS desktop actions, and live viewport/screen streaming.
- 180.1.1.5 Define non-goals for the first release: general consumer remote support, uncontrolled full-disk automation, arbitrary remote shell by default, hidden/background browsing through personal profiles, and unapproved destructive file operations.
- 180.1.1.6 Define primary operator personas: solo creator, AI-assisted technical artist, remote build operator, agent supervisor, infrastructure admin, and safety reviewer.
- 180.1.1.7 Define task archetypes: scene assembly, asset import/export, material editing, Unreal map setup, browser research/admin workflows, rendering, build/cook/package, troubleshooting, and visual QA.
- 180.1.1.8 Write a "deep integration first" decision rule: route commands through typed APIs when available; use browser DOM/CDP for web flows; use Computer Use only for native UI gaps, unexpected dialogs, installers, permissions prompts, and verification.
- 180.1.1.9 Define human-in-the-loop moments: first device pairing, profile access, credential entry, destructive changes, external posting, purchases, cloud spend, deletion outside project roots, and deployment.
- 180.1.1.10 Establish measurable product goals: sub-500 ms command routing on LAN, usable live stream under 250 ms one-way LAN latency, reconnect within 5 seconds after network loss, and 100 percent audit coverage for privileged actions.
180.1.2 Domain Ownership and Integration Boundaries#
- 180.1.2.1 Assign Bellona as owner of engine/DCC/browser bridge adapters, remote host packaging, and creative-control command schemas.
- 180.1.2.2 Assign Psyche as owner of desktop fallback primitives: screenshot, OCR/accessibility extraction, click, type, scroll, focus, and visual verification loops.
- 180.1.2.3 Assign Iris/Nous as owners of agent orchestration, model provider routing, planner policies, and tool-selection heuristics.
- 180.1.2.4 Assign Yemaya as owner of production workflows, project context, operator-facing studio/control-room integration, and creative review loops.
- 180.1.2.5 Assign Contracts/Proto/OpenAPI as owners of stable command, event, stream, device, session, permission, and audit schemas.
- 180.1.2.6 Assign Security/Themis/Maat integration responsibility for device trust, policy enforcement, approval workflows, audit retention, and organization-level governance.
- 180.1.2.7 Define integration handoff events between Bellona and Isis: generated asset available, import requested, render requested, artifact exported, output validated, and provenance recorded.
- 180.1.2.8 Define integration handoff events between Bellona and Hathor: world published, narrative package exported, Unreal level update requested, Blender storyboard scene update requested.
- 180.1.2.9 Define integration handoff events between Bellona and Yemaya: production task assigned, agent session started, review requested, operator approval granted, asset sync completed.
- 180.1.2.10 Create an ownership matrix for every module in the target layout, including code owners, test owners, and release owners.
180.1.3 Release Shape and Milestones#
- 180.1.3.1 Define Phase A as "single LAN prototype": one remote host, one gateway, Blender adapter, browser CDP adapter, basic live stream, and MCP tools for agent control.
- 180.1.3.2 Define Phase B as "cloud reachable": outbound tunnel, device pairing, remote gateway, auth, audit, reconnect, and Hetzner/AWS agent access.
- 180.1.3.3 Define Phase C as "Unreal depth": Unreal plugin, editor command routing, map/actor/Blueprint/Sequencer commands, screenshots, and build/cook/package support.
- 180.1.3.4 Define Phase D as "Control Room product": live WebRTC view, timeline, approvals, pause/takeover, replay, artifact panel, and session sharing.
- 180.1.3.5 Define Phase E as "production hardening": signing, auto-update, observability, incident response, enterprise policy, performance budgets, and destructive-action safeguards.
- 180.1.3.6 Define Phase F as "multi-machine creative fleet": multiple remote hosts, engine capability discovery, scheduling, render farm dispatch, role-based sessions, and shared operator rooms.
- 180.1.3.7 Create release gates for each phase with required automated tests, manual verification scripts, security review, and rollback path.
- 180.1.3.8 Create a compatibility matrix for macOS version, Apple Silicon model, Blender versions, Unreal versions, Chrome versions, network mode, and gateway deployment target.
- 180.1.3.9 Define a "demo complete" scenario: cloud agent creates a Blender object, imports it into Unreal, opens a browser production task, and shows all progress in the Control Room.
- 180.1.3.10 Define a "production ready" scenario: a remote host can be paired, controlled, observed, audited, updated, disconnected, recovered, and uninstalled without manual file edits.
180.2 Existing Code Audit and Architecture Alignment#
180.2.1 Bellona Bridge Inventory#
- 180.2.1.1 Audit
libs/bellona/bridge-coremessage envelopes, connection lifecycle, request/response correlation, heartbeat behavior, validation hooks, and reconnect behavior. - 180.2.1.2 Audit
apps/bellona/bridge-blenderto determine which commands are already proxy commands versus local placeholders. - 180.2.1.3 Audit
libs/bellona/blender/python/bellona_addonfor addon server startup, handler registry, permission assumptions, threading model, and Blender main-thread constraints. - 180.2.1.4 Audit
libs/bellona/blender/src/bridgefor command coverage, batch execution, event subscriptions, Python execution helpers, timeout handling, and reconnect behavior. - 180.2.1.5 Audit
libs/bellona/blender-agentto separate existing action-schema, workflow-planning, validation, audit, and MCP-like exports from actual runtime transport gaps. - 180.2.1.6 Audit
apps/bellona/bridge-unrealfor command coverage, engine-client registration, API-client forwarding, timeout policy, and missing Unreal-side plugin assumptions. - 180.2.1.7 Audit
libs/bellona/unrealfor bridge client, launcher, asset tools, project parser/writer, MetaHuman utilities, and version discovery. - 180.2.1.8 Build a bridge gap matrix listing every desired Blender and Unreal command, current implementation status, adapter owner, test status, and production blocker.
- 180.2.1.9 Decide whether bridge services remain standalone apps, become embedded Remote Host modules, or support both modes through a shared adapter interface.
- 180.2.1.10 Create migration notes to avoid duplicating bridge-core concepts inside the new remote protocol.
180.2.2 Psyche Computer Use Inventory#
- 180.2.2.1 Audit
services/psyche/computer-use/actionsfor reusable action models covering screenshot, click, keyboard, clipboard, browser, files, wait, and bash. - 180.2.2.2 Audit
services/psyche/computer-use/visionfor screenshot, OCR, accessibility, UI detection, preprocessing, progress detection, and semantic scene understanding that can run against macOS captures. - 180.2.2.3 Audit
services/psyche/computer-use/screen_sharingfor capture, encoding, WebRTC/RTP assumptions, overlay support, and applicability to macOS windows. - 180.2.2.4 Audit
services/psyche/computer-use/environmentto distinguish Linux/Xvfb container assumptions from host-machine control. - 180.2.2.5 Define a new desktop fallback abstraction that reuses action types without forcing the remote host into an Xvfb/container model.
- 180.2.2.6 Map Codex Computer Use and Claude Computer Use primitives to the shared fallback abstraction: screenshot, click, drag, key, type, scroll, window focus, application state, and visual validation.
- 180.2.2.7 Identify macOS-specific implementation libraries for screenshot capture, Accessibility tree extraction, keyboard/mouse events, window focus, and application activation.
- 180.2.2.8 Create a policy that fallback actions are blocked unless a session has explicit desktop-control permission and visible live stream.
- 180.2.2.9 Define failure reporting when fallback actions cannot run because macOS Screen Recording or Accessibility permission is missing.
- 180.2.2.10 Add test fixtures for fallback action serialization without requiring real desktop interaction in CI.
180.2.3 Gateway, Orchestrator, and Event Bus Alignment#
- 180.2.3.1 Audit
services/psyche/orchestratorsession lifecycle, participant, pipeline, communication, and resource allocation models for reuse in remote-control sessions. - 180.2.3.2 Audit
services/psyche/api-gatewayWebSocket manager for subscription routing, session events, ack/error handling, and metadata patterns. - 180.2.3.3 Decide whether
apps/bellona/remote-gatewayshould embed its own WebSocket server or reuse/extend the API gateway's event routing. - 180.2.3.4 Define event-bus topics for session started, device online, device offline, command queued, command started, command completed, command failed, approval requested, approval resolved, stream started, artifact produced, and session ended.
- 180.2.3.5 Define correlation IDs that link user request, agent plan, MCP tool call, remote command, engine operation, screenshot, artifact, and audit entry.
- 180.2.3.6 Create an idempotency policy for gateway command enqueueing and retry after transport failure.
- 180.2.3.7 Define gateway stateless versus stateful responsibilities: stateless command validation, stateful active sessions, durable audit, and recoverable command queues.
- 180.2.3.8 Define horizontal scaling behavior for multiple gateway instances and sticky routing for active WebRTC/control sessions.
- 180.2.3.9 Define backpressure behavior when the remote host is busy, offline, rendering, compiling shaders, or blocked on human approval.
- 180.2.3.10 Create a compatibility plan for existing Bellona bridge clients so current commands can be proxied through the remote gateway.
180.3 Protocols, Contracts, and Data Model#
180.3.1 Remote Protocol Core#
- 180.3.1.1 Create
libs/bellona/remote-protocolwith strict TypeScript types and Zod schemas for device, session, command, event, stream, artifact, permission, approval, and audit envelopes. - 180.3.1.2 Define
RemoteDevicewith stable device ID, display name, host kind, OS version, hardware model, architecture, installed adapters, capabilities, trust state, and last seen timestamp. - 180.3.1.3 Define
RemoteSessionwith session ID, organization ID, project ID, actor ID, agent ID, remote device ID, requested surfaces, live stream state, permissions, and lifecycle status. - 180.3.1.4 Define
RemoteCommandEnvelopewith command ID, session ID, target surface, command type, schema version, payload, timeout, priority, idempotency key, approval policy, and expected result type. - 180.3.1.5 Define
RemoteCommandResultwith success status, structured data, stderr/stdout where applicable, screenshot references, artifact references, duration, adapter version, warnings, and error codes. - 180.3.1.6 Define
RemoteEventEnvelopefor asynchronous notifications from host to gateway: state changed, command progress, engine event, browser event, permission missing, stream keyframe, artifact ready, and heartbeat. - 180.3.1.7 Define
RemoteArtifactfor screenshots, renders, exported assets, logs, traces, generated project files, crash reports, and command replay bundles. - 180.3.1.8 Define
RemoteStreamDescriptorfor WebRTC session metadata, codec, resolution, frame rate, window/app/screen source, TURN usage, and recording policy. - 180.3.1.9 Define
RemoteApprovalRequestwith action summary, risk class, affected files/apps/accounts, policy reason, deadline, and allowed operator responses. - 180.3.1.10 Generate JSON Schema, OpenAPI fragments, and Proto definitions from the shared protocol source or keep them synchronized via contract-generation tests.
180.3.2 Command Taxonomy#
- 180.3.2.1 Define command namespaces:
device.*,session.*,stream.*,blender.*,unreal.*,browser.*,desktop.*,file.*,process.*,approval.*,artifact.*, anddiagnostic.*. - 180.3.2.2 Define
device.capabilitiesto report installed apps, adapter versions, engine versions, macOS permissions, display layout, network state, and resource availability. - 180.3.2.3 Define
session.start,session.pause,session.resume,session.stop,session.takeover, andsession.release_takeover. - 180.3.2.4 Define
stream.start,stream.stop,stream.switch_source,stream.set_quality,stream.record_start, andstream.record_stop. - 180.3.2.5 Define
file.transfer_to_host,file.transfer_from_host,file.list_project_root,file.hash,file.snapshot,file.restore_snapshot, andfile.open_in_app. - 180.3.2.6 Define
process.launch_app,process.quit_app,process.restart_adapter,process.health,process.tail_log, andprocess.collect_diagnostics. - 180.3.2.7 Define
desktop.screenshot,desktop.focus_window,desktop.click,desktop.drag,desktop.type,desktop.key,desktop.scroll,desktop.clipboard_set, anddesktop.wait_for_visual. - 180.3.2.8 Define
approval.request,approval.resolve,approval.cancel, andapproval.policy_explain. - 180.3.2.9 Define
diagnostic.self_test,diagnostic.network_test,diagnostic.permission_test,diagnostic.adapter_test, anddiagnostic.bundle. - 180.3.2.10 Create a compatibility registry that maps older Bellona bridge commands to new command namespaces.
180.3.3 Persistent Data Model#
- 180.3.3.1 Add database schema for remote devices, device keys, pairing codes, trusted gateways, installed adapters, capability snapshots, and device health history.
- 180.3.3.2 Add database schema for remote sessions, active surfaces, stream records, participants, agent ownership, operator takeover, and lifecycle transitions.
- 180.3.3.3 Add database schema for command ledger entries, command attempts, retry state, approval state, result summaries, and artifact pointers.
- 180.3.3.4 Add database schema for audit entries with immutable correlation IDs, actor identity, policy decision, command hash, result hash, and evidence links.
- 180.3.3.5 Add database schema for project root grants, allowed file paths, denied paths, external account grants, and browser profile grants.
- 180.3.3.6 Add database schema for device software versions, auto-update channels, rollout cohorts, install status, crash telemetry, and rollback state.
- 180.3.3.7 Add retention policies for screenshots, video recordings, command payloads, browser traces, engine logs, and sensitive artifacts.
- 180.3.3.8 Add redaction flags for secrets, credentials, personal browser data, protected files, and excluded app windows.
- 180.3.3.9 Add migration tests proving old bridge sessions and new remote sessions can coexist.
- 180.3.3.10 Build seed/test factories for remote devices, sessions, commands, approvals, streams, and artifacts.
180.4 Security, Trust, and Policy#
180.4.1 Device Pairing and Identity#
- 180.4.1.1 Design one-time device pairing with a short code or QR code shown locally on the Remote Host and confirmed in the gateway.
- 180.4.1.2 Issue per-device asymmetric keys during pairing and store private keys in the macOS Keychain.
- 180.4.1.3 Require mTLS or signed session tokens for every host-gateway connection after pairing.
- 180.4.1.4 Bind each device certificate to organization, device ID, user account, installation fingerprint, and allowed gateway origins.
- 180.4.1.5 Implement device revocation from the gateway with immediate disconnect and local trust-state update.
- 180.4.1.6 Implement local unpairing from the Remote Host menu bar app that removes keys, stops tunnels, and disables control.
- 180.4.1.7 Require explicit operator confirmation before accepting a new gateway URL or changing organization ownership.
- 180.4.1.8 Detect cloned device credentials by tracking impossible concurrent connections and hardware fingerprint changes.
- 180.4.1.9 Rotate device keys on schedule and after sensitive incidents without requiring full app reinstall.
- 180.4.1.10 Add tests for pairing success, expired pairing code, reused pairing code, revoked device, rotated key, and mismatched gateway.
180.4.2 Permission and Approval Model#
- 180.4.2.1 Define permission scopes for each surface:
blender.read,blender.write,blender.python,unreal.read,unreal.write,unreal.console,browser.read,browser.write,browser.profile,desktop.view,desktop.control,files.read,files.write,process.launch, andshell.run. - 180.4.2.2 Define risk classes:
observe_only,project_local_write,engine_state_mutation,external_network,credential_sensitive,destructive,financial,publishing,privileged_system, andunknown. - 180.4.2.3 Require approval for commands that delete files, run shell, execute arbitrary Blender Python, execute Unreal console commands outside an allowlist, access real browser profiles, or publish externally.
- 180.4.2.4 Implement policy evaluation before queueing commands and again on the host before execution.
- 180.4.2.5 Implement local host-side deny rules that cannot be overridden by a compromised gateway.
- 180.4.2.6 Add per-project allowed roots and block access to home directories, credentials, SSH keys, browser password stores, system settings, and unrelated repositories by default.
- 180.4.2.7 Add an operator "pause all agents" control that immediately stops command execution while preserving streams and diagnostics.
- 180.4.2.8 Add an operator "take over" mode that blocks new agent input and lets the human use the local/remote UI.
- 180.4.2.9 Add approval timeout behavior: deny by default after a configurable timeout unless the risk class is observe-only.
- 180.4.2.10 Add policy unit tests for every risk class and permission scope combination.
180.4.3 Secrets, Browser Data, and Prompt-Injection Defense#
- 180.4.3.1 Treat browser automation as prompt-injection exposed and require browser task isolation, origin tracking, and tool-output sanitization.
- 180.4.3.2 Block agents from reading password managers, browser saved passwords, cookies, localStorage, and session tokens unless a task has an explicit scoped profile grant.
- 180.4.3.3 Prefer dedicated automation browser profiles over personal Chrome profiles; require explicit consent for real-profile reuse.
- 180.4.3.4 Implement visible credential-entry mode where the agent can wait while the human types secrets locally and the stream can be blurred.
- 180.4.3.5 Add origin allowlists and denylist enforcement for browser tasks that can access authenticated accounts.
- 180.4.3.6 Strip or summarize untrusted web page content before passing it to privileged planner prompts.
- 180.4.3.7 Add confirmation gates before submitting forms, sending messages, making purchases, uploading files, or changing account settings.
- 180.4.3.8 Store remote host tokens in Keychain, gateway secrets in the platform secret manager, and session-scoped tokens with short TTLs.
- 180.4.3.9 Add redaction to logs, screenshots, traces, and command payloads for detected tokens, API keys, passwords, and personal data.
- 180.4.3.10 Build adversarial browser tests where a web page instructs the agent to exfiltrate secrets, bypass policy, or ignore Oshun rules.
180.4.4 Supply Chain, Signing, and Updates#
- 180.4.4.1 Code-sign and notarize the macOS Remote Host app, helper tools, launch agent, and browser/engine plugin packages.
- 180.4.4.2 Define update channels:
dev,internal,beta,stable, andpinned. - 180.4.4.3 Require signed update manifests and cryptographic checksum validation before installing host, adapter, or plugin updates.
- 180.4.4.4 Implement staged rollout with canary devices and automatic rollback on crash-loop, failed self-test, or gateway incompatibility.
- 180.4.4.5 Build SBOM generation for host app, Node/TS services, Python Blender addon, Unreal plugin, browser adapter, and native helpers.
- 180.4.4.6 Add dependency vulnerability scans and license checks to release gates.
- 180.4.4.7 Implement plugin compatibility checks before updating Blender or Unreal adapters.
- 180.4.4.8 Store previous adapter versions so the operator can roll back without reinstalling the full host app.
- 180.4.4.9 Add tamper detection for modified host binaries, unsigned helper tools, and unexpected plugin files.
- 180.4.4.10 Add release runbooks for signing certificate rotation, compromised update key, and emergency update revocation.
180.5 Networking, Transport, and Connectivity#
180.5.1 LAN Mode#
- 180.5.1.1 Implement mDNS/Bonjour discovery for Remote Host devices on the same local network.
- 180.5.1.2 Require device identity verification after discovery so mDNS is convenience only, not trust.
- 180.5.1.3 Support direct TLS WebSocket command routing between controller and host when both are on the same LAN and policy allows it.
- 180.5.1.4 Support direct WebRTC media between Control Room and host on LAN with gateway-assisted signaling.
- 180.5.1.5 Detect local network changes and fail over from direct LAN transport to outbound gateway tunnel without losing the session ledger.
- 180.5.1.6 Add LAN latency, packet loss, bandwidth, and jitter telemetry to session diagnostics.
- 180.5.1.7 Provide a "LAN only" mode for air-gapped/local studio workflows.
- 180.5.1.8 Provide a "disable LAN discovery" mode for locked-down enterprise networks.
- 180.5.1.9 Test LAN mode across home Wi-Fi, wired Ethernet, guest Wi-Fi isolation, and VPN-active conditions.
- 180.5.1.10 Document router/firewall behavior required for direct LAN streaming and command transport.
180.5.2 Remote Network and Cloud Mode#
- 180.5.2.1 Implement host-initiated outbound tunnel to the Remote Gateway over port 443 so no inbound port is required on the MacBook.
- 180.5.2.2 Support WebSocket over TLS as the initial reliable control transport.
- 180.5.2.3 Evaluate QUIC/WebTransport for future lower-latency, multiplexed control and media metadata.
- 180.5.2.4 Implement reconnect with exponential backoff, session resume tokens, command replay safety, and duplicate command suppression.
- 180.5.2.5 Support multiple gateway regions or URLs with pinned trust and failover priority.
- 180.5.2.6 Support TURN relay for WebRTC media when direct peer connection fails.
- 180.5.2.7 Support Tailscale/WireGuard as an optional private-network deployment mode without making it mandatory.
- 180.5.2.8 Add network impairment tests: high latency, packet loss, gateway restart, host sleep/wake, NAT change, Wi-Fi disconnect, and laptop lid close.
- 180.5.2.9 Define cloud egress and TURN cost budgets for long live stream sessions.
- 180.5.2.10 Add a cloud deployment reference for Hetzner and AWS with TLS termination, TURN, observability, and secrets management.
180.5.3 Transport Multiplexing and Backpressure#
- 180.5.3.1 Separate logical channels for control commands, events, logs, file transfer, artifact metadata, and stream signaling.
- 180.5.3.2 Ensure large file transfers cannot starve control commands or approval messages.
- 180.5.3.3 Add per-channel flow control and bounded queues on both gateway and host.
- 180.5.3.4 Define command priority levels: emergency stop, approval, observe, interactive control, engine command, file transfer, batch job, and background telemetry.
- 180.5.3.5 Add host busy states for rendering, shader compile, build, file import, CPU pressure, memory pressure, battery low, thermal pressure, and operator takeover.
- 180.5.3.6 Implement cancellation for queued and running commands when adapter support exists.
- 180.5.3.7 Implement command timeout classes by adapter and risk: quick query, interactive edit, render, build, transfer, and diagnostics.
- 180.5.3.8 Add resumable file transfer for large assets and render outputs.
- 180.5.3.9 Add host-side disk-space checks before receiving files or triggering renders/builds.
- 180.5.3.10 Add backpressure integration with agent planners so agents do not keep issuing edits while the host is saturated.
180.6 Remote Host macOS App#
180.6.1 Host Runtime and Process Model#
- 180.6.1.1 Create
apps/bellona/remote-hostas the installable macOS host app with a menu bar status UI and background service. - 180.6.1.2 Create a LaunchAgent that starts the host runtime after user login and restarts it after crashes.
- 180.6.1.3 Implement a local supervisor that manages Blender adapter, Unreal adapter, browser adapter, desktop fallback helper, stream helper, and file transfer helper processes.
- 180.6.1.4 Keep engine/browser debug ports bound to
127.0.0.1and expose them only through the authenticated host runtime. - 180.6.1.5 Implement host health checks for CPU, memory, disk, battery, thermal pressure, network, app availability, permissions, and adapter state.
- 180.6.1.6 Implement local logs with rotation and a one-click diagnostic bundle.
- 180.6.1.7 Implement graceful shutdown that drains or cancels commands, stops streams, disconnects tunnels, and records final audit state.
- 180.6.1.8 Implement sleep/wake handling and session recovery after macOS resumes.
- 180.6.1.9 Implement a local allowlist of controllable apps and block automation of excluded apps such as password managers by default.
- 180.6.1.10 Add host runtime unit tests with mocked adapters and lifecycle transitions.
180.6.2 macOS Permissions and UX#
- 180.6.2.1 Build permission onboarding for Screen Recording, Accessibility, Automation, Files and Folders, local network access, and microphone/camera only if future voice/video needs it.
- 180.6.2.2 Detect permission state at startup and show exact missing permissions in the host UI and gateway diagnostics.
- 180.6.2.3 Provide guided deep links to macOS Settings panes where supported.
- 180.6.2.4 Block desktop fallback control until Accessibility permission is granted.
- 180.6.2.5 Block live desktop/window stream until Screen Recording permission is granted.
- 180.6.2.6 Provide a visible local status indicator when a remote session is active.
- 180.6.2.7 Provide local pause, stop session, unpair, collect diagnostics, and quit controls.
- 180.6.2.8 Provide per-session prompts when an agent requests expanded permissions.
- 180.6.2.9 Add accessibility labels and keyboard navigability to the host UI.
- 180.6.2.10 Test permission onboarding on clean macOS installs, upgrades, revoked permissions, and MDM-managed machines.
180.6.3 App and Adapter Discovery#
- 180.6.3.1 Detect installed Blender versions, binary paths, Python compatibility, addon status, and user preferences path.
- 180.6.3.2 Detect installed Unreal Engine versions, Epic launcher paths,
.uprojectassociations, plugin status, and project roots. - 180.6.3.3 Detect Chrome/Chromium/Brave/Edge availability and CDP launch support.
- 180.6.3.4 Detect Playwright browser installations and install status for managed automation profiles.
- 180.6.3.5 Detect existing Bellona bridge services and report whether they can be reused or should be managed by Remote Host.
- 180.6.3.6 Report adapter capability differences by engine/app version to the gateway.
- 180.6.3.7 Provide install/repair actions for Blender addon, Unreal plugin, and browser automation helper.
- 180.6.3.8 Verify adapters by running a read-only smoke command after install.
- 180.6.3.9 Maintain an adapter inventory cache and refresh it on app launch, app update, and user request.
- 180.6.3.10 Add tests for missing apps, multiple app versions, unsupported versions, broken addon/plugin install, and repaired install.
180.7 Blender Deep-Control Adapter#
180.7.1 Blender Transport and Addon Runtime#
- 180.7.1.1 Convert the existing Blender addon server into a Remote Host managed adapter with authenticated localhost-only transport.
- 180.7.1.2 Ensure all Blender API calls execute safely on Blender's main thread or through Blender-supported scheduling primitives.
- 180.7.1.3 Add a versioned handshake between host and Blender addon reporting Blender version, addon version, enabled features, scene path, and active project root.
- 180.7.1.4 Add heartbeat and reconnect when Blender restarts or the addon reloads.
- 180.7.1.5 Add read-only mode for observation and scene queries.
- 180.7.1.6 Add command timeout and cancellation where Blender operators permit it.
- 180.7.1.7 Add per-command undo checkpoints using Blender undo where possible.
- 180.7.1.8 Add dry-run support for planned operations that can be represented without mutating the scene.
- 180.7.1.9 Add command tracing with operator name, object IDs, material IDs, file paths, and result references.
- 180.7.1.10 Add addon runtime tests using headless Blender in CI where feasible and mocked addon tests where headless Blender is unavailable.
180.7.2 Scene, Object, and Asset Commands#
- 180.7.2.1 Implement
blender.scene.infowith scenes, collections, objects, materials, cameras, lights, frame range, render engine, and file path. - 180.7.2.2 Implement
blender.object.create_primitivefor mesh primitives with transform, collection, material, and naming options. - 180.7.2.3 Implement
blender.object.delete,blender.object.rename,blender.object.duplicate,blender.object.parent,blender.object.set_transform, andblender.object.set_visibility. - 180.7.2.4 Implement
blender.collection.create,blender.collection.move_object, and collection visibility/renderability controls. - 180.7.2.5 Implement
blender.material.create_principled,blender.material.assign,blender.material.set_parameter, andblender.material.find_by_semantics. - 180.7.2.6 Implement
blender.asset.importfor GLB/GLTF, FBX, OBJ, USD/USDZ where supported, Alembic, image textures, and audio references. - 180.7.2.7 Implement
blender.asset.exportfor GLB/GLTF, FBX, USD, OBJ, Alembic, image sequences, and.blendpackaging. - 180.7.2.8 Implement
blender.file.open,blender.file.save,blender.file.save_as,blender.file.pack_resources, andblender.file.recover_last. - 180.7.2.9 Implement semantic selectors for "all emissive objects", "selected character rig", "large static props", "objects in collection", and "materials missing textures."
- 180.7.2.10 Add golden-scene tests validating object creation, material assignment, import/export, undo, and scene-info round trips.
180.7.3 Advanced Blender Automation#
- 180.7.3.1 Expose safe, structured Geometry Nodes graph creation for common workflows: scatter, cable runs, kitbash arrays, modular facades, terrain dressing, and procedural props.
- 180.7.3.2 Expose modifier stack operations: add, configure, reorder, apply, disable viewport, disable render, and snapshot before apply.
- 180.7.3.3 Expose mesh edit macros for selected safe operations: bevel, extrude, inset, merge by distance, normal recalculation, UV unwrap, decimate, and triangulate.
- 180.7.3.4 Expose camera and lighting presets for product shot, three-point lighting, HDRI lookdev, cinematic blocking, orthographic asset preview, and turntable render.
- 180.7.3.5 Expose render commands for viewport screenshot, OpenGL preview, Cycles/Eevee still render, playblast, and thumbnail capture.
- 180.7.3.6 Expose rigging macros for armature creation, IK/FK setup, retarget prep, control shapes, and pose library operations with approval.
- 180.7.3.7 Expose simulation prep macros for cloth, rigid body, soft body, particle, fluid, and smoke as guided setup, not uncontrolled bake.
- 180.7.3.8 Expose Python execution only as a privileged command with approval, project-root constraints, static preflight checks, and audit.
- 180.7.3.9 Integrate
libs/bellona/blender-agentvalidation rules before applying high-level automation plans. - 180.7.3.10 Add benchmark scenes for performance and correctness of advanced macros.
180.8 Unreal Engine Deep-Control Adapter#
180.8.1 Unreal Plugin and Transport#
- 180.8.1.1 Build or complete the Unreal Editor plugin that connects to the local Remote Host adapter over authenticated localhost transport.
- 180.8.1.2 Implement plugin install/repair/uninstall from the Remote Host app for project-level and engine-level plugin deployment.
- 180.8.1.3 Add handshake reporting Unreal version, project path, plugin version, editor mode, map name, world type, PIE state, and enabled capabilities.
- 180.8.1.4 Support reconnect when the editor restarts, project reloads, or plugin hot reloads.
- 180.8.1.5 Route commands through Unreal's editor-safe APIs rather than UI automation where possible.
- 180.8.1.6 Define main-thread/task-graph execution requirements for editor mutations.
- 180.8.1.7 Add timeout and progress reporting for slow operations: asset import, shader compile, map load, cook, package, and render.
- 180.8.1.8 Add transaction/undo integration through Unreal editor transactions.
- 180.8.1.9 Add read-only observation mode for projects where mutation is not approved.
- 180.8.1.10 Add automated plugin tests using Unreal automation where feasible, plus protocol-level tests with a mocked editor adapter.
180.8.2 World, Level, Actor, and Asset Commands#
- 180.8.2.1 Implement
unreal.project.infowith project path, engine version, plugins, maps, content roots, source-control state, and build target metadata. - 180.8.2.2 Implement
unreal.map.open,unreal.map.save,unreal.map.create, andunreal.map.list. - 180.8.2.3 Implement
unreal.world.querywith actors, components, levels, data layers, world partition status, cameras, lights, and selected objects. - 180.8.2.4 Implement
unreal.actor.spawn,unreal.actor.destroy,unreal.actor.set_transform,unreal.actor.set_property,unreal.actor.attach, andunreal.actor.select. - 180.8.2.5 Implement
unreal.blueprint.call_functionwith schema validation and allowlist support. - 180.8.2.6 Implement
unreal.material.set_parameterfor scalar, vector, texture, and runtime material instance targets. - 180.8.2.7 Implement
unreal.asset.importfor GLB/FBX/USD/textures andunreal.asset.reimportfor source asset updates. - 180.8.2.8 Implement
unreal.asset.find,unreal.asset.rename,unreal.asset.move,unreal.asset.delete, and redirector fixup with approval for destructive actions. - 180.8.2.9 Implement
unreal.console.executewith an allowlist by default and approval for arbitrary console commands. - 180.8.2.10 Add project fixture tests for map open/save, actor spawn, property mutation, material update, asset import, and undo.
180.8.3 Sequencer, Rendering, Build, and Virtual Production Hooks#
- 180.8.3.1 Implement
unreal.sequencer.list,unreal.sequencer.open,unreal.sequencer.set_frame,unreal.sequencer.play, andunreal.sequencer.render_preview. - 180.8.3.2 Implement
unreal.camera.capture_screenshotfor viewport, active camera, and named camera actors. - 180.8.3.3 Implement Movie Render Queue/Movie Render Graph job creation, launch, progress, cancel, and artifact collection.
- 180.8.3.4 Implement editor play-in-editor controls:
pie.start,pie.stop,pie.eject, and basic input injection where allowed. - 180.8.3.5 Implement build/cook/package commands with platform target, configuration, maps, output path, logs, progress, and cancellation.
- 180.8.3.6 Implement shader compile and derived data cache status reporting so agents can wait intelligently.
- 180.8.3.7 Implement virtual production hooks for Live Link status, nDisplay/Switchboard observation, camera tracking state, and stage monitor screenshots as later-stage capabilities.
- 180.8.3.8 Implement diagnostic collection for Unreal logs, crash reports, plugin logs, project settings, and adapter telemetry.
- 180.8.3.9 Add performance benchmarks for map query, actor mutation, screenshot capture, asset import, and render command overhead.
- 180.8.3.10 Add integration tests using a small Unreal fixture project and a mocked heavy-project response model.
180.9 Browser Automation Adapter#
180.9.1 Browser Control Strategy#
- 180.9.1.1 Implement a managed Chromium/Chrome automation profile launched with CDP enabled and isolated from personal browsing by default.
- 180.9.1.2 Implement
browser.navigate,browser.back,browser.forward,browser.reload,browser.new_tab,browser.close_tab, andbrowser.switch_tab. - 180.9.1.3 Implement DOM-aware commands:
browser.snapshot,browser.click_element,browser.type_into,browser.select,browser.submit, andbrowser.wait_for. - 180.9.1.4 Implement screenshot and screencast capture for active tab and browser window.
- 180.9.1.5 Implement download and upload handling with project-root restrictions.
- 180.9.1.6 Implement browser profile grants for persistent auth with explicit operator consent.
- 180.9.1.7 Implement site-specific policy controls for allowed domains, denied domains, form-submission approval, and file-upload approval.
- 180.9.1.8 Implement accessibility tree extraction where available to improve agent grounding.
- 180.9.1.9 Implement browser trace recording for debugging failed automations.
- 180.9.1.10 Add Playwright/CDP tests for navigation, clicking, typing, downloads, uploads, screenshots, profile isolation, and blocked domains.
180.9.2 Browser Use and External Browser-Agent Integration#
- 180.9.2.1 Evaluate Browser Use as an optional high-level browser agent for messy web workflows and compare it against direct Playwright/CDP command reliability.
- 180.9.2.2 Add an adapter interface that can route a browser task to direct CDP, Browser Use open-source local mode, Browser Use cloud mode, or another MCP browser provider.
- 180.9.2.3 Support remote CDP URLs as a browser target when a browser is running on the controlled MacBook or a cloud browser provider.
- 180.9.2.4 Preserve Control Room visibility even when Browser Use or another agent is driving the browser.
- 180.9.2.5 Normalize browser-agent step events into the remote command timeline.
- 180.9.2.6 Add policy gates before delegating authenticated browser sessions to third-party hosted browser infrastructure.
- 180.9.2.7 Add a preference order: direct CDP for deterministic tasks, Browser Use for ambiguous web tasks, Computer Use fallback for browser UI chrome or extension dialogs.
- 180.9.2.8 Add tests that verify Browser Use integration cannot bypass Oshun approval gates.
- 180.9.2.9 Add task templates for production-board updates, asset marketplace search, documentation lookup, account-free research, and authenticated admin workflows.
- 180.9.2.10 Document the cost, privacy, auth, and reliability tradeoffs of direct CDP versus Browser Use cloud versus local Browser Use.
180.9.3 Chrome Extension Consideration#
- 180.9.3.1 Evaluate whether a Chrome extension adds value beyond CDP: DOM extraction, page overlays, human consent UI, profile-aware grants, and site-specific guardrails.
- 180.9.3.2 Avoid requiring an extension for basic automation because CDP/Playwright is lower friction for managed browser sessions.
- 180.9.3.3 If an extension is built, restrict it to explicit allowlisted profiles and show a visible automation indicator.
- 180.9.3.4 Require extension manifest permissions to be minimal, auditable, and domain-scoped where possible.
- 180.9.3.5 Add extension-origin authentication so web pages cannot impersonate the host or gateway.
- 180.9.3.6 Add extension tests for prompt-injection pages, malicious iframes, clipboard access, file upload prompts, and form submission.
- 180.9.3.7 Add an uninstall/disable path from the Remote Host UI.
- 180.9.3.8 Keep extension data out of command logs unless explicitly needed and redacted.
- 180.9.3.9 Document when extension mode is appropriate: authenticated personal browser assistance, overlays, or web-app-specific affordances.
- 180.9.3.10 Keep extension mode optional until CDP and Browser Use paths prove insufficient.
180.10 Desktop Fallback and Computer Use#
180.10.1 Desktop Observation#
- 180.10.1.1 Implement macOS screenshot capture for full display, selected display, selected app window, and selected browser/engine viewport.
- 180.10.1.2 Implement window inventory with app name, window title, bounds, display, visibility, minimized state, and focus state.
- 180.10.1.3 Implement application activation and window focus for Blender, Unreal, Chrome, Finder, terminal, and permission dialogs.
- 180.10.1.4 Implement OCR and UI element detection over captured screenshots where Accessibility tree is unavailable.
- 180.10.1.5 Implement Accessibility tree extraction for native macOS UI where permissions and APIs permit it.
- 180.10.1.6 Implement visual wait conditions: text appears, window appears, pixel region changes, progress bar completes, modal disappears.
- 180.10.1.7 Implement sensitive-window exclusion so password managers, private documents, and denied apps are blanked or hidden in streams and screenshots.
- 180.10.1.8 Add screenshot redaction for detected secrets and excluded regions.
- 180.10.1.9 Add visual evidence capture before and after fallback actions.
- 180.10.1.10 Add tests for screenshot capture shape, redaction, window selection, focus targeting, and visual wait serialization.
180.10.2 Desktop Control#
- 180.10.2.1 Implement click, double click, right click, drag, scroll, key press, hotkey, text input, and clipboard operations through macOS Accessibility/event APIs.
- 180.10.2.2 Implement coordinate normalization across Retina scaling, multiple displays, window-relative coordinates, and stream coordinates.
- 180.10.2.3 Add movement/click overlays to the live stream so the operator can see what the agent is doing.
- 180.10.2.4 Add rate limits to prevent runaway clicking or typing.
- 180.10.2.5 Add emergency stop that interrupts active desktop actions.
- 180.10.2.6 Add fallback action approval for anything outside allowlisted apps/windows.
- 180.10.2.7 Add "visual verification required" for fallback mutations: screenshot before, action, screenshot after, compare expected result.
- 180.10.2.8 Add fallback recipes for common native UI gaps: file dialogs, permissions prompts, app update prompts, plugin enable dialogs, and crash dialogs.
- 180.10.2.9 Add fallback failure classification: missing permission, wrong window, stale coordinates, unexpected modal, inaccessible element, and visual mismatch.
- 180.10.2.10 Add regression tests with mocked screenshots and known UI patterns.
180.10.3 Codex and Claude Computer Use Compatibility#
- 180.10.3.1 Map Anthropic-style Computer Use tool actions to the Remote Host desktop fallback commands.
- 180.10.3.2 Map Codex Computer Use style actions to the same fallback command layer.
- 180.10.3.3 Add an adapter that can expose the remote host as a computer-use environment to agents while preserving Oshun policy gates.
- 180.10.3.4 Add a tool-selection policy that discourages pixel control when Blender/Unreal/CDP structured commands exist.
- 180.10.3.5 Add visual action transcripts compatible with agent debugging: screenshot ID, action, coordinates, target window, and result.
- 180.10.3.6 Add an "ask human to switch/control locally" action when Computer Use confidence is low or UI is sensitive.
- 180.10.3.7 Add tests that a Computer Use agent cannot bypass file, browser, or app policy by clicking around direct APIs.
- 180.10.3.8 Add latency budget measurements for screenshot-action-loop control on LAN and cloud.
- 180.10.3.9 Add session replay that can reconstruct Computer Use actions from screenshots and event logs.
- 180.10.3.10 Document when to use Computer Use: visual inspection, native UI gaps, unexpected dialogs, manual app workflows, and final verification.
180.11 MCP Gateway and Agent Tooling#
180.11.1 MCP Server#
- 180.11.1.1 Create
libs/bellona/mcp-gatewayexposing remote-control tools through MCP over stdio for local clients and Streamable HTTP for remote/cloud clients. - 180.11.1.2 Implement MCP authentication and session binding so an MCP client can only access permitted devices and projects.
- 180.11.1.3 Expose tool discovery grouped by device and adapter: device, session, stream, Blender, Unreal, browser, desktop, files, process, approvals, and diagnostics.
- 180.11.1.4 Expose MCP resources for current scene state, Unreal world state, browser page snapshot, active stream metadata, command timeline, and recent artifacts.
- 180.11.1.5 Expose MCP prompts for safe creative automation: "inspect before edit", "plan with dry-run", "request approval", "prefer deep integration", and "produce replayable result."
- 180.11.1.6 Add tool schemas that are narrow and typed rather than generic "run arbitrary command" tools.
- 180.11.1.7 Add MCP notifications for command progress, approval needed, stream state, and host offline.
- 180.11.1.8 Add compatibility tests with Claude Code, Codex MCP clients where available, and a reference MCP test harness.
- 180.11.1.9 Add rate limits and budget controls per MCP client and agent session.
- 180.11.1.10 Document local and cloud MCP setup, including Hetzner/AWS deployment, tokens, and device grants.
180.11.2 Agent Orchestration#
- 180.11.2.1 Define an agent planner policy that chooses tools in this order: domain API, engine/browser adapter, high-level Browser Use agent, desktop fallback, human request.
- 180.11.2.2 Add a planning preflight that checks device online state, permissions, app availability, project root, stream availability, and approval requirements.
- 180.11.2.3 Add planner support for long-running remote jobs with progress polling, cancellation, and checkpointing.
- 180.11.2.4 Add planner support for visual verification after every high-impact edit.
- 180.11.2.5 Add planner support for dry-run plans and human-readable impact summaries before risky actions.
- 180.11.2.6 Add multi-agent locking so two agents do not concurrently mutate the same Blender scene, Unreal map, browser profile, or file root without a coordinator.
- 180.11.2.7 Add agent handoff between cloud model, local model, browser specialist, Blender specialist, Unreal specialist, and human operator.
- 180.11.2.8 Add structured memory of device/app capabilities so agents stop repeatedly probing unchanged state.
- 180.11.2.9 Add autonomous stop criteria: task done, approval denied, policy blocked, repeated failure, host busy too long, or low confidence.
- 180.11.2.10 Add evaluation tasks measuring agent success with and without deep integration versus Computer Use-only baselines.
180.11.3 External Agent Clients#
- 180.11.3.1 Provide setup docs for Claude Code MCP access to the Remote Gateway.
- 180.11.3.2 Provide setup docs for Codex/OpenAI-agent access to the Remote Gateway.
- 180.11.3.3 Provide setup docs for local developer scripts using the TypeScript SDK.
- 180.11.3.4 Provide setup docs for Python automation using generated OpenAPI/Proto clients.
- 180.11.3.5 Provide an agent-safe CLI for starting sessions, listing devices, starting streams, sending commands, and tailing timelines.
- 180.11.3.6 Provide sample workflows: Blender create-and-render, Unreal import-and-place, browser research-and-update, and fallback dialog handling.
- 180.11.3.7 Provide a hosted agent reference deployment on Hetzner with gateway URL, MCP auth, and TURN configuration.
- 180.11.3.8 Provide an AWS reference deployment with ECS/Fargate or EC2, load balancer, Redis/Postgres, object storage, and observability.
- 180.11.3.9 Provide local-only scripts for the two-MacBook LAN setup without cloud dependency.
- 180.11.3.10 Add contract tests proving every external client sees the same tool schema and command behavior.
180.12 Live Control Room and Operator Experience#
180.12.1 Control Room Core#
- 180.12.1.1 Create
apps/bellona/control-roomor integrate equivalent views into Yemaya Studio Desktop/Web, depending on product direction. - 180.12.1.2 Show a device list with online/offline state, active sessions, installed adapters, missing permissions, and health.
- 180.12.1.3 Show live stream tiles for Blender, Unreal, browser, or full desktop with source switching.
- 180.12.1.4 Show current agent task, plan, active tool call, pending approval, and last result.
- 180.12.1.5 Show command timeline with filters by adapter, risk class, status, actor, artifact, and approval state.
- 180.12.1.6 Show artifact panel for screenshots, renders, exports, logs, traces, and replay bundles.
- 180.12.1.7 Provide pause, resume, stop, takeover, release takeover, approve, deny, and request clarification controls.
- 180.12.1.8 Provide a clear local/remote control state indicator so the operator knows whether the agent or human is currently driving.
- 180.12.1.9 Provide session notes and bookmark important moments in the timeline.
- 180.12.1.10 Add responsive layouts for laptop, external monitor, and tablet-sized operator views.
180.12.2 Live Stream and Interaction Design#
- 180.12.2.1 Implement WebRTC playback with adaptive quality, reconnect, freeze detection, stream statistics, and TURN fallback indication.
- 180.12.2.2 Overlay agent cursor, clicks, drags, keyboard actions, selected window, and current target element/object where possible.
- 180.12.2.3 Overlay engine-specific hints: selected Blender object, active Unreal actor, current browser element, or blocked permission.
- 180.12.2.4 Provide stream source modes: app window, display, browser tab, Blender viewport, Unreal viewport, and screenshot-only low-bandwidth mode.
- 180.12.2.5 Provide privacy controls: blur excluded windows, hide desktop outside target app, pause video during credential entry, and disable recording.
- 180.12.2.6 Provide human remote-control input only after explicit takeover and policy permission.
- 180.12.2.7 Provide non-invasive observation mode where the operator can watch and comment without affecting the agent.
- 180.12.2.8 Provide low-latency mode for LAN and quality mode for remote/cloud sessions.
- 180.12.2.9 Provide stream health warnings for high latency, dropped frames, relay usage, host CPU pressure, or missing keyframes.
- 180.12.2.10 Add Playwright visual tests for stream placeholder states, approval overlays, timeline updates, and takeover state transitions.
180.12.3 Approval and Audit UX#
- 180.12.3.1 Render approval cards with concise risk summary, exact command, affected app, affected files/accounts, expected result, and alternatives.
- 180.12.3.2 Provide "approve once", "approve for this session", "deny", "edit constraints", and "ask agent to revise plan."
- 180.12.3.3 Require stronger confirmation for destructive, publishing, financial, credential-sensitive, or shell actions.
- 180.12.3.4 Show policy explanation when an action is blocked.
- 180.12.3.5 Show audit replay with command timeline synchronized to screenshots/video and artifacts.
- 180.12.3.6 Provide exportable audit reports for a session or project.
- 180.12.3.7 Provide search across commands, artifacts, file paths, object names, actor names, URLs, and approvals.
- 180.12.3.8 Provide incident flagging for suspicious agent behavior, unexpected browser content, or repeated failures.
- 180.12.3.9 Add keyboard shortcuts for pause, stop, approve, deny, and takeover with safe confirmation where needed.
- 180.12.3.10 Add accessibility automation for approval cards, timeline, stream controls, and emergency actions.
180.13 File, Artifact, and Project Workflow#
180.13.1 Project Roots and File Transfer#
- 180.13.1.1 Define project root grants so agents can only read/write approved directories for the active production.
- 180.13.1.2 Implement file transfer to host with hashing, resumability, destination validation, conflict policy, and audit entries.
- 180.13.1.3 Implement file transfer from host for renders, screenshots, exports, logs, and selected project artifacts.
- 180.13.1.4 Implement preflight checks for available disk, path traversal, file type allowlist, file size, and overwrite approval.
- 180.13.1.5 Implement file snapshots before destructive operations or major scene/project mutations.
- 180.13.1.6 Implement restore from snapshot for project-local files.
- 180.13.1.7 Integrate with S3/MinIO output storage for generated artifacts and provenance.
- 180.13.1.8 Integrate with Isis output registry for generated assets imported into Blender/Unreal.
- 180.13.1.9 Integrate with Yemaya asset management for project attachment, review state, and production task linkage.
- 180.13.1.10 Add tests for transfer interruption, duplicate files, denied paths, large assets, hash mismatch, and restore.
180.13.2 Cross-App Creative Flows#
- 180.13.2.1 Implement "Isis to Blender": generated mesh/texture is selected, transferred, imported, materialized, preview rendered, and provenance attached.
- 180.13.2.2 Implement "Blender to Unreal": selected asset is exported from Blender, transferred/imported into Unreal, placed in map, and verified by screenshot.
- 180.13.2.3 Implement "Hathor to Unreal": narrative/world package is compiled to Unreal content, imported, map updated, and validation results reported.
- 180.13.2.4 Implement "Yemaya task to browser": agent opens production board or review page, updates status, attaches artifacts, and requests approval before external submission.
- 180.13.2.5 Implement "Unreal render to Yemaya": Movie Render Queue output is collected, uploaded, attached to a review item, and logged.
- 180.13.2.6 Implement "browser research to asset action": agent researches references in browser, stores citations/screenshots, then uses Blender/Unreal adapters to apply an approved change.
- 180.13.2.7 Implement "visual QA loop": agent captures before/after screenshots, compares expected visual changes, and produces a QA note.
- 180.13.2.8 Implement "failure recovery": if Blender/Unreal command fails, agent collects diagnostics, proposes fallback, and waits for approval before risky retries.
- 180.13.2.9 Implement "human handoff": agent pauses with context, operator takes over, completes local step, and agent resumes from updated state.
- 180.13.2.10 Add end-to-end tests for the top five cross-app flows.
180.14 Observability, Audit, and Replay#
180.14.1 Telemetry and Metrics#
- 180.14.1.1 Add structured logs for gateway, host, adapters, MCP server, Control Room, and agent planner decisions.
- 180.14.1.2 Add metrics for device online state, command latency, command failure rate, stream latency, reconnect count, approval latency, and adapter health.
- 180.14.1.3 Add per-adapter metrics for Blender command duration, Unreal command duration, browser action duration, desktop fallback success rate, and file transfer throughput.
- 180.14.1.4 Add host metrics for CPU, memory, disk, battery, thermal pressure, active app state, and process restarts.
- 180.14.1.5 Add WebRTC metrics for bitrate, frame rate, dropped frames, jitter, packet loss, codec, and relay/direct mode.
- 180.14.1.6 Add alerting for device offline, command failure spike, policy violation attempt, host crash loop, stream unavailable, and update failure.
- 180.14.1.7 Add traces linking agent request to MCP tool call, gateway command, host execution, adapter call, result, artifact, and UI event.
- 180.14.1.8 Add dashboards for local development, production gateway, per-device health, and per-session audit.
- 180.14.1.9 Add privacy-aware log sampling and retention controls.
- 180.14.1.10 Add synthetic monitors that run read-only commands against test devices.
180.14.2 Audit Ledger and Replay#
- 180.14.2.1 Create an append-only command audit ledger with content hashes for command payloads, results, artifacts, and approvals.
- 180.14.2.2 Record every permission grant, approval, denial, policy block, operator takeover, and emergency stop.
- 180.14.2.3 Record before/after evidence for mutating commands when feasible.
- 180.14.2.4 Store replay bundles containing timeline, screenshots, stream segments where enabled, command payloads, result summaries, and artifact references.
- 180.14.2.5 Provide a replay viewer in the Control Room.
- 180.14.2.6 Provide machine-readable audit export for compliance, debugging, and incident review.
- 180.14.2.7 Add tamper-evident hash chains for high-trust audit streams.
- 180.14.2.8 Add redaction and access controls for audit entries that include browser content, screenshots, or personal files.
- 180.14.2.9 Add audit comparison between planned actions and executed actions to detect drift.
- 180.14.2.10 Add tests proving audit records are emitted for success, failure, cancellation, approval, denial, block, and crash cases.
180.15 Reliability, Recovery, and Operations#
180.15.1 Failure Modes#
- 180.15.1.1 Define failure taxonomy: host offline, gateway offline, adapter offline, app not installed, app crashed, permission missing, command timeout, policy denied, approval timeout, network degraded, stream failed, and artifact transfer failed.
- 180.15.1.2 Implement clear user-facing and agent-facing error codes for every failure class.
- 180.15.1.3 Implement automatic host reconnect without losing device identity or active session context.
- 180.15.1.4 Implement adapter restart for Blender, Unreal, browser, and stream helpers without restarting the full host when possible.
- 180.15.1.5 Implement app crash detection and recovery prompts for Blender/Unreal/Chrome.
- 180.15.1.6 Implement command retry only for idempotent commands or commands with explicit retry safety.
- 180.15.1.7 Implement snapshot/rollback guidance after failed mutating commands.
- 180.15.1.8 Implement degraded mode: command-only without live stream, observe-only without control, or local-only without cloud.
- 180.15.1.9 Implement operator notification when a session needs human attention.
- 180.15.1.10 Add chaos tests for process crash, gateway restart, network loss, permission revocation, and host sleep.
180.15.2 Runbooks and Support#
- 180.15.2.1 Write runbook for first-time two-MacBook LAN setup.
- 180.15.2.2 Write runbook for Hetzner gateway deployment.
- 180.15.2.3 Write runbook for AWS gateway deployment.
- 180.15.2.4 Write runbook for Blender addon install and repair.
- 180.15.2.5 Write runbook for Unreal plugin install and repair.
- 180.15.2.6 Write runbook for macOS Screen Recording and Accessibility permission troubleshooting.
- 180.15.2.7 Write runbook for stream failure and TURN fallback.
- 180.15.2.8 Write runbook for device revocation and key rotation.
- 180.15.2.9 Write runbook for audit export after suspicious agent behavior.
- 180.15.2.10 Write runbook for safe uninstall, including LaunchAgent, Keychain entries, plugins, browser profiles, and logs.
180.16 Testing, Verification, and Quality Gates#
180.16.1 Automated Test Matrix#
- 180.16.1.1 Add unit tests for remote protocol schemas, validators, command routing, policy evaluation, and audit envelope generation.
- 180.16.1.2 Add integration tests for gateway-host command exchange with mocked host adapters.
- 180.16.1.3 Add integration tests for host-adapter process lifecycle with mocked Blender, Unreal, browser, and desktop helpers.
- 180.16.1.4 Add Blender headless tests for supported commands where CI can install Blender.
- 180.16.1.5 Add Unreal plugin tests using a minimal fixture project where CI infrastructure allows it, plus mocked tests for normal CI.
- 180.16.1.6 Add Playwright tests for browser adapter workflows.
- 180.16.1.7 Add Playwright tests for Control Room UI flows: device list, stream placeholder, command timeline, approvals, pause, takeover, and replay.
- 180.16.1.8 Add network impairment tests with simulated latency, packet loss, reconnect, and tunnel failover.
- 180.16.1.9 Add security tests for denied paths, missing permission, revoked device, prompt-injection browser page, and unapproved destructive action.
- 180.16.1.10 Add e2e smoke test for the demo scenario from cloud agent to remote Mac to Blender/browser with live timeline.
180.16.2 Manual and Hardware-in-the-Loop QA#
- 180.16.2.1 Create a hardware QA checklist for MacBook Air M1/M2/M3, multiple displays, Retina scaling, lid open/closed, battery/charger, and Wi-Fi/Ethernet.
- 180.16.2.2 Create Blender QA scenes for primitive edits, material edits, asset import/export, render capture, undo, and crash recovery.
- 180.16.2.3 Create Unreal QA projects for actor edits, map save, Blueprint call, material update, screenshot, Sequencer, and build.
- 180.16.2.4 Create browser QA tasks for unauthenticated browsing, authenticated profile grant, file download, file upload, blocked domain, and form submission approval.
- 180.16.2.5 Create desktop fallback QA tasks for native file dialog, permission prompt, app focus, modal handling, and visual wait.
- 180.16.2.6 Create cloud network QA for Hetzner agent, AWS agent, TURN relay, VPN active, captive portal, and NAT changes.
- 180.16.2.7 Create stream QA for low latency, high quality, source switching, privacy blur, freeze recovery, and recording.
- 180.16.2.8 Create operator interruption QA for pause, stop, takeover, denied approval, revoked device, and local unpair.
- 180.16.2.9 Create audit replay QA where a session is reconstructed from logs, screenshots, commands, artifacts, and approvals.
- 180.16.2.10 Require signoff evidence before each release: test logs, screenshots, stream recordings where allowed, diagnostic bundles, and known-risk notes.
180.16.3 Performance and Scalability Gates#
- 180.16.3.1 Define LAN and cloud latency budgets for command ack, command result, screenshot capture, stream startup, source switch, and approval propagation.
- 180.16.3.2 Benchmark command throughput under normal agent usage and under file-transfer/render workloads.
- 180.16.3.3 Benchmark WebRTC stream CPU usage on MacBook Air while Blender and Unreal are running.
- 180.16.3.4 Benchmark browser automation latency with direct CDP, Browser Use local, and Browser Use cloud.
- 180.16.3.5 Benchmark gateway session capacity for concurrent hosts, streams, command queues, and MCP clients.
- 180.16.3.6 Benchmark artifact transfer for small screenshots, medium GLB assets, large Unreal builds, and render outputs.
- 180.16.3.7 Add load tests for 10, 100, and 1,000 registered devices even if production starts with one.
- 180.16.3.8 Add thermal and battery impact testing for long sessions on MacBook Air.
- 180.16.3.9 Add performance regressions to CI where feasible and to nightly hardware tests where not feasible.
- 180.16.3.10 Block stable release when latency, crash rate, stream failure, or command failure exceeds defined thresholds.
180.17 Documentation, Developer Experience, and SDKs#
180.17.1 Documentation#
- 180.17.1.1 Write
docs/domains/bellona/extras/remote-control/architecture.mdwith topology diagrams, trust boundaries, protocols, adapters, and workflows. - 180.17.1.2 Write
docs/domains/bellona/extras/remote-control/security.mdcovering pairing, auth, policy, approvals, browser data, prompt injection, redaction, and audit. - 180.17.1.3 Write
docs/domains/bellona/extras/remote-control/networking.mdcovering LAN, gateway, TURN, Tailscale/WireGuard option, firewall, and troubleshooting. - 180.17.1.4 Write
docs/domains/bellona/extras/remote-control/blender.mdwith addon setup, command catalog, examples, limitations, and debugging. - 180.17.1.5 Write
docs/domains/bellona/extras/remote-control/unreal.mdwith plugin setup, command catalog, examples, limitations, and debugging. - 180.17.1.6 Write
docs/domains/bellona/extras/remote-control/browser.mdwith CDP, Playwright, Browser Use, profiles, auth, and prompt-injection guidance. - 180.17.1.7 Write
docs/domains/bellona/extras/remote-control/computer-use.mdwith fallback rules, visual verification, and Computer Use compatibility. - 180.17.1.8 Write
docs/domains/bellona/extras/remote-control/control-room.mdwith operator flows, approvals, pause/takeover, replay, and audit exports. - 180.17.1.9 Write
docs/domains/bellona/extras/remote-control/deployment.mdwith local, Hetzner, AWS, and private-network deployment options. - 180.17.1.10 Write
docs/domains/bellona/extras/remote-control/roadmap.mdlinking this phase to concrete implementation milestones and release gates.
180.17.2 SDK, CLI, and Examples#
- 180.17.2.1 Add a TypeScript SDK for device/session/command/stream operations.
- 180.17.2.2 Add Python client generation for automation scripts and notebook experimentation.
- 180.17.2.3 Add
bellona remoteCLI commands:pair,devices,sessions,stream,command,approve,diagnose, andunpair. - 180.17.2.4 Add example script: create Blender scene, render preview, export GLB, and upload artifact.
- 180.17.2.5 Add example script: open Unreal project, import GLB, place actor, capture screenshot, and save map.
- 180.17.2.6 Add example script: open browser, navigate production board, attach artifact, and request form-submission approval.
- 180.17.2.7 Add example script: cloud agent on Hetzner controls local Blender through gateway and records audit.
- 180.17.2.8 Add example script: desktop fallback handles a native file dialog with screenshots and visual verification.
- 180.17.2.9 Add mock host and mock gateway utilities for development without a second MacBook.
- 180.17.2.10 Add generated command reference from schema comments so docs and MCP tool descriptions do not drift.
180.18 Deployment, Operations, and Commercial Hardening#
180.18.1 Gateway Deployment#
- 180.18.1.1 Build container image for
apps/bellona/remote-gatewaywith health checks, graceful shutdown, and configuration validation. - 180.18.1.2 Provide docker-compose local deployment with Postgres, Redis, object storage, gateway, TURN, and sample Control Room.
- 180.18.1.3 Provide Hetzner deployment with VM sizing, firewall, TLS certs, systemd services, backups, monitoring, and update plan.
- 180.18.1.4 Provide AWS deployment with ECS/Fargate or EC2, ALB, RDS/Postgres, ElastiCache/Redis, S3, CloudWatch/OpenTelemetry, and TURN.
- 180.18.1.5 Provide private LAN deployment with no cloud dependency.
- 180.18.1.6 Provide high-availability deployment with multiple gateway instances, shared state, and sticky stream signaling.
- 180.18.1.7 Provide backup/restore for device registry, session audit, artifacts, and policy configuration.
- 180.18.1.8 Provide migration strategy for schema changes and protocol version compatibility.
- 180.18.1.9 Provide production config validation for TLS, TURN, origins, auth, retention, and rate limits.
- 180.18.1.10 Add deployment smoke tests for all supported deployment modes.
180.18.2 Remote Host Distribution#
- 180.18.2.1 Build macOS app packaging with signed installer or
.dmg, LaunchAgent registration, and clean uninstall. - 180.18.2.2 Provide first-run pairing flow and permission onboarding.
- 180.18.2.3 Provide automatic update with signed manifests and rollback.
- 180.18.2.4 Provide offline/local update option for private studios.
- 180.18.2.5 Provide MDM-friendly install documentation for managed fleets.
- 180.18.2.6 Provide app health self-test before pairing completes.
- 180.18.2.7 Provide diagnostics upload with user approval and redaction.
- 180.18.2.8 Provide host crash reporting that excludes screenshots and sensitive command payloads by default.
- 180.18.2.9 Provide signed Blender addon package and Unreal plugin package through the host installer or adapter repair flow.
- 180.18.2.10 Add release checklist for notarization, malware scanning, update signature verification, and clean uninstall test.
180.18.3 Governance and Cost Controls#
- 180.18.3.1 Add organization-level controls for who can pair devices, start sessions, approve risky actions, access streams, and export audits.
- 180.18.3.2 Add per-project and per-device budgets for cloud agent tokens, gateway compute, TURN bandwidth, artifact storage, and remote render/build jobs.
- 180.18.3.3 Add spend alerts and hard stops for runaway remote-agent loops.
- 180.18.3.4 Add policy templates: solo creator, small studio, enterprise studio, education lab, and local-only private mode.
- 180.18.3.5 Add audit retention policies by organization tier and project sensitivity.
- 180.18.3.6 Add administrator reports for device usage, session hours, command counts, approval rates, failure rates, and cost.
- 180.18.3.7 Add billing/event metering hooks if productized as a hosted service.
- 180.18.3.8 Add legal/privacy review for remote screen capture, browser session handling, employee monitoring risks, and consent.
- 180.18.3.9 Add data residency options for audit artifacts and stream recordings.
- 180.18.3.10 Add periodic access review for paired devices, active operators, API tokens, and MCP clients.
180.19 Acceptance Criteria#
180.19.1 MVP Acceptance#
- 180.19.1.1 A Remote Host can be installed on a second MacBook Air, paired with the gateway, and shown as online.
- 180.19.1.2 A Control Room user can start a session, see a live stream, pause the agent, and stop the session.
- 180.19.1.3 An MCP client can list devices, start a session, query Blender scene state, create a Blender object, capture a screenshot, and receive a structured result.
- 180.19.1.4 The browser adapter can launch an isolated browser profile, navigate to a page, click/type through CDP, capture a screenshot, and show progress in the Control Room.
- 180.19.1.5 Desktop fallback can focus Blender/Chrome, take a screenshot, click a known UI target, and visually verify the result.
- 180.19.1.6 Every command, result, approval, screenshot, and artifact is linked by correlation ID in the audit ledger.
- 180.19.1.7 A cloud-hosted agent on Hetzner or AWS can control the remote MacBook through the outbound tunnel without inbound ports on the MacBook.
- 180.19.1.8 Revoking the device immediately blocks new commands and disconnects active control channels.
- 180.19.1.9 Missing macOS permissions are detected and surfaced with actionable diagnostics.
- 180.19.1.10 Relevant automated tests pass: protocol, policy, gateway, host, Blender smoke, browser smoke, Control Room smoke, and audit tests.
180.19.2 Full Vision Acceptance#
- 180.19.2.1 Agents can control Blender, Unreal, browser, files, and selected desktop UI through a unified, policy-enforced MCP gateway.
- 180.19.2.2 Agents consistently prefer deep Blender/Unreal/CDP commands and use Computer Use only for validated fallback cases.
- 180.19.2.3 Operators can watch, approve, deny, pause, takeover, resume, replay, and export any remote creative session.
- 180.19.2.4 Same-LAN sessions work with direct low-latency transport and remote-network sessions work through outbound gateway/TURN fallback.
- 180.19.2.5 Blender workflows cover scene query, object/material edit, import/export, render, undo, dry-run, and advanced automation macros.
- 180.19.2.6 Unreal workflows cover project/map/world query, actor edit, Blueprint call, material update, asset import, Sequencer, screenshot, render, and build/cook/package.
- 180.19.2.7 Browser workflows cover isolated profiles, optional real profile grants, direct CDP, optional Browser Use delegation, downloads, uploads, and prompt-injection safeguards.
- 180.19.2.8 Security controls cover device identity, mTLS/signed tokens, local deny rules, scoped permissions, approvals, secrets redaction, audit hashing, signed updates, and revocation.
- 180.19.2.9 Operations cover install, update, rollback, diagnostics, host sleep/wake, reconnect, network impairment, adapter crash, and clean uninstall.
- 180.19.2.10 The system can support a fleet of remote creative hosts and cloud agents while preserving per-device policy, per-project context, observable behavior, and reproducible audit.
180.20 Future Extensions#
180.20.1 Multi-Host Creative Fleet#
- 180.20.1.1 Add fleet scheduling that routes tasks to hosts by app version, project availability, GPU/CPU load, network latency, permissions, and current operator.
- 180.20.1.2 Add role-based host allocation: Blender workstation, Unreal workstation, browser/admin workstation, render node, build node, and QA viewer.
- 180.20.1.3 Add multi-host session rooms where multiple agents operate different machines under one production task.
- 180.20.1.4 Add distributed locks across shared project roots and source-control branches.
- 180.20.1.5 Add source-control-aware workflows for Perforce, Git LFS, Plastic/Unity Version Control, and local file snapshots.
- 180.20.1.6 Add render/build queue integration for long-running jobs that continue after interactive sessions end.
- 180.20.1.7 Add host pooling for cloud Mac instances if future macOS virtualization or rented Mac hardware becomes part of the workflow.
- 180.20.1.8 Add organization-wide device health dashboard and capacity planning.
- 180.20.1.9 Add automated maintenance windows for adapter/plugin updates.
- 180.20.1.10 Add fleet-level security posture scoring.
180.20.2 Advanced Agentic Creative Production#
- 180.20.2.1 Add semantic scene diffing across Blender and Unreal so agents can explain exactly what changed.
- 180.20.2.2 Add visual regression testing for agent edits using before and after screenshots/renders.
- 180.20.2.3 Add autonomous style-guide enforcement for worlds, characters, props, lighting, materials, and production boards.
- 180.20.2.4 Add voice-driven operator supervision integrated with Iris and Control Room sessions.
- 180.20.2.5 Add collaborative multi-user rooms where several humans and agents can observe or control different surfaces with explicit locks.
- 180.20.2.6 Add learned adapter-selection policies from historical success/failure data.
- 180.20.2.7 Add self-healing agent behavior that detects broken adapters, repairs plugins, and resumes from checkpoints after approval.
- 180.20.2.8 Add digital twin state mirrors for Blender/Unreal scenes so agents can plan without constantly querying heavy apps.
- 180.20.2.9 Add hardware controller integration for operator takeover, stream switching, and approval shortcuts.
- 180.20.2.10 Add mobile observer mode for approving or stopping remote sessions from a phone while agents run on cloud infrastructure.