Eve library

Eve current-page screenshots

Other

Revision 1

On this page

The web chat's camera button enables Page screenshots for the open session. Eve can then call inspect_page when a chart, image or layout needs visual inspection. Text and controls continue to use read_page. Permission starts off and resets when the panel closes or the session changes. Disabling it cancels capture and clears the collapsed local preview.

The browser renders the current viewport with html2canvas-pro. This is a DOM rendering, so it can differ from the browser's native pixels. Form values, contenteditable regions, data-assistant-private sections, Eve's panel, embedded media and custom elements are hidden or omitted. Authors must mark other sensitive page regions private. Cross-origin images without CORS and some CSS/graphics may be missing. No other tab or desktop content is accessible.

One image per turn is allowed, with at most 1.6 million pixels, a 1600-pixel side and 768 kB of PNG bytes. The session-bound client-tool exchange validates the image header, dimensions, path and freshness. Capture times out after 20 seconds; the vision request after 45 seconds. Errors return to the existing tool-result handling so Eve can explain that visual inspection was unavailable.

The image goes to google/gemini-3.5-flash-lite through OpenRouter with data_collection: deny, zdr: true and require_parameters: true. Only its bounded text description returns to the main chat model through the existing untrusted-tool-output and secret-disclosure guards. Images are not saved in chat history or turn traces. The preview exists only in browser memory. Vision usage contributes to turn metrics; unknown provider cost stays unknown.

Configuration#

Set OPENROUTER_API_KEY in the BFF process. Override the image-capable model with OSHUN_ASSISTANT_VISION_MODEL, or set OSHUN_ASSISTANT_VISION_ENABLED=false to remove the vision tool. Restart the BFF after changing configuration. The registry marks this operator-configured leg as unmeasured: a smoke test does not establish comparative visual accuracy or an injection-resistance score.

Verification#

Run the focused desktop/mobile browser gate from the repository root:

sh
pnpm exec playwright test --config apps/oshun/web/playwright.page-inspection.config.ts

It starts a test BFF and Next dev server on ports 4010/3010. Only the model providers are doubled. Browser rendering, authentication, session ownership, SSE, client-tool delivery and accessibility checks remain real. Pixel assertions verify that chart colors are present while private-region and form colors are absent. The test saves a synthetic chart capture outside the repo under /tmp/eve-page-evidence for optional live verification.

Service tests cover the model request, bounds, stale/invalid images, errors, cross-session result rejection and the full turn round trip. Web tests cover redaction, consent reset, cancellation and navigation. page-vision.live.spec.ts is a billed opt-in probe: provide the key only to that process and set EVE_PAGE_VISION_LIVE=1. EVE_PAGE_VISION_IMAGE selects the synthetic test PNG; EVE_PAGE_VISION_EVIDENCE optionally saves the description and usage externally.

On 2026-09-06, the live probe correctly identified the taller blue bar from the browser's synthetic mobile capture. OpenRouter reported 1,219 input tokens, 20 output tokens and $0.0004157 for that request. This is a functional smoke test with the required privacy routing filters, not a comparative benchmark.

Local verification passed the relevant service, contract, security/admission and chat tests; all four desktop/mobile browser flows with shared axe checks; the live probe; scoped TypeScript and repository lint; and the BFF production bundle. Web compilation was verified through Next's development server. A full Next production build was not run under this host's memory-safety policy.

The camera control includes a collapsed preview. Each reply's Details drawer also pairs its retained screenshot with the vision description and the guarded tool result. See reply inspection for retention, redaction and keyboard behavior. Grounded replies retain their evidence inspector.