Eve library

OSHUN Mobile E2E Tests

Other

Revision 1

On this page

Mobile end-to-end coverage for auth, onboarding, and session recovery using Maestro.

Prerequisites#

  1. Install Maestro:

    bash
    brew install mobile-dev-inc/tap/maestro
    
  2. Start the OSHUN BFF:

    bash
    pnpm nx run @oshun/bff:dev
    

    For the PRO-gated shell/authenticated-domain-rooms.yaml flow (Arete room + week-posture strip), start the BFF with the dev-only tier seams instead, so a fresh sign-up is provisioned as a verified pro account and lands in the room rather than the access-locked gate:

    bash
    OSHUN_DEV_SIGNUP_PLAN=pro OSHUN_DEV_AUTO_VERIFY_EMAIL=true \
      pnpm nx run @oshun/bff:dev
    # or the convenience alias that bakes in both seams:
    pnpm nx run @oshun/bff:dev-e2e-pro
    

    Both vars are inert under a production runtime (resolveDevSignupPlan returns free when isProductionRuntime is true), so they are safe dev/test-only seams. Start a fresh BFF per run — the in-memory auth store rejects a repeat sign-up with the same email (EMAIL_ALREADY_REGISTERED).

  3. Start Metro for the dev client with E2E hooks enabled:

    bash
    EXPO_PUBLIC_OSHUN_BFF_URL=http://127.0.0.1:4010 \
    EXPO_PUBLIC_E2E_DISABLE_AUTOFILL=true \
      pnpm --dir apps/oshun/mobile exec expo start --dev-client --port 8081
    
  4. Build and launch the iOS simulator app with the local BFF URL:

    bash
    EXPO_PUBLIC_OSHUN_BFF_URL=http://127.0.0.1:4010 \
      pnpm --dir apps/oshun/mobile exec expo run:ios --simulator "iPhone 16"
    

Commands#

  • pnpm --dir apps/oshun/mobile run e2e:smoke
  • pnpm --dir apps/oshun/mobile run e2e:core
  • pnpm --dir apps/oshun/mobile run e2e:nisaba
  • pnpm --dir apps/oshun/mobile run e2e:signoff
  • pnpm --dir apps/oshun/mobile run e2e:regression
  • pnpm --dir apps/oshun/mobile run e2e:offline-retry
  • pnpm --dir apps/oshun/mobile run e2e:assistant-transport
  • pnpm --dir apps/oshun/mobile run e2e:matrix:assistant-transport
  • pnpm --dir apps/oshun/mobile run e2e:matrix:assistant-transport:dry
  • pnpm --dir apps/oshun/mobile run e2e:matrix
  • pnpm --dir apps/oshun/mobile run e2e:matrix:dry
  • pnpm --dir apps/oshun/mobile run e2e:matrix:ios
  • pnpm --dir apps/oshun/mobile run e2e:matrix:android
  • pnpm --dir apps/oshun/mobile run e2e:matrix:nisaba
  • pnpm --dir apps/oshun/mobile run e2e:matrix:nisaba:dry
  • pnpm --dir apps/oshun/mobile run e2e:matrix:nisaba:ios
  • pnpm --dir apps/oshun/mobile run e2e:matrix:nisaba:android
  • pnpm --dir apps/oshun/mobile run e2e:matrix:signoff
  • pnpm --dir apps/oshun/mobile run e2e:matrix:signoff:dry
  • pnpm --dir apps/oshun/mobile run e2e:matrix:signoff:ios
  • pnpm --dir apps/oshun/mobile run e2e:matrix:signoff:android
  • pnpm --dir apps/oshun/mobile run e2e:matrix:offline-retry
  • pnpm --dir apps/oshun/mobile run e2e:matrix:offline-retry:dry
  • pnpm --dir apps/oshun/mobile run e2e:matrix:offline-retry:ios
  • pnpm --dir apps/oshun/mobile run e2e:matrix:offline-retry:android
  • pnpm --dir apps/oshun/mobile run e2e:auth
  • pnpm --dir apps/oshun/mobile run e2e:onboarding
  • pnpm --dir apps/oshun/mobile run e2e:recovery

The suite runner generates unique account emails for each flow so the smoke and regression runs remain repeatable against the in-memory auth service.

Offline retry coverage:

  • offline-and-retry-suite.yaml groups Android offline shell continuity, activity retry editing, manual retry, and Tara/Veritas/Nisaba cache replay.
  • QA can run that contract directly with:
    • pnpm --dir apps/oshun/mobile run e2e:offline-retry
    • pnpm --dir apps/oshun/mobile run e2e:matrix:offline-retry
    • pnpm --dir apps/oshun/mobile run e2e:matrix:offline-retry:dry
    • pnpm --dir apps/oshun/mobile run e2e:matrix:offline-retry:ios
    • pnpm --dir apps/oshun/mobile run e2e:matrix:offline-retry:android

Shell accessibility coverage:

  • shell/navigation-accessibility-audit.yaml validates structural tab reach, thumb-zone domain-switcher access, and the shell-to-domain handoff path.
  • shell/assistant-accessibility-audit.yaml validates the assistant sheet disclosure stack, text prompt composer, and assisted-action handoff path.
  • shell/external-app-links.yaml validates release-candidate Universal Links and Android App Links from https://oshun.app and https://www.oshun.app into domain and assistant destinations, plus https://app.oshun.com V3 and V6 app-host handoffs.
  • The flow is included in both smoke-suite.yaml and regression-suite.yaml.

Core experience coverage:

  • core-experience-suite.yaml groups the launch-critical auth, navigation, home, search, library, and assistant flows into one repeatable signoff suite.
  • shell/veritas-briefing-explainer-flows.yaml is now part of the active core and regression suites, proving the Veritas mobile handoff from Home briefing into claim briefs, the global assistant sheet, and the compact explainer assistant/follow-up lane.
  • shell/tara-home-quick-start-completion.yaml now proves a real Home Tara quick-start can launch live audio, complete through the route-scoped Tara E2E harness path, auto-save the reflection, and return to Home without splitting that journey across separate synthetic flows.
  • shell/search-library-assistant-journey.yaml closes the missing explore search coverage by saving a real search result into Library and continuing into Assistant from the preserved surface.
  • shell/activity-metis-continuity.yaml now signs off the native Metis continuity path from Activity into the mobile shell preview, then verifies the real study-entry contract from that route: free course preview handoff plus Pro and Premium upgrade gating for tutoring and assessments.

Nisaba study coverage:

  • nisaba-study-suite.yaml is the dedicated Maestro contract for the Nisaba mobile study loop.
  • It groups the task-critical Nisaba study flows into one explicit suite:
    • shell/nisaba-domain-home.yaml
    • shell/nisaba-reading-queue.yaml
    • shell/nisaba-daily-passage.yaml
    • shell/nisaba-assistant-explanation.yaml
    • shell/nisaba-highlights-annotations.yaml
    • shell/nisaba-quick-compare.yaml
    • shell/nisaba-lexicon-morphology.yaml
    • shell/nisaba-saved-materials.yaml
    • shell/nisaba-study-reminders.yaml
  • Together those flows cover domain home, reading queue, daily passage, assistant-assisted explanation with inspectable citations, compact highlights, annotation flow, quick compare, glossary lookup, morphology lookup, saved manuscripts/editions, and study reminders.
  • QA can run that contract directly with:
    • pnpm --dir apps/oshun/mobile run e2e:nisaba
    • pnpm --dir apps/oshun/mobile run e2e:matrix:nisaba
    • pnpm --dir apps/oshun/mobile run e2e:matrix:nisaba:ios
    • pnpm --dir apps/oshun/mobile run e2e:matrix:nisaba:android

Redesigned shell signoff coverage:

  • shell-redesign-signoff-suite.yaml is the dedicated release-review suite for the redesigned mobile shell.
  • It groups the strongest home, navigation, continuation, and contextual assistant-entry flows instead of asking QA to assemble that bar manually.
  • shell/home-daypart-and-return-journeys.yaml now asserts the redesigned Home launcher, quick-action, and assistant-entry hierarchy before release signoff.
  • shell/library-and-assistant-navigation.yaml now signs off assistant entry continuity from Home through Explore, Activity, Profile, and Library while asserting the persistent shell launcher context in each state.
  • shell/redesigned-shell-visual-captures.yaml is the device-specific visual capture flow for redesigned shell review.
  • ../scripts/capture-shell-redesign-visuals.sh runs that capture flow against ios-current, android-lower-tier, or both profiles.
  • Capture outputs are written into the ignored artifacts/shell-redesign-signoff/screenshots/ tree so QA can review the final iphone-17 and android-lower-tier screenshot sets without polluting the git worktree.
  • For Android Expo Go signoff, the capture runner accepts the existing APP_ID, DEV_CLIENT_LINK, EXPO_GO_LINK, and VERITAS_ROUTE_LINK overrides and now bootstraps the Maestro Android helper packages automatically when the emulator is missing them.

Device matrix coverage:

  • device-matrix.yaml is the canonical QA matrix for:
    • ios-current -> iPhone 17 with fallback to iPhone 17 Pro and iPhone 16
    • android-lower-tier -> OSHUN Lower Tier Android
  • scripts/run-maestro-device-matrix.sh runs the core suite against one profile or both profiles.
  • The same runner accepts signoff and offline-retry as suite arguments for redesigned shell release review and offline/cache replay on either matrix profile.
  • --dry-run validates the resolved matrix commands without requiring a booted simulator or emulator.
  • assistant-transport exercises native cancel, background/foreground, no automatic replay, and transcript continuity on both matrix profiles, plus an Android-only airplane-mode recovery cell. Build/start the tested client with EXPO_PUBLIC_OSHUN_MOBILE_ASSISTANT_TRANSPORT=streaming; the default remains buffered until admission. The paired Android benchmark runner and preregistered thresholds are documented in docs/audits/eve-phase-8.4-mobile-assistant-transport-contract.md.
  • For Android runs, bind the lower-tier target explicitly when needed:
    • ANDROID_LOWER_TIER_DEVICE_NAME="OSHUN Lower Tier Android" pnpm --dir apps/oshun/mobile run e2e:matrix:android
    • ANDROID_LOWER_TIER_DEVICE_ID="<adb-device-id>" pnpm --dir apps/oshun/mobile run e2e:matrix:android

Authenticated domain-room coverage (Arete, PRO-gated)#

  • shell/authenticated-domain-rooms.yaml signs up a fresh user against the local BFF, opens Arete, and asserts the authenticated arete-week-posture-strip renders (GET /v1/arete/room) instead of the anonymous fixture fallback.

  • It is NOT wired into any named suite, so run it directly against a BFF started with the pro-tier seams (see step 2 above):

    bash
    # In a separate shell, after the pro-tier BFF + Metro + simulator are up:
    maestro test apps/oshun/mobile/e2e/flows/shell/authenticated-domain-rooms.yaml
    
  • e2e/config.yaml supplies the default AUTH_FLOW_* env the sign-up sub-flow needs; pass -e AUTH_FLOW_EMAIL=<unique> (or restart the BFF) to avoid an EMAIL_ALREADY_REGISTERED collision on a repeat run against the same BFF.