Week 1: Stabilize and Observe#
Metrics Review Cadence#
Daily review of key metrics at the 09:30 standup:
| Metric | Target | Source |
|---|---|---|
| Crash rate (iOS) | < 0.5% | Sentry |
| Crash rate (Android) | < 0.5% | Sentry + Play Console Vitals |
| ANR rate (Android) | < 0.2% | Play Console Vitals |
| BFF error rate | < 1% | BFF Prometheus metrics |
| BFF P95 latency | < 500ms | BFF Prometheus metrics |
| Onboarding completion | >= 85% | Analytics: onboarding.completed |
| D1 retention | >= 40% | Analytics: session.start D+1 |
| Domain launch rate | >= 70% | Analytics: domain.launched |
| Search usage | >= 20% | Analytics: search.executed |
Activation Tracking#
A user is "activated" when they complete all of:
- Account creation.
- Onboarding flow completion.
- At least one domain launch.
- Return visit on D+1.
Track activation rate as the primary leading indicator of product-market fit.
Alert Tuning#
After 7 days of production data:
- Review alert noise-to-signal ratio. Reduce false positives.
- Adjust thresholds based on observed baselines.
- Add new alerts for patterns discovered during launch (e.g., specific API endpoints with elevated error rates).
Week 2: Identify and Fix UX Friction#
Friction Identification Methods#
-
Funnel Analysis: Identify drop-off points in:
- Onboarding flow (step-by-step completion rates)
- Home → domain launch conversion
- Search → result tap conversion
- Profile → settings engagement
-
Session Recordings (if consented): Review anonymized session replays for confusion patterns, dead-end navigations, and rage taps.
-
Support Ticket Analysis: Categorize incoming support requests:
- Navigation confusion
- Feature discoverability
- Performance complaints
- Content quality feedback
-
Beta Survey Follow-Up: Re-survey beta cohort members who gave low NPS.
-
App Store Reviews: Monitor and respond to 1-3 star reviews. Extract actionable themes.
Top 10 UX Friction Fix Process#
- Collect all friction signals from the methods above.
- Score each by: frequency × severity × fix effort.
- Rank and select top 10.
- Assign owners and target ship dates.
- Ship fixes via OTA updates (JS-only) or expedited store releases.
- Verify improvement via before/after metrics.
Template for tracking:
| # | Friction Point | Frequency | Severity | Effort | Owner | Status | Ship Date |
|---|---|---|---|---|---|---|---|
| 1 | [Description] | High/Med/Low | P1-P3 | S/M/L | [Name] | [Status] | [Date] |
Week 3: Experiment and Optimize#
Recommendation Quality Tuning#
The home feed recommendation engine needs calibration based on real usage data:
-
Baseline measurement: Record current metrics:
- Home feed tap-through rate (CTR)
- Domain balance (are all 4 domains getting engagement?)
- Scroll depth (how far do users scroll before stopping?)
- Time-of-day patterns (do evening users engage more with Nyx?)
-
Signal weighting adjustment:
- Increase weight for domains the user has explicitly favorited.
- Decrease weight for domains the user consistently ignores.
- Add recency bias: prioritize fresh content over stale.
- Add time-of-day signals: Tara mornings, Nyx evenings, Arete check-in times.
-
A/B test iterations:
- Experiment:
home-feed-ranking-v2— test new signal weights. - Traffic: 10% for 7 days.
- Primary metric: home feed CTR.
- Guardrails: session duration drop, domain diversity index.
- Experiment:
Notification Quality Tuning#
- Opt-in rate tracking: What percentage of users enable notifications?
- Tap-through rate: Which notification types drive re-engagement?
- Dismiss rate: Which notifications are ignored or dismissed?
- Unsubscribe rate: Are notifications causing users to disable them?
Tuning actions:
- Reduce notification frequency for users with high dismiss rate.
- Personalize notification timing based on user's active hours.
- A/B test notification copy for Arete daily check-in reminders.
- Remove low-performing notification types.
Week 4: Review and Roadmap Rebalance#
30-Day Metrics Report#
Compile a comprehensive report covering:
-
Acquisition:
- Total installs (iOS + Android + web)
- Install-to-signup conversion
- Signup-to-activation conversion
- Acquisition channel breakdown
-
Activation:
- Onboarding completion rate
- First domain launch rate
- Activation rate (defined above)
- Time-to-activation (median)
-
Retention:
- D1, D7, D14, D30 retention curves
- Retention by domain preference
- Retention by onboarding path
- Cohort comparison (Week 1 vs. Week 2 vs. Week 3 signups)
-
Engagement:
- DAU/MAU ratio
- Sessions per DAU
- Domain diversity (avg domains used per user)
- Cross-domain journey frequency
- Feature adoption rates (search, favorites, offline, sharing)
-
Quality:
- Crash-free session rate
- App Store / Play Store rating
- Support ticket volume and resolution time
- NPS score (from in-app survey)
-
Performance:
- Cold start P50/P95 (iOS and Android)
- Home feed TTI P50/P95
- Domain launch P50/P95
- BFF latency P50/P95/P99
Roadmap Rebalance#
Based on the 30-day report:
-
Kill underperforming features: If a feature has < 5% adoption after 30 days and isn't trending up, consider removing it or redesigning.
-
Double down on winners: Features with high engagement and positive feedback should get investment priority.
-
Address retention gaps: If D7 retention is below target, prioritize the friction fixes that most impact the Day 2-7 experience.
-
Domain balance: If one domain has significantly lower engagement:
- Investigate: is it a discovery problem or a content quality problem?
- Adjust home feed prominence to increase exposure.
- Partner with domain team to improve domain-specific value proposition.
-
Update POST_MVP_SCOPE.md: Re-prioritize Wave 1-3 features based on real usage data. Move high-demand features up. Deprioritize features that user data doesn't support.
Domain-Specific Acceptance Verification#
Tara#
| Criterion | Method | Target |
|---|---|---|
| Launch meditation from home in <= 2 taps | Beta script 2 | 100% pass |
| Resume session syncs across devices | Cross-device test | No data loss |
| Audio playback survives interruptions | Manual test | Graceful resume |
| Audio plays in background | Manual test | Continues |
| Favorites persist after reinstall | Reinstall test | All preserved |
Veritas#
| Criterion | Method | Target |
|---|---|---|
| Open top story in <= 2 taps | Beta script 2 | 100% pass |
| Confidence indicators visible on small screens | Device matrix test | Readable SE |
| Save/follow persists across mobile and web | Cross-platform test | Synced |
| Evidence trail is navigable | Manual test | Tap-accessible |
Nyx#
| Criterion | Method | Target |
|---|---|---|
| Access nightly highlights quickly | Beta script 3 | < 3 seconds |
| Core content usable in low-connectivity | Offline test | Cached content |
| Night-mode ergonomics are comfortable | User survey | >= 80% positive |
| Sky map renders without jank | Performance test | < 16ms frames |
Arete#
| Criterion | Method | Target |
|---|---|---|
| Daily check-in in <= 60 seconds | Beta script 3 | 100% pass |
| Goals/habits/journal integrated in shell | Manual test | All accessible |
| Progress and streak visuals are clear | User survey | >= 80% positive |
| Journal entries persist and sync | Cross-device test | No data loss |
Cross-Platform Definition of Done#
| Platform | Check | Method | Status |
|---|---|---|---|
| iOS | Smoke tests pass | Maestro e2e | [ ] |
| iOS | Accessibility audit passes | axe / VoiceOver | [ ] |
| iOS | Performance budgets met | Startup + jank | [ ] |
| Android | Smoke tests pass | Maestro e2e | [ ] |
| Android | Accessibility audit passes | axe / TalkBack | [ ] |
| Android | Performance budgets met | Startup + jank | [ ] |
| Web | Responsive layout verified | Playwright viewport | [ ] |
| Web | Accessibility audit passes | Lighthouse >= 90% | [ ] |
| Web | SEO checks pass | Lighthouse >= 85% | [ ] |
| PWA | Installability verified | manifest.json | [ ] |
| PWA | Offline mode works | sw.js test | [ ] |
| PWA | Update flow works | SW update message | [ ] |
| All | Shared auth verified | Cross-surface test | [ ] |
| All | Entitlements synced | Subscription test | [ ] |
| All | Analytics events validated | Event audit | [ ] |
| All | Monitoring and alerts live | Dashboard check | [ ] |
| All | Runbooks documented | ON_CALL_RUNBOOK.md | [ ] |
| All | Documentation up to date | docs/releases/v1/* review | [ ] |