Eve library

End-to-End Testing Guide

Guide

Revision 1

On this page

This document covers end-to-end (E2E) testing for the Lilith platform using Playwright.

Overview#

The Lilith platform uses Playwright for comprehensive E2E testing with 377+ test files covering all major user flows.

Configuration#

Playwright Setup#

Location: playwright.config.ts

typescript
import { defineConfig, devices } from '@playwright/test';
import os from 'os';

export default defineConfig({
  testDir: 'tests',
  timeout: 30000,
  fullyParallel: true,
  workers: Math.max(2, Math.floor(os.cpus().length * 0.75)),
  retries: process.env.CI ? 2 : 0,

  reporter: process.env.CI
    ? [
        ['list'],
        [
          'json',
          { outputFile: 'reports/test-results/playwright-results.json' },
        ],
        ['github'],
      ]
    : [['list']],

  use: {
    baseURL: 'http://localhost:3000',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
    trace: 'on-first-retry',
  },

  webServer: {
    command: 'cd web && npm run dev',
    port: 3000,
    reuseExistingServer: !process.env.CI,
    timeout: 120000,
  },

  projects: [{ name: 'chromium', use: { ...devices['Desktop Chrome'] } }],
});

Key Features#

  • Parallel Execution: Uses 75% of CPU cores locally
  • CI Optimizations: Retries, JSON reporting, GitHub annotations
  • Failure Artifacts: Screenshots, videos, traces on failure
  • Web Server Auto-Start: Starts dev server automatically

Test Categories#

Authentication Tests#

Location: tests/auth.*.spec.ts

Test File Coverage
auth.comprehensive.spec.ts Complete auth flow testing
auth.webauthn.spec.ts WebAuthn/Passkey authentication
auth.dsr-dsar.spec.ts Data subject access requests
auth.privacy-policy-versioning.spec.ts Policy version handling
auth.personalization-vectors.spec.ts Personalization features

Example: Login Flow Test#

typescript
import { test, expect } from '@playwright/test';

test.describe('Authentication Flow', () => {
  test('should login successfully', async ({ page }) => {
    await page.goto('/login');

    await page.fill('[data-testid="email-input"]', 'test@example.com');
    await page.fill('[data-testid="password-input"]', 'SecurePassword123!');
    await page.click('[data-testid="login-button"]');

    await expect(page).toHaveURL('/dashboard');
    await expect(page.locator('[data-testid="user-avatar"]')).toBeVisible();
  });
});

Content Tests#

Location: tests/content.*.spec.ts

Test File Coverage
content.crud.spec.ts Content CRUD operations
content.versioned-crud.spec.ts Version control
ai-content-templates.spec.ts AI-generated content

Example: Content Browsing Test#

typescript
test('should browse content catalog', async ({ page }) => {
  await page.goto('/content');

  // Filter by category
  await page.click('[data-testid="filter-meditation"]');

  // Verify content cards appear
  const cards = page.locator('[data-testid="content-card"]');
  await expect(cards).toHaveCount.greaterThan(0);

  // Click first card
  await cards.first().click();

  // Verify detail page
  await expect(page.locator('[data-testid="content-title"]')).toBeVisible();
  await expect(page.locator('[data-testid="play-button"]')).toBeVisible();
});

Meditation Player Tests#

Location: tests/file-length-audit.*.spec.ts

Test File Coverage
file-length-audit.haptic-meditation-service.spec.ts Haptic feedback
file-length-audit.audio-player.spec.ts Audio playback
file-length-audit.native-meditation-widget.spec.ts Native widgets

Example: Meditation Player Test#

typescript
test('should play meditation audio', async ({ page }) => {
  await page.goto('/meditations/guided-relaxation');

  // Start playback
  await page.click('[data-testid="play-button"]');

  // Verify playing state
  await expect(page.locator('[data-testid="pause-button"]')).toBeVisible();
  await expect(page.locator('[data-testid="progress-bar"]')).toHaveAttribute(
    'aria-valuenow',
    expect.stringMatching(/[1-9]\d*/)
  );

  // Pause
  await page.click('[data-testid="pause-button"]');
  await expect(page.locator('[data-testid="play-button"]')).toBeVisible();
});

Chat Functionality Tests#

Location: tests/bff.chat*.spec.ts

Test File Coverage
bff.chat-sse.spec.ts SSE streaming
debug-chat-page.spec.ts Chat UI testing

Example: Chat Test#

typescript
test('should send and receive chat messages', async ({ page }) => {
  await page.goto('/chat');

  // Send message
  await page.fill('[data-testid="message-input"]', 'Hello, AI!');
  await page.click('[data-testid="send-button"]');

  // Wait for response (SSE stream)
  await expect(
    page.locator('[data-testid="message-bubble"].assistant')
  ).toBeVisible({ timeout: 30000 });

  // Verify message content
  const response = page
    .locator('[data-testid="message-bubble"].assistant')
    .last();
  await expect(response).not.toBeEmpty();
});

Subscription/Payment Tests#

Location: tests/payment-*.spec.ts

Test File Coverage
payment-settlement-integration.spec.ts Payment flows

Example: Subscription Test#

typescript
test('should complete subscription purchase', async ({ page }) => {
  await page.goto('/pricing');

  // Select plan
  await page.click('[data-testid="premium-plan"]');

  // Fill payment details (test mode)
  await page.fill('[data-testid="card-number"]', '4242424242424242');
  await page.fill('[data-testid="card-expiry"]', '12/25');
  await page.fill('[data-testid="card-cvc"]', '123');

  // Submit
  await page.click('[data-testid="subscribe-button"]');

  // Verify success
  await expect(page).toHaveURL('/subscription/success');
  await expect(page.locator('[data-testid="plan-active"]')).toBeVisible();
});

Accessibility Tests#

Location: tests/accessibility.spec.ts

typescript
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test.describe('Accessibility', () => {
  test('should have no accessibility violations on home page', async ({
    page,
  }) => {
    await page.goto('/');

    const accessibilityScanResults = await new AxeBuilder({ page }).analyze();

    expect(accessibilityScanResults.violations).toEqual([]);
  });

  test('should support keyboard navigation', async ({ page }) => {
    await page.goto('/');

    // Tab through interactive elements
    await page.keyboard.press('Tab');
    await expect(page.locator(':focus')).toBeVisible();

    // Verify skip link
    await page.keyboard.press('Enter');
    await expect(page.locator('#main-content')).toBeFocused();
  });
});

Running Tests#

Basic Commands#

bash
# Run all tests
npx playwright test

# Run specific test file
npx playwright test tests/auth.comprehensive.spec.ts

# Run tests matching pattern
npx playwright test --grep "authentication"

# Run in headed mode (see browser)
npx playwright test --headed

# Run with UI mode
npx playwright test --ui

Parallel Execution#

bash
# Set number of workers
npx playwright test --workers=4

# Run tests in parallel across files
npx playwright test --fullyParallel

Sharding for CI#

bash
# Shard 1 of 3
npx playwright test --shard=1/3

# Shard 2 of 3
npx playwright test --shard=2/3

# Shard 3 of 3
npx playwright test --shard=3/3

Debug Mode#

bash
# Debug with Playwright Inspector
npx playwright test --debug

# Debug specific test
npx playwright test tests/auth.spec.ts:10 --debug

CI/CD Integration#

GitHub Actions#

yaml
name: E2E Tests

on: [push, pull_request]

jobs:
  e2e-tests:
    runs-on: ubuntu-latest

    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps chromium

      - name: Run E2E tests (shard ${{ matrix.shard }}/4)
        run: npx playwright test --shard=${{ matrix.shard }}/4
        env:
          CI: true

      - name: Upload test results
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-report-${{ matrix.shard }}
          path: playwright-report/
          retention-days: 30

Test Best Practices#

1. Use Data Test IDs#

html
<!-- Good -->
<button data-testid="login-button">Login</button>

<!-- Avoid -->
<button class="btn-primary">Login</button>

2. Wait for Network Idle#

typescript
// Wait for all network requests to complete
await page.waitForLoadState('networkidle');

// Or wait for specific request
await page.waitForResponse(
  (resp) => resp.url().includes('/api/user') && resp.status() === 200
);

3. Use Locator Assertions#

typescript
// Good - auto-waits and retries
await expect(page.locator('[data-testid="user-name"]')).toHaveText('John Doe');

// Avoid - doesn't auto-wait
const text = await page.locator('[data-testid="user-name"]').textContent();
expect(text).toBe('John Doe');

4. Isolate Tests#

typescript
test.beforeEach(async ({ page }) => {
  // Reset state before each test
  await page.goto('/logout');
});

5. Handle Flaky Tests#

typescript
test('potentially flaky operation', async ({ page }) => {
  // Increase timeout for slow operations
  test.setTimeout(60000);

  // Retry assertions
  await expect(async () => {
    const response = await page.locator('[data-testid="result"]').textContent();
    expect(response).toBe('expected');
  }).toPass({ timeout: 10000 });
});

Debugging Failed Tests#

View Trace#

bash
# Open trace viewer
npx playwright show-trace trace.zip

Screenshots#

typescript
// Take screenshot programmatically
await page.screenshot({ path: 'debug-screenshot.png' });

// Full page screenshot
await page.screenshot({ path: 'full-page.png', fullPage: true });

Console Logs#

typescript
// Capture console messages
page.on('console', (msg) => console.log(msg.text()));

// Capture page errors
page.on('pageerror', (error) => console.error(error));

Test Organization#

text
tests/
├── auth/                     # Authentication tests
│   ├── login.spec.ts
│   ├── register.spec.ts
│   └── oauth.spec.ts
├── content/                  # Content tests
│   ├── browse.spec.ts
│   ├── search.spec.ts
│   └── player.spec.ts
├── chat/                     # Chat functionality tests
│   ├── messaging.spec.ts
│   └── streaming.spec.ts
├── fixtures/                 # Test fixtures
│   ├── users.json
│   └── content.json
└── utils/                    # Test utilities
    ├── auth-helpers.ts
    └── api-helpers.ts

See Also#