Guide for setting up and configuring the Lilith platform testing environment.
Table of Contents#
- Prerequisites
- Test Framework Configuration
- Test Utilities
- Database Testing
- Mock Factories
- Test Fixtures
- Environment Configuration
- Running Tests
Prerequisites#
Required Software#
| Software | Version | Purpose |
|---|---|---|
| Node.js | 20.x | Runtime |
| Docker | 24.x | Test containers |
| PostgreSQL | 15.x | Database tests |
| Redis | 7.x | Cache tests |
Installation#
# Install dependencies
npm install
# Install test dependencies
npm install -D vitest @vitest/coverage-v8 @faker-js/faker testcontainers
# Install Playwright for E2E
npx playwright install
Test Framework Configuration#
Vitest Configuration#
// vitest.config.ts
import { defineConfig } from 'vitest/config';
import path from 'path';
export default defineConfig({
test: {
// Test environment
environment: 'node',
// Global test utilities
globals: true,
// Test file patterns
include: ['src/**/*.test.ts', 'src/**/*.spec.ts'],
exclude: ['node_modules', 'dist'],
// Setup files run before each test file
setupFiles: ['./src/__tests__/setup.ts'],
// Coverage configuration
coverage: {
provider: 'v8',
reporter: ['text', 'json', 'html', 'lcov'],
reportsDirectory: './coverage',
exclude: [
'node_modules/',
'dist/',
'**/*.d.ts',
'**/*.test.ts',
'**/*.spec.ts',
'**/fixtures/**',
'**/mocks/**',
],
thresholds: {
lines: 80,
branches: 80,
functions: 80,
statements: 80,
},
},
// Test timeouts
testTimeout: 10000,
hookTimeout: 10000,
// Pool configuration for parallelization
pool: 'threads',
poolOptions: {
threads: {
singleThread: false,
isolate: true,
},
},
// Retry configuration
retry: process.env.CI ? 2 : 0,
// Reporter
reporters: process.env.CI ? ['default', 'junit'] : ['default'],
// Output file for CI
outputFile: {
junit: './test-results/junit.xml',
},
},
// Path aliases
resolve: {
alias: {
'@': path.resolve(__dirname, './src'),
'@test': path.resolve(__dirname, './src/__tests__'),
},
},
});
Global Setup File#
// src/__tests__/setup.ts
import { beforeAll, afterAll, beforeEach, afterEach, vi } from 'vitest';
// Set test environment variables
process.env.NODE_ENV = 'test';
process.env.LOG_LEVEL = 'silent';
// Global before all tests
beforeAll(async () => {
// Initialize test database connection pool
// await initTestDatabase();
});
// Global after all tests
afterAll(async () => {
// Close connections
// await closeTestDatabase();
});
// Before each test
beforeEach(() => {
// Reset mocks
vi.clearAllMocks();
});
// After each test
afterEach(() => {
// Cleanup
vi.restoreAllMocks();
});
// Global error handling
process.on('unhandledRejection', (reason) => {
console.error('Unhandled Rejection in test:', reason);
});
Playwright Configuration#
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests/e2e',
fullyParallel: true,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 1 : undefined,
reporter: 'html',
use: {
baseURL: 'http://localhost:3000',
trace: 'on-first-retry',
screenshot: 'only-on-failure',
video: 'retain-on-failure',
},
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
},
{
name: 'webkit',
use: { ...devices['Desktop Safari'] },
},
{
name: 'Mobile Chrome',
use: { ...devices['Pixel 5'] },
},
{
name: 'Mobile Safari',
use: { ...devices['iPhone 12'] },
},
],
webServer: {
command: 'npm run dev',
url: 'http://localhost:3000',
reuseExistingServer: !process.env.CI,
},
});
Test Utilities#
@lilith/testing Package#
The @lilith/testing package provides comprehensive test utilities, mocks,
fixtures, helpers, containers, and config for the Lilith platform.
// Import everything from the main package
import {
createMockLogger,
createUser,
assertApiSuccess,
PostgresContainer,
createServiceVitestConfig,
} from '@lilith/testing';
// Or import from submodules for better tree-shaking
import { createMockLogger } from '@lilith/testing/mocks';
import { createUser } from '@lilith/testing/fixtures';
import { assertApiSuccess } from '@lilith/testing/helpers';
import { PostgresContainer } from '@lilith/testing/containers';
import { createServiceVitestConfig } from '@lilith/testing/config';
Mock Utilities#
The package provides type-safe mocks for common testing scenarios:
import {
createMockLogger,
createMockHttpClient,
createMockRedisClient,
createMockDatabaseClient,
createMockEventEmitter,
createMockTimers,
} from '@lilith/testing';
// Mock logger that captures all log entries
const logger = createMockLogger();
logger.info('User logged in', { userId: '123' });
logger.warn('Rate limit approaching');
console.log(logger.logs); // Array of all log entries
logger.clear(); // Clear captured logs
// Mock HTTP client with response stubbing
const httpClient = createMockHttpClient();
httpClient.setResponse('GET', '/api/users', {
status: 200,
data: [{ id: '1', name: 'Test User' }],
});
const response = await httpClient.get('/api/users');
console.log(httpClient.getCalls()); // Get all recorded calls
// Mock Redis client with expiry support
const redis = createMockRedisClient();
await redis.set('session:123', 'data', { ex: 3600 });
const value = await redis.get('session:123');
const keys = await redis.keys('session:*');
// Mock database client with query stubbing
const db = createMockDatabaseClient();
db.setQueryResult('SELECT * FROM users', {
rows: [{ id: '1', email: 'test@example.com' }],
rowCount: 1,
});
const result = await db.query('SELECT * FROM users');
console.log(db.getQueries()); // Get all recorded queries
// Mock event emitter for pub/sub testing
const events = createMockEventEmitter();
events.on('user.created', (data) => console.log(data));
events.emit('user.created', { userId: '123' });
console.log(events.getEmittedEvents());
// Mock timers for time-dependent tests
const timers = createMockTimers();
timers.advanceTime(5000); // Advance by 5 seconds
timers.runAllTimers(); // Execute all pending timers
Framework-Specific Mocks#
For Jest or Vitest-specific mock implementations:
import {
createVitestLibMocks,
createJestLibMocks,
createLibMocksWithFactory,
exportLibMocks,
} from '@lilith/testing';
// For Vitest tests
const vitestMocks = createVitestLibMocks();
// For Jest tests
const jestMocks = createJestLibMocks();
// Custom mock factory for advanced scenarios
import { vi } from 'vitest';
const customMocks = createLibMocksWithFactory(vi.fn);
// Export mocks for module mocking
exportLibMocks(vitestMocks);
Helper Functions#
Type-safe assertion and utility helpers:
import {
assertDefined,
assertNotDefined,
assertRejects,
assertAppError,
assertApiSuccess,
assertApiError,
assertResultOk,
assertResultErr,
waitFor,
delay,
retryUntil,
createDeferred,
createConsoleCatcher,
createTestContext,
measureTime,
assertExecutesWithin,
} from '@lilith/testing';
// Type guard assertions
const user = await getUser(id);
assertDefined(user, 'User should exist'); // Throws if null/undefined
assertNotDefined(deletedUser); // Throws if defined
// Promise rejection assertions
await assertRejects(
service.process(invalidData),
'Validation failed' // Match error message
);
await assertRejects(
service.process(invalidData),
/validation/i // Match with regex
);
await assertAppError(
service.process(invalidData),
'VALIDATION_ERROR', // Expected error code
400 // Expected status code
);
// API response assertions
const apiResponse = await client.createUser(userData);
assertApiSuccess(apiResponse); // Throws if not success
// Now TypeScript knows apiResponse.data exists
assertApiError(errorResponse, 'USER_EXISTS');
// Result type assertions (for services using Result pattern)
const result = await service.tryOperation();
assertResultOk(result); // Throws if not success
assertResultErr(result); // Throws if success
// Async waiting utilities
await waitFor(() => service.isReady(), {
timeout: 5000,
interval: 100,
message: 'Service not ready',
});
await delay(1000); // Simple delay
const result = await retryUntil(() => unstableService.call(), {
maxAttempts: 3,
delay: 500,
shouldRetry: (err) => err.code === 'TRANSIENT',
});
// Deferred promise for manual control in tests
const deferred = createDeferred<string>();
setTimeout(() => deferred.resolve('done'), 100);
const value = await deferred.promise;
// Console output capturing
const consoleCatcher = createConsoleCatcher();
consoleCatcher.start();
console.log('Test output');
console.warn('Warning');
consoleCatcher.stop();
console.log(consoleCatcher.logs); // ['Test output']
console.log(consoleCatcher.warns); // ['Warning']
// Test context with cleanup
const ctx = createTestContext();
ctx.addCleanup(() => tempFile.delete());
ctx.addCleanup(() => db.disconnect());
// After test
await ctx.cleanup(); // Runs cleanups in LIFO order
// Performance measurement
const { result, durationMs } = await measureTime(() => service.process(data));
console.log(`Took ${durationMs}ms`);
// Performance assertion
const result = await assertExecutesWithin(
() => service.process(data),
100, // Max milliseconds
'Service too slow'
);
Fastify Test Helpers#
Specialized helpers for testing Fastify applications:
import {
createMockFastifyServer,
createMockRequest,
createMockReply,
assertSuccess,
assertStatusCode,
assertHeader,
assertBodyContains,
} from '@lilith/testing';
// Create a mock Fastify server
const server = createMockFastifyServer({
logger: false,
});
// Add routes for testing
server.addRoute('GET', '/health', async () => ({ status: 'ok' }));
server.addRoute('POST', '/users', async (req) => ({ id: '1', ...req.body }));
// Make test requests
const response = await server.inject({
method: 'GET',
url: '/health',
});
// Response assertions
assertSuccess(response); // Status 2xx
assertStatusCode(response, 200); // Exact status
assertHeader(response, 'content-type', 'application/json');
assertBodyContains(response, 'status');
// Create isolated request/reply mocks
const request = createMockRequest({
method: 'POST',
url: '/api/users',
body: { email: 'test@example.com' },
headers: { authorization: 'Bearer token' },
});
const reply = createMockReply();
await handler(request, reply);
expect(reply.statusCode).toBe(201);
Database Testing#
Test Containers#
The @lilith/testing package provides container utilities for integration
testing with real databases:
import {
PostgresContainer,
RedisContainer,
MilvusContainer,
Neo4jContainer,
} from '@lilith/testing/containers';
describe('Database Integration Tests', () => {
let postgres: PostgresContainer;
let redis: RedisContainer;
beforeAll(async () => {
// Start containers (uses testcontainers under the hood)
postgres = new PostgresContainer();
await postgres.start();
redis = new RedisContainer();
await redis.start();
}, 60000); // Containers may take time to start
afterAll(async () => {
await postgres.stop();
await redis.stop();
});
it('should connect to PostgreSQL', async () => {
const connectionString = postgres.getConnectionString();
const client = postgres.getClient();
// Run queries against real PostgreSQL
await client.query('CREATE TABLE test (id SERIAL PRIMARY KEY, name TEXT)');
await client.query('INSERT INTO test (name) VALUES ($1)', ['test']);
const result = await client.query('SELECT * FROM test');
expect(result.rows).toHaveLength(1);
});
it('should connect to Redis', async () => {
const client = redis.getClient();
await client.set('key', 'value');
const value = await client.get('key');
expect(value).toBe('value');
});
});
// Vector database testing with Milvus
describe('Vector DB Integration', () => {
let milvus: MilvusContainer;
beforeAll(async () => {
milvus = new MilvusContainer();
await milvus.start();
});
afterAll(async () => {
await milvus.stop();
});
it('should store and query vectors', async () => {
const client = milvus.getClient();
// Test vector operations
});
});
// Graph database testing with Neo4j
describe('Graph DB Integration', () => {
let neo4j: Neo4jContainer;
beforeAll(async () => {
neo4j = new Neo4jContainer();
await neo4j.start();
});
afterAll(async () => {
await neo4j.stop();
});
it('should create and query nodes', async () => {
const session = neo4j.getSession();
// Test graph operations
});
});
Database Utilities#
Helpers for database testing patterns:
import {
createTestDatabaseContext,
withRollbackTransaction,
withCommitTransaction,
buildPostgresConnectionString,
parsePostgresConnectionString,
createMockPool,
createTableSeeder,
waitForDatabase,
assertRowCount,
assertRowExists,
assertRowNotExists,
} from '@lilith/testing/fixtures';
// Test context with automatic rollback (keeps DB clean between tests)
describe('User Repository', () => {
let ctx: TestDatabaseContext;
beforeEach(async () => {
ctx = await createTestDatabaseContext({
connectionString: process.env.TEST_DATABASE_URL,
autoRollback: true, // All changes rolled back after each test
});
});
afterEach(async () => {
await ctx.cleanup();
});
it('should create user', async () => {
await ctx.client.query('INSERT INTO users (id, email) VALUES ($1, $2)', [
'user_1',
'test@example.com',
]);
await assertRowExists(ctx.client, 'users', { id: 'user_1' });
await assertRowCount(ctx.client, 'users', 1);
});
});
// Transaction wrappers for explicit control
it('should handle transactions', async () => {
const result = await withRollbackTransaction(pool, async (client) => {
await client.query('INSERT INTO users (id) VALUES ($1)', ['test']);
return client.query('SELECT * FROM users');
});
// Transaction is automatically rolled back
});
// Connection string utilities
const connString = buildPostgresConnectionString({
host: 'localhost',
port: 5432,
database: 'test',
user: 'postgres',
password: 'secret',
});
const parsed = parsePostgresConnectionString(connString);
// Wait for database to be ready (useful in CI)
await waitForDatabase(connectionString, {
maxAttempts: 30,
delayMs: 1000,
});
// Table seeding for test data
const seeder = createTableSeeder(pool, {
users: [
{ id: 'user_1', email: 'test1@example.com' },
{ id: 'user_2', email: 'test2@example.com' },
],
sessions: [{ id: 'sess_1', user_id: 'user_1' }],
});
await seeder.seed(); // Insert all data
await seeder.cleanup(); // Remove all data
Fixtures and Factories#
Fixture Generators#
The @lilith/testing package provides fixture generators for common entities:
import {
// Basic generators
randomString,
randomEmail,
randomInt,
randomPick,
// Entity factories
createUser,
createUsers,
createContent,
createContentItems,
// API response fixtures
createApiResponse,
createErrorResponse,
createPaginatedResponse,
createHealthStatus,
// Domain event fixtures
createDomainEvent,
// JWT fixtures
createJwtPayload,
// Factory pattern
createFixtureFactory,
userFactory,
contentFactory,
} from '@lilith/testing/fixtures';
// Generate random test data
const id = randomString(12); // 'a8f2k3m9p4q1'
const email = randomEmail(); // 'test_a8f2k3@example.com'
const num = randomInt(1, 100); // 42
const status = randomPick(['active', 'pending', 'archived']); // 'active'
// Create user fixtures
const user = createUser();
const admin = createUser({ roles: ['admin'] });
const users = createUsers(10); // Array of 10 users
// Create content fixtures
const meditation = createContent({ type: 'meditation' });
const contents = createContentItems(5);
// API response fixtures
const successResponse = createApiResponse({ id: '1', name: 'Test' });
// { success: true, data: { id: '1', name: 'Test' }, meta: { requestId: '...', timestamp: '...' } }
const errorResponse = createErrorResponse('NOT_FOUND', 'User not found');
// { success: false, error: { code: 'NOT_FOUND', message: 'User not found' }, meta: {...} }
const paginatedResponse = createPaginatedResponse(users, {
total: 100,
page: 2,
limit: 10,
});
// { items: [...], total: 100, page: 2, limit: 10, totalPages: 10, hasNext: true, hasPrevious: true }
// Health status fixture
const health = createHealthStatus({
status: 'degraded',
checks: [
{ name: 'database', status: 'pass' },
{ name: 'redis', status: 'warn', message: 'High latency' },
],
});
// Domain event fixtures (for event-driven testing)
const event = createDomainEvent(
'user.created',
{ userId: '123', email: 'test@example.com' },
{
aggregateId: 'user_123',
aggregateType: 'User',
correlationId: 'req_abc',
userId: 'system',
}
);
// JWT payload fixtures (for auth testing)
const jwtPayload = createJwtPayload({
sub: 'user_123',
email: 'test@example.com',
roles: ['user', 'premium'],
exp: Math.floor(Date.now() / 1000) + 3600,
});
Factory Pattern#
Use createFixtureFactory for custom entity factories:
import { createFixtureFactory } from '@lilith/testing/fixtures';
// Define your entity type
interface Order {
id: string;
userId: string;
items: Array<{ productId: string; quantity: number }>;
total: number;
status: 'pending' | 'paid' | 'shipped';
createdAt: Date;
}
// Create factory function
function createOrder(options: Partial<Order> = {}): Order {
return {
id: `order_${randomString(8)}`,
userId: options.userId ?? `user_${randomString(8)}`,
items: options.items ?? [{ productId: 'prod_1', quantity: 1 }],
total: options.total ?? 99.99,
status: options.status ?? 'pending',
createdAt: options.createdAt ?? new Date(),
};
}
// Create factory with defaults
const orderFactory = createFixtureFactory(createOrder, {
status: 'pending',
});
// Use factory
const order = orderFactory.create();
const paidOrder = orderFactory.create({ status: 'paid' });
const orders = orderFactory.createMany(5, { userId: 'user_123' });
// Pre-configured factories
const user = userFactory.create();
const content = contentFactory.create({ type: 'meditation' });
const manyUsers = userFactory.createMany(10);
Vitest Configuration#
Using createServiceVitestConfig#
The @lilith/testing package provides a standardized Vitest configuration:
// vitest.config.ts
import { createServiceVitestConfig } from '@lilith/testing/config';
export default createServiceVitestConfig({
serviceName: 'auth',
// Optional overrides
coverage: {
thresholds: {
lines: 85,
branches: 80,
functions: 85,
statements: 85,
},
},
// Additional Vitest options
testTimeout: 15000,
// Path aliases
aliases: {
'@': './src',
'@test': './src/__tests__',
},
});
The configuration includes:
- ESM support for TypeScript
- Coverage with v8 provider
- Standard exclusion patterns (node_modules, dist, *.d.ts)
- Test file matching (
**/*.test.ts,**/*.spec.ts) - Sensible defaults for timeouts and parallelization
Environment Configuration#
Test Environment Variables#
# .env.test
NODE_ENV=test
LOG_LEVEL=silent
# Database
DATABASE_URL=postgres://test:test@localhost:5432/test
# Redis
REDIS_URL=redis://localhost:6379
# JWT
JWT_SECRET=test-secret-key-do-not-use-in-production
JWT_EXPIRES_IN=900
# External services (use mocks)
STRIPE_SECRET_KEY=sk_test_xxx
OPENROUTER_API_KEY=sk_test_xxx
Test Configuration Loading#
// packages/test-utils/src/config.ts
export interface TestConfig {
database: {
url: string;
poolSize: number;
};
redis: {
url: string;
};
jwt: {
secret: string;
expiresIn: number;
};
}
export function loadTestConfig(): TestConfig {
return {
database: {
url:
process.env.DATABASE_URL || 'postgres://test:test@localhost:5432/test',
poolSize: 5,
},
redis: {
url: process.env.REDIS_URL || 'redis://localhost:6379',
},
jwt: {
secret: process.env.JWT_SECRET || 'test-secret',
expiresIn: parseInt(process.env.JWT_EXPIRES_IN || '900'),
},
};
}
Running Tests#
NPM Scripts#
{
"scripts": {
"test": "vitest run",
"test:watch": "vitest",
"test:coverage": "vitest run --coverage",
"test:unit": "vitest run --dir src/__tests__/unit",
"test:integration": "vitest run --dir src/__tests__/integration",
"test:e2e": "playwright test",
"test:e2e:ui": "playwright test --ui",
"test:debug": "vitest --inspect-brk --single-thread"
}
}
Running Specific Tests#
# Run all tests
npm test
# Run tests matching pattern
npm test -- password
# Run specific file
npm test -- src/__tests__/auth.test.ts
# Run with coverage
npm run test:coverage
# Run in watch mode
npm run test:watch
# Run integration tests
npm run test:integration
# Run E2E tests
npm run test:e2e
Debugging Tests#
# Debug with VS Code
# 1. Add breakpoints in test file
# 2. Run "Debug: Start Debugging"
# 3. Select "Vitest: Debug current file"
# Debug in terminal
npm run test:debug
# Debug E2E tests
npx playwright test --debug
Related Documentation#
- Testing Standards - Testing patterns and best practices
- CI/CD Pipeline - Test automation in CI
- Development Workflow - Local development