This guide covers integration testing for the Lilith platform, including database testing with test containers, fixtures, and cleanup utilities.
Overview#
The @lilith/testing package provides comprehensive utilities for integration
testing:
- Test Containers: Real database instances using testcontainers
- Database Fixtures: Seeding utilities for test data
- Cleanup Scripts: Utilities for test isolation
- Helpers: Assertion and utility functions
Installation#
The testing package is available as a workspace package:
typescript
import {
PostgresContainer,
RedisContainer,
MilvusContainer,
Neo4jContainer,
} from '@lilith/testing/containers';
import {
PostgresFixtures,
RedisFixtures,
Neo4jFixtures,
} from '@lilith/testing/fixtures';
import {
cleanupPostgres,
cleanupRedis,
cleanupNeo4j,
CleanupManager,
} from '@lilith/testing/helpers';
Test Containers#
PostgreSQL Container#
typescript
import { PostgresContainer } from '@lilith/testing/containers';
import { describe, beforeAll, afterAll, it } from 'vitest';
describe('User Repository', () => {
let container: PostgresContainer;
beforeAll(async () => {
container = new PostgresContainer({
database: 'users_test',
enablePgVector: true, // Enable pgvector extension
});
await container.start();
}, 60000); // 60 second timeout for container startup
afterAll(async () => {
await container.stop();
});
it('should connect to database', async () => {
const connectionString = container.getConnectionString();
// Use with pg, drizzle, prisma, etc.
expect(connectionString).toContain('postgresql://');
});
it('should execute SQL', async () => {
const result = await container.exec('SELECT 1 as test');
expect(result.exitCode).toBe(0);
});
});
PostgreSQL Options#
typescript
interface PostgresContainerOptions {
imageTag?: string; // Default: '16-alpine'
database?: string; // Default: 'test_db'
username?: string; // Default: 'test_user'
password?: string; // Default: 'test_password'
enablePgVector?: boolean; // Default: true
startupTimeout?: number; // Default: 60000
}
Redis Container#
typescript
import { RedisContainer } from '@lilith/testing/containers';
describe('Cache Service', () => {
let container: RedisContainer;
beforeAll(async () => {
container = new RedisContainer({
password: 'test_password', // Optional
});
await container.start();
});
afterAll(async () => {
await container.stop();
});
it('should set and get values', async () => {
await container.set('key', 'value', 60);
const result = await container.get('key');
expect(result).toBe('value');
});
it('should flush all data', async () => {
await container.flushAll();
});
});
Neo4j Container#
typescript
import { Neo4jContainer } from '@lilith/testing/containers';
describe('Graph Queries', () => {
let container: Neo4jContainer;
beforeAll(async () => {
container = new Neo4jContainer({
imageTag: '5-community',
password: 'test_password',
});
await container.start();
await container.waitForReady();
}, 90000);
afterAll(async () => {
await container.stop();
});
it('should execute Cypher queries', async () => {
await container.execCypher(`
CREATE (u:User {name: 'Alice'})
RETURN u
`);
const result = await container.execCypher(`
MATCH (u:User) RETURN count(u) as count
`);
expect(result).toBeDefined();
});
it('should clear database', async () => {
await container.clearDatabase();
});
});
Milvus Container#
typescript
import { MilvusContainer } from '@lilith/testing/containers';
describe('Vector Search', () => {
let container: MilvusContainer;
beforeAll(async () => {
container = new MilvusContainer({
imageTag: 'v2.3-latest',
});
await container.start();
await container.waitForReady();
}, 120000); // Milvus needs more time
afterAll(async () => {
await container.stop();
});
it('should provide connection info', () => {
const address = container.getAddress();
expect(address).toMatch(/localhost:\d+/);
});
});
Database Fixtures#
PostgreSQL Fixtures#
typescript
import { PostgresContainer } from '@lilith/testing/containers';
import { PostgresFixtures } from '@lilith/testing/fixtures';
describe('Content API', () => {
let container: PostgresContainer;
let fixtures: PostgresFixtures;
beforeAll(async () => {
container = new PostgresContainer();
await container.start();
fixtures = new PostgresFixtures(container);
await fixtures.createAllTables();
});
beforeEach(async () => {
// Seed test data
const userIds = await fixtures.seedUsers(5);
await fixtures.seedContent(10, userIds[0]);
});
afterEach(async () => {
await fixtures.clearAll();
});
it('should query seeded data', async () => {
const result = await container.exec('SELECT COUNT(*) FROM users');
expect(result.stdout).toContain('5');
});
});
Available PostgreSQL Fixture Methods#
createUsersTable()- Create users schemacreateContentTable()- Create content schemacreateConversationsTable()- Create conversations and messages schemacreateEmbeddingsTable(dimensions)- Create vector embeddings tablecreateAllTables()- Create all base tablesseedUsers(count)- Insert test usersseedContent(count, authorId?)- Insert test contentseedConversation(userId, messageCount)- Insert conversation with messagesseedEmbeddings(count, dimensions)- Insert vector embeddingsclearAll()- Truncate all tables
Redis Fixtures#
typescript
import { RedisContainer } from '@lilith/testing/containers';
import { RedisFixtures } from '@lilith/testing/fixtures';
describe('Session Service', () => {
let container: RedisContainer;
let fixtures: RedisFixtures;
beforeAll(async () => {
container = new RedisContainer();
await container.start();
fixtures = new RedisFixtures(container);
});
it('should seed sessions', async () => {
const sessionIds = await fixtures.seedSessions(10, 3600);
expect(sessionIds).toHaveLength(10);
});
it('should seed feature flags', async () => {
await fixtures.seedFeatureFlags({
'dark-mode': true,
'beta-features': false,
});
});
});
Neo4j Fixtures#
typescript
import { Neo4jContainer } from '@lilith/testing/containers';
import { Neo4jFixtures } from '@lilith/testing/fixtures';
describe('Recommendation Engine', () => {
let container: Neo4jContainer;
let fixtures: Neo4jFixtures;
beforeAll(async () => {
container = new Neo4jContainer();
await container.start();
await container.waitForReady();
fixtures = new Neo4jFixtures(container);
});
it('should seed knowledge graph', async () => {
const graph = await fixtures.seedKnowledgeGraph({
userCount: 10,
contentCount: 20,
topics: ['meditation', 'yoga', 'mindfulness'],
});
expect(graph.userIds).toHaveLength(10);
expect(graph.contentIds).toHaveLength(20);
});
});
Cleanup Utilities#
Basic Cleanup#
typescript
import {
cleanupPostgres,
cleanupRedis,
cleanupNeo4j,
} from '@lilith/testing/helpers';
afterEach(async () => {
await cleanupPostgres(postgresContainer);
await cleanupRedis(redisContainer);
await cleanupNeo4j(neo4jContainer);
});
Cleanup Manager#
For managing multiple databases:
typescript
import { CleanupManager } from '@lilith/testing/helpers';
describe('Multi-Database Test', () => {
let cleanupManager: CleanupManager;
let postgres: PostgresContainer;
let redis: RedisContainer;
beforeAll(async () => {
cleanupManager = new CleanupManager();
postgres = new PostgresContainer();
redis = new RedisContainer();
await postgres.start();
await redis.start();
cleanupManager.registerPostgres(postgres);
cleanupManager.registerRedis(redis);
});
afterEach(async () => {
const result = await cleanupManager.cleanupAll();
if (!result.success) {
console.warn('Cleanup errors:', result.errors);
}
});
afterAll(async () => {
await postgres.stop();
await redis.stop();
});
});
Cleanup Options#
typescript
// PostgreSQL cleanup options
await cleanupPostgres(container, {
excludeTables: ['migrations'], // Don't truncate these tables
includeTables: ['users', 'posts'], // Only truncate these tables
resetSequences: true, // Reset auto-increment sequences
vacuum: true, // Run VACUUM ANALYZE
});
// Redis cleanup options
await cleanupRedis(container, {
patterns: ['session:*', 'cache:*'], // Only delete matching keys
flushAll: false, // Don't flush entire database
});
// Neo4j cleanup options
await cleanupNeo4j(container, {
excludeLabels: ['Config'], // Don't delete these node labels
includeLabels: ['User', 'Content'], // Only delete these labels
dropConstraints: false, // Keep constraints
dropIndexes: false, // Keep indexes
});
Best Practices#
1. Container Timeouts#
Always set appropriate timeouts for container startup:
typescript
beforeAll(async () => {
container = new PostgresContainer();
await container.start();
}, 60000); // 60 seconds for Postgres
beforeAll(async () => {
container = new MilvusContainer();
await container.start();
}, 120000); // 120 seconds for Milvus
2. Parallel Test Isolation#
Use unique databases for parallel test files:
typescript
const testId = process.env.VITEST_POOL_ID || '0';
container = new PostgresContainer({
database: `test_db_${testId}`,
});
3. Reuse Containers in Test Suites#
Don't restart containers between tests:
typescript
// Good - reuse container
beforeAll(async () => {
container = new PostgresContainer();
await container.start();
});
afterEach(async () => {
await cleanupPostgres(container);
});
afterAll(async () => {
await container.stop();
});
4. Use Fixtures for Complex Data#
typescript
// Good - use fixtures for complex seeding
const { userIds, contentIds, topics } = await fixtures.seedKnowledgeGraph({
userCount: 10,
contentCount: 50,
});
// Avoid - manual seeding for each test
for (let i = 0; i < 10; i++) {
await container.exec(`INSERT INTO users...`);
}
5. Health Checks Before Tests#
typescript
beforeAll(async () => {
container = new PostgresContainer();
await container.start();
// Wait for container to be fully ready
const isHealthy = await container.isHealthy();
expect(isHealthy).toBe(true);
});
CI/CD Configuration#
GitHub Actions#
yaml
name: Integration Tests
on: [push, pull_request]
jobs:
integration-tests:
runs-on: ubuntu-latest
services:
# Note: testcontainers manages its own containers,
# but you may need Docker available
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run integration tests
run: npm run test:integration
env:
TESTCONTAINERS_RYUK_DISABLED: 'true' # If using Ryuk causes issues
Environment Variables#
bash
# Container configuration
TESTCONTAINERS_RYUK_DISABLED=true # Disable resource reaper if needed
DOCKER_HOST=unix:///var/run/docker.sock
# Test configuration
TEST_POSTGRES_IMAGE=postgres:16-alpine
TEST_REDIS_IMAGE=redis:7-alpine
TEST_MILVUS_IMAGE=milvusdb/milvus:v2.3-latest
TEST_NEO4J_IMAGE=neo4j:5-community
Troubleshooting#
Container Startup Failures#
- Check Docker is running:
docker info - Check available memory: Containers need sufficient RAM
- Check port conflicts: Other services might use required ports
- Increase timeouts: Some containers take longer on first pull
Test Isolation Issues#
- Always use cleanup: Call cleanup after each test
- Check for shared state: Ensure tests don't depend on each other
- Use unique identifiers: Generate unique IDs for test data
Slow Tests#
- Reuse containers: Don't restart between tests
- Use lightweight images: Alpine variants are faster
- Batch operations: Seed data in batches
- Parallelize carefully: Balance between speed and resource usage
See Also#
- External Integration Tests - Testing external APIs
- Unit Testing Guide - Mocking and unit tests
- E2E Testing Guide - End-to-end tests with Playwright