Auto-generated from source code - Last updated: 2025-09-28
Overview#
The Moderation Service provides real-time content moderation capabilities with comprehensive toxicity detection, policy enforcement, and user reputation management.
Base URL#
http://localhost:8080 (configurable via PORT environment variable)
Authentication#
All endpoints support optional Bearer token authentication via Authorization header.
API Endpoints#
Health Check#
GET /health#
Returns service health status with filter information.
Response Example:
{
"status": "healthy",
"service": "moderation",
"filters_active": 10,
"timestamp": "2025-09-28T15:45:00.000Z",
"version": "1.0.0"
}
Response Fields:
status: Service health status (healthy/degraded/unhealthy)service: Service identifierfilters_active: Number of active content filterstimestamp: Response generation timeversion: Service version
Content Moderation#
POST /v1/moderate#
Analyzes content for policy violations and toxicity.
Request Body:
{
"content": "Text content to moderate",
"user_id": "user123",
"context": "chat|comment|post",
"check_types": ["toxicity", "harassment", "spam"]
}
Request Fields:
content(required): Text content to analyzeuser_id(required): User identifier for reputation trackingcontext(optional): Content context for enhanced analysischeck_types(optional): Specific moderation categories to check
Response Example:
{
"flagged": false,
"action": "allow",
"moderation_id": "mod_1695987600_abc123",
"categories": {
"harassment": false,
"hate_speech": false,
"threats": false,
"profanity": false
},
"scores": {
"harassment": 0.1,
"hate_speech": 0.05,
"threats": 0.0,
"profanity": 0.2
},
"reasons": [],
"confidence": 0.95,
"processing_time": 45
}
Response Fields:
flagged: Whether content violates policiesaction: Recommended action (allow/review/block)moderation_id: Unique identifier for this moderation requestcategories: Boolean flags for each moderation categoryscores: Confidence scores (0-1) for each categoryreasons: Array of specific violation reasonsconfidence: Overall confidence in the moderation decisionprocessing_time: Analysis time in milliseconds
Filter Management#
GET /v1/filters#
Lists all available content filters and their configurations.
Response Example:
{
"filters": {
"toxicity": {
"enabled": true,
"threshold": 0.7,
"action": "block"
},
"harassment": {
"enabled": true,
"threshold": 0.6,
"action": "review"
}
},
"total_filters": 10
}
PUT /v1/filters/:category#
Updates configuration for a specific content filter.
Path Parameters:
category: Filter category (toxicity, harassment, threats, etc.)
Request Body:
{
"enabled": true,
"threshold": 0.8,
"action": "block"
}
Request Fields:
enabled(optional): Enable/disable the filterthreshold(optional): Confidence threshold (0-1) for triggeringaction(optional): Action to take (allow/review/block)
Response Example:
{
"success": true,
"filter": {
"category": "toxicity",
"enabled": true,
"threshold": 0.8,
"action": "block",
"updated_at": "2025-09-28T15:45:00.000Z"
}
}
User Reputation#
GET /v1/users/:userId/reputation#
Retrieves user reputation score and history.
Path Parameters:
userId: User identifier
Response Example:
{
"user_id": "user123",
"reputation_score": 0.85,
"total_interactions": 150,
"violations": 2,
"last_violation": "2025-09-20T10:30:00.000Z",
"account_standing": "good",
"restrictions": []
}
GET /v1/users/:userId/history#
Returns user's moderation history with pagination.
Query Parameters:
limit: Number of records to return (default: 50, max: 1000)offset: Number of records to skip (default: 0)start_date: Filter by start date (ISO 8601)end_date: Filter by end date (ISO 8601)
Response Example:
{
"user_id": "user123",
"total_records": 127,
"records": [
{
"moderation_id": "mod_1695987600_abc123",
"content_hash": "sha256:abc123...",
"flagged": false,
"action": "allow",
"timestamp": "2025-09-28T15:45:00.000Z",
"categories_flagged": []
}
],
"pagination": {
"limit": 50,
"offset": 0,
"has_more": true
}
}
Queue Management#
GET /v1/queue#
Returns items pending human review.
Query Parameters:
priority: Filter by priority (high/medium/low)category: Filter by violation categorylimit: Number of items to return
Response Example:
{
"queue_size": 23,
"items": [
{
"queue_id": "queue_abc123",
"content_hash": "sha256:def456...",
"user_id": "user456",
"categories": ["harassment"],
"priority": "high",
"created_at": "2025-09-28T15:30:00.000Z",
"expires_at": "2025-09-29T15:30:00.000Z"
}
]
}
POST /v1/queue/:queueId/decision#
Records human reviewer decision for queued content.
Path Parameters:
queueId: Queue item identifier
Request Body:
{
"decision": "approve|reject|escalate",
"reviewer_id": "reviewer123",
"notes": "Optional reviewer notes"
}
Error Responses#
All endpoints return consistent error responses:
{
"error": "Error description",
"code": "ERROR_CODE",
"details": {},
"timestamp": "2025-09-28T15:45:00.000Z",
"trace_id": "trace_abc123"
}
Common Error Codes:
INVALID_REQUEST: Malformed request bodyCONTENT_TOO_LARGE: Content exceeds size limitsRATE_LIMITED: Too many requestsFILTER_NOT_FOUND: Invalid filter categoryUSER_NOT_FOUND: Invalid user ID
Rate Limiting#
- Standard endpoints: 1000 requests/hour per IP
- Moderation endpoint: 100 requests/minute per user
- Admin endpoints: 500 requests/hour per API key
Content Filtering Categories#
- Toxicity: General toxic language detection
- Harassment: Personal attacks and bullying
- Hate Speech: Content targeting protected groups
- Threats: Violent threats and intimidation
- Self Harm: Content promoting self-harm
- Sexual Content: Inappropriate sexual material
- Violence: Graphic violence descriptions
- Profanity: Offensive language detection
- Spam: Repetitive or promotional content
- Misinformation: False or misleading claims
- PII Exposure: Personal information leaks
Integration Examples#
Basic Content Moderation#
const response = await fetch('/v1/moderate', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': 'Bearer your-token'
},
body: JSON.stringify({
content: 'User submitted content',
user_id: 'user123',
context: 'chat'
})
});
const result = await response.json();
if (result.flagged) {
console.log('Content flagged:', result.reasons);
// Handle policy violation
} else {
console.log('Content approved');
// Allow content through
}
Filter Configuration#
// Disable harassment filter
await fetch('/v1/filters/harassment', {
method: 'PUT',
headers: {
'Content-Type': 'application/json',
'Authorization': 'Bearer admin-token'
},
body: JSON.stringify({
enabled: false
})
});
User Reputation Check#
const reputation = await fetch(`/v1/users/${userId}/reputation`)
.then(r => r.json());
if (reputation.reputation_score < 0.5) {
// Apply additional scrutiny
console.log('Low reputation user');
}
Testing#
This API documentation is tested and validated through comprehensive test suites with 74 passing tests covering all endpoints and scenarios.
This documentation is auto-generated from the moderation service source code and verified through automated testing.