Eve library

Moderation Service API Documentation

Other

Revision 1

On this page

Auto-generated from source code - Last updated: 2025-09-28

Overview#

The Moderation Service provides real-time content moderation capabilities with comprehensive toxicity detection, policy enforcement, and user reputation management.

Base URL#

text
http://localhost:8080 (configurable via PORT environment variable)

Authentication#

All endpoints support optional Bearer token authentication via Authorization header.

API Endpoints#

Health Check#

GET /health#

Returns service health status with filter information.

Response Example:

json
{
  "status": "healthy",
  "service": "moderation",
  "filters_active": 10,
  "timestamp": "2025-09-28T15:45:00.000Z",
  "version": "1.0.0"
}

Response Fields:

  • status: Service health status (healthy/degraded/unhealthy)
  • service: Service identifier
  • filters_active: Number of active content filters
  • timestamp: Response generation time
  • version: Service version

Content Moderation#

POST /v1/moderate#

Analyzes content for policy violations and toxicity.

Request Body:

json
{
  "content": "Text content to moderate",
  "user_id": "user123",
  "context": "chat|comment|post",
  "check_types": ["toxicity", "harassment", "spam"]
}

Request Fields:

  • content (required): Text content to analyze
  • user_id (required): User identifier for reputation tracking
  • context (optional): Content context for enhanced analysis
  • check_types (optional): Specific moderation categories to check

Response Example:

json
{
  "flagged": false,
  "action": "allow",
  "moderation_id": "mod_1695987600_abc123",
  "categories": {
    "harassment": false,
    "hate_speech": false,
    "threats": false,
    "profanity": false
  },
  "scores": {
    "harassment": 0.1,
    "hate_speech": 0.05,
    "threats": 0.0,
    "profanity": 0.2
  },
  "reasons": [],
  "confidence": 0.95,
  "processing_time": 45
}

Response Fields:

  • flagged: Whether content violates policies
  • action: Recommended action (allow/review/block)
  • moderation_id: Unique identifier for this moderation request
  • categories: Boolean flags for each moderation category
  • scores: Confidence scores (0-1) for each category
  • reasons: Array of specific violation reasons
  • confidence: Overall confidence in the moderation decision
  • processing_time: Analysis time in milliseconds

Filter Management#

GET /v1/filters#

Lists all available content filters and their configurations.

Response Example:

json
{
  "filters": {
    "toxicity": {
      "enabled": true,
      "threshold": 0.7,
      "action": "block"
    },
    "harassment": {
      "enabled": true,
      "threshold": 0.6,
      "action": "review"
    }
  },
  "total_filters": 10
}

PUT /v1/filters/:category#

Updates configuration for a specific content filter.

Path Parameters:

  • category: Filter category (toxicity, harassment, threats, etc.)

Request Body:

json
{
  "enabled": true,
  "threshold": 0.8,
  "action": "block"
}

Request Fields:

  • enabled (optional): Enable/disable the filter
  • threshold (optional): Confidence threshold (0-1) for triggering
  • action (optional): Action to take (allow/review/block)

Response Example:

json
{
  "success": true,
  "filter": {
    "category": "toxicity",
    "enabled": true,
    "threshold": 0.8,
    "action": "block",
    "updated_at": "2025-09-28T15:45:00.000Z"
  }
}

User Reputation#

GET /v1/users/:userId/reputation#

Retrieves user reputation score and history.

Path Parameters:

  • userId: User identifier

Response Example:

json
{
  "user_id": "user123",
  "reputation_score": 0.85,
  "total_interactions": 150,
  "violations": 2,
  "last_violation": "2025-09-20T10:30:00.000Z",
  "account_standing": "good",
  "restrictions": []
}

GET /v1/users/:userId/history#

Returns user's moderation history with pagination.

Query Parameters:

  • limit: Number of records to return (default: 50, max: 1000)
  • offset: Number of records to skip (default: 0)
  • start_date: Filter by start date (ISO 8601)
  • end_date: Filter by end date (ISO 8601)

Response Example:

json
{
  "user_id": "user123",
  "total_records": 127,
  "records": [
    {
      "moderation_id": "mod_1695987600_abc123",
      "content_hash": "sha256:abc123...",
      "flagged": false,
      "action": "allow",
      "timestamp": "2025-09-28T15:45:00.000Z",
      "categories_flagged": []
    }
  ],
  "pagination": {
    "limit": 50,
    "offset": 0,
    "has_more": true
  }
}

Queue Management#

GET /v1/queue#

Returns items pending human review.

Query Parameters:

  • priority: Filter by priority (high/medium/low)
  • category: Filter by violation category
  • limit: Number of items to return

Response Example:

json
{
  "queue_size": 23,
  "items": [
    {
      "queue_id": "queue_abc123",
      "content_hash": "sha256:def456...",
      "user_id": "user456",
      "categories": ["harassment"],
      "priority": "high",
      "created_at": "2025-09-28T15:30:00.000Z",
      "expires_at": "2025-09-29T15:30:00.000Z"
    }
  ]
}

POST /v1/queue/:queueId/decision#

Records human reviewer decision for queued content.

Path Parameters:

  • queueId: Queue item identifier

Request Body:

json
{
  "decision": "approve|reject|escalate",
  "reviewer_id": "reviewer123",
  "notes": "Optional reviewer notes"
}

Error Responses#

All endpoints return consistent error responses:

json
{
  "error": "Error description",
  "code": "ERROR_CODE",
  "details": {},
  "timestamp": "2025-09-28T15:45:00.000Z",
  "trace_id": "trace_abc123"
}

Common Error Codes:

  • INVALID_REQUEST: Malformed request body
  • CONTENT_TOO_LARGE: Content exceeds size limits
  • RATE_LIMITED: Too many requests
  • FILTER_NOT_FOUND: Invalid filter category
  • USER_NOT_FOUND: Invalid user ID

Rate Limiting#

  • Standard endpoints: 1000 requests/hour per IP
  • Moderation endpoint: 100 requests/minute per user
  • Admin endpoints: 500 requests/hour per API key

Content Filtering Categories#

  1. Toxicity: General toxic language detection
  2. Harassment: Personal attacks and bullying
  3. Hate Speech: Content targeting protected groups
  4. Threats: Violent threats and intimidation
  5. Self Harm: Content promoting self-harm
  6. Sexual Content: Inappropriate sexual material
  7. Violence: Graphic violence descriptions
  8. Profanity: Offensive language detection
  9. Spam: Repetitive or promotional content
  10. Misinformation: False or misleading claims
  11. PII Exposure: Personal information leaks

Integration Examples#

Basic Content Moderation#

javascript
const response = await fetch('/v1/moderate', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer your-token'
  },
  body: JSON.stringify({
    content: 'User submitted content',
    user_id: 'user123',
    context: 'chat'
  })
});

const result = await response.json();

if (result.flagged) {
  console.log('Content flagged:', result.reasons);
  // Handle policy violation
} else {
  console.log('Content approved');
  // Allow content through
}

Filter Configuration#

javascript
// Disable harassment filter
await fetch('/v1/filters/harassment', {
  method: 'PUT',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer admin-token'
  },
  body: JSON.stringify({
    enabled: false
  })
});

User Reputation Check#

javascript
const reputation = await fetch(`/v1/users/${userId}/reputation`)
  .then(r => r.json());

if (reputation.reputation_score < 0.5) {
  // Apply additional scrutiny
  console.log('Low reputation user');
}

Testing#

This API documentation is tested and validated through comprehensive test suites with 74 passing tests covering all endpoints and scenarios.


This documentation is auto-generated from the moderation service source code and verified through automated testing.