Eve library

Security Incident Response Runbooks

Guide

Revision 1

On this page

Threat Detection & Response Procedures#

High-Severity Anomaly Response#

Detection Criteria#

  • Security event spike > 3-sigma deviation from baseline
  • Critical severity events > threshold
  • Multiple simultaneous threat types

Response Procedure#

  1. Immediate Assessment (0-5 minutes)

    • Access security dashboard: GET /v1/security/dashboard
    • Identify affected services and threat types
    • Check system health: GET /v1/health
  2. Threat Analysis (5-15 minutes)

    • Correlate with application logs
    • Check rate limiter telemetry
    • Analyze traffic patterns in WAF/CDN
  3. Containment (15-30 minutes)

    • Apply rate limiting if needed
    • Block malicious IPs via WAF
    • Scale affected services if under load
  4. Investigation (30-60 minutes)

    • Deep dive into affected user accounts
    • Check for credential stuffing patterns
    • Analyze attack vectors and entry points

Escalation Criteria#

  • Critical severity events > 10/hour
  • Multiple service compromise indicators
  • Data exfiltration attempts detected

Authentication Failure Investigation#

Common Patterns#

  • Credential Stuffing: High volume auth failures from multiple IPs
  • Brute Force: Repeated failures from single IP
  • Account Takeover: Successful auth after multiple failures

Investigation Steps#

  1. Query auth service logs: kubectl logs -n lilith deployment/auth-service
  2. Check rate limiting metrics
  3. Analyze IP geolocation patterns
  4. Review affected user accounts

Mitigation Actions#

  • Temporarily block suspicious IPs
  • Force password reset for affected accounts
  • Enable additional MFA requirements
  • Update rate limiting thresholds

Safety Policy Violations#

Detection#

  • safety.policy_violation events
  • moderation.flagged content
  • Persona boundary violations

Response Process#

  1. Review flagged content in moderation queue
  2. Analyze user interaction patterns
  3. Update safety filters if needed
  4. Escalate to safety team for review

Automated Response Triggers#

High-Priority Alerts#

  • Critical security events: Immediate PagerDuty alert
  • Anomaly detection: Slack notification
  • Service health degradation: Auto-scaling trigger

Response Automation#

javascript
// Example automated response configuration
const responseConfig = {
  'rate_limit_exceeded': {
    severity: 'medium',
    action: 'increase_rate_limits',
    cooldown: '5m'
  },
  'jailbreak_attempt': {
    severity: 'high',
    action: 'alert_security_team',
    escalation: 'immediate'
  },
  'auth_failed': {
    severity: 'low',
    action: 'track_ip_patterns',
    threshold: 50
  }
};

Monitoring Commands#

Dashboard Access#

bash
# Security dashboard
curl ${OBSERVABILITY_URL}/v1/security/dashboard

# Raw security events
curl ${OBSERVABILITY_URL}/v1/security/events?timeframe=1h

# Anomaly details
curl ${OBSERVABILITY_URL}/v1/security/anomalies

Kubernetes Investigation#

bash
# Check security-related pods
kubectl get pods -n lilith -l security=monitoring

# View security event logs
kubectl logs -n monitoring deployment/observability-service | grep security

# Check network policies
kubectl get networkpolicies -n lilith

Generated automatically by security-docs-generator.js Last updated: 2025-09-27T18:05:41.712Z