Date: 2026-04-11 Scope: Every task needed to close all gaps for full ML sovereignty Method: Cross-referenced codebase analysis, data flywheel mapping, and April 2026 SOTA research across LLMs, vision, audio, 3D, training techniques, inference optimization, and MLOps.
How This Document Is Organized#
- Phase A — Data Flywheel (passive training data collection from normal system operation)
- Phase B — Centralized Dataset Management & Synthetic Data Generation
- Phase C — Production Model Serving (SOTA 2026)
- Phase D — Training & Fine-Tuning Infrastructure (SOTA 2026)
- Phase E — Alignment & Safety Training (SOTA 2026)
- Phase F — Continuous Evaluation & Benchmarking
- Phase G — MLOps, Experiment Management & Governance
- Phase H — Edge & On-Device Deployment
- Phase I — Domain-Specific Training Pipelines
- Phase J — GPU Infrastructure & Training Clusters
- Phase K — SOTA Model Integration (April 2026 open-weight models)
Each task is a concrete, implementable unit of work. No stubs.
Phase A: Data Flywheel — Passive Training Data Generation#
Every interaction in the system should produce training data as a byproduct. These tasks wire up collectors at each domain touchpoint identified in the codebase audit.
A.1 Iris Conversation Data Collection#
- [ ] A.1.1 Build conversation-to-training-pair exporter that converts completed Iris turns (user message + assistant response + turn metadata from conversation-core ConversationEngineEvent) into SFT training examples in ShareGPT/OpenAI format
- [ ] A.1.2 Build preference pair collector from Iris feedback system (memory/personalization/feedback FeedbackProcessor: positive/negative/correction signals → chosen/rejected pairs for DPO)
- [ ] A.1.3 Build implicit preference detector from Iris conversation analytics (message.streaming → regeneration detection, context.summarized → quality signal, tool.failed → correction signal)
- [ ] A.1.4 Build conversation quality scorer that assigns quality labels to completed conversations using multi-dimensional rubric (helpfulness, accuracy, safety, style) for reward model training
- [ ] A.1.5 Build model routing outcome logger that records which model was selected by @iris/model-routing, the response quality, and user satisfaction — producing routing optimization training data
- [ ] A.1.6 Build RAG retrieval feedback collector from Iris knowledge/ interactions (which retrieved documents were used in final response vs ignored → retrieval quality labels)
- [ ] A.1.7 Build tool-use training data collector from Iris agent tool calls (tool.called + tool.completed events → function calling training examples)
- [ ] A.1.8 Build conversation summarization training data from Iris context.summarized events (original messages + generated summary → summarization pairs)
- [ ] A.1.9 Build embedding quality feedback from Iris memory/embeddings (similar items retrieved → positive pairs; irrelevant items → negative pairs for contrastive training)
- [ ] A.1.10 Build user personalization training data from Iris adaptation rules (which adaptation rules improved user satisfaction → personalization model training data)
A.2 Psyche Avatar & Interaction Data Collection#
- [ ] A.2.1 Build emotion recognition training data pipeline from Psyche emotion-recognition EmotionTrack (detected emotions with confidence + FACS action units + micro-expressions → emotion classifier training data, filtered by quality metrics: face visibility > 0.8, landmark stability > 0.7)
- [ ] A.2.2 Build avatar expression training data from Psyche avatar-expressions (FLAME parameters: 100 shape + 50 expression + pose coefficients per frame → expression model training data)
- [ ] A.2.3 Build lip-sync training data from Psyche voice-engine (phoneme sequences + prosody data + LipSyncFrame viseme weights → audio-to-face training pairs)
- [ ] A.2.4 Build voice quality training data from Psyche voice-engine (synthesized audio + user engagement metrics → TTS quality ranking data)
- [ ] A.2.5 Build behavior coordination training data from Psyche behavior-coordinator (behavior sequences executed + transitions + user engagement response → behavior policy training data)
- [ ] A.2.6 Build gaze tracking calibration data from Psyche perception-engine (tracked gaze + baseline calibration per user → gaze model fine-tuning data)
- [ ] A.2.7 Build avatar rendering quality data from Psyche avatar-core (render quality presets selected + FPS achieved + user quality complaints → adaptive quality model training data)
A.3 Isis Generation Data Collection#
- [ ] A.3.1 Build generation prompt-quality pairing pipeline from Isis job-envelope + outputs (text prompt + negative prompt + parameters + output files + ProvenanceRecord → prompt engineering training data with quality labels)
- [ ] A.3.2 Build user selection/rejection collector from Isis outputs (which generated outputs users selected vs discarded/regenerated → image/video/audio preference pairs for diffusion alignment)
- [ ] A.3.3 Build generation parameter optimization data from Isis operation-orchestrator (parameter combinations + quality tier assignments → parameter recommendation model training data)
- [ ] A.3.4 Build LoRA effectiveness tracker from Isis model-fine-tuning (which LoRA adapters produce higher-quality outputs for specific styles/subjects → LoRA routing training data)
- [ ] A.3.5 Build ComfyUI workflow outcome data from Isis workflows (workflow configurations + execution results + quality scores → workflow recommendation training data)
- [ ] A.3.6 Build 3D generation quality data from Isis 3d-inference-local (model used + input type + output quality metrics from 3d-quality-gates → 3D model quality prediction training data)
- [ ] A.3.7 Build video enhancement before/after pairs from Isis video-enhancement (original + enhanced frames with RIFE interpolation / upscaling → super-resolution training pairs)
- [ ] A.3.8 Build texture generation quality data from Isis ai-texturing (input mesh + generated texture + quality score → texture generation training data)
A.4 Sophia Knowledge & RAG Data Collection#
- [ ] A.4.1 Build search query-result relevance data from Sophia semantic-search (query text + filters + returned results + clicked results vs ignored → relevance training data for reranker fine-tuning)
- [ ] A.4.2 Build citation verification training data from Sophia evaluation (citation text + verification status + citation checks → citation quality classifier training data)
- [ ] A.4.3 Build RAG faithfulness training data from Sophia evaluation (question + retrieved context + generated answer + faithfulness/relevancy scores → RAG quality training data for RAGAS-style evaluation models)
- [ ] A.4.4 Build document chunking quality data from Sophia document-parser (document + chunking strategy + downstream retrieval quality → optimal chunking model training data)
- [ ] A.4.5 Build embedding model comparison data from Sophia embeddings (same query against multiple embedding models + retrieval quality per model → embedding model selection training data)
- [ ] A.4.6 Build knowledge graph entity extraction training data from Sophia knowledge-base (document + extracted entities + verified relationships → NER and relation extraction training data)
A.5 Kuan Yin Safety & Moderation Data Collection#
- [ ] A.5.1 Build moderation decision training data from Kuan Yin foundation (content + HarmCategory detected + severity + confidence → harm classifier training data, respecting privacy: PII scrubbed before storage)
- [ ] A.5.2 Build intent classification training data from Kuan Yin foundation (content + detected Intent + valence + confidence → intent classifier training data)
- [ ] A.5.3 Build appeal outcome training data from Kuan Yin rehabilitation (original decision + appeal reason + appeal outcome → moderation quality improvement data for false positive reduction)
- [ ] A.5.4 Build emotional state modeling data from Kuan Yin foundation (content + EmotionalState with Plutchik wheel dimensions + arousal/valence/dominance → emotion model training data)
- [ ] A.5.5 Build intervention effectiveness data from Kuan Yin foundation (intervention category + urgency + applied + user response + escalation → intervention policy training data)
A.6 Hathor NPC & Worldbuilding Data Collection#
- [ ] A.6.1 Build NPC dialogue training data from Hathor llm-npc (NPCPersonality + player query + world state + generated dialogue + quality assessment → character-consistent dialogue fine-tuning data)
- [ ] A.6.2 Build NPC emotional response data from Hathor llm-npc (interaction context + NPCEmotionalState transitions + player engagement → emotion-aware dialogue model training data)
- [ ] A.6.3 Build NPC memory relevance data from Hathor llm-npc (retrieved memories + access count + memory importance + dialogue quality → memory retrieval model training data)
- [ ] A.6.4 Build world simulation outcome data from Hathor simulation (economy transactions + political events + player decisions + downstream consequences → world model training data)
- [ ] A.6.5 Build NPC personality consistency data from Hathor llm-npc (Big Five traits + speech patterns + generated responses → personality consistency evaluation data)
A.7 Veritas/Yaa News Data Collection#
- [ ] A.7.1 Build article classification training data from Yaa NLP pipeline (articles + NER results + topic classifications → NER and topic classifier fine-tuning data, especially for Ghanaian entities)
- [ ] A.7.2 Build fact-checking training data from Yaa fact-verification (claims + evidence + verification outcomes → fact-checking model training data)
- [ ] A.7.3 Build bias detection training data from Yaa bias analysis (articles + GPL scores + political lean annotations → bias classifier training data)
- [ ] A.7.4 Build editorial decision training data from Yaa newsroom agents (agent decisions + story selections + placement choices → editorial AI training data)
- [ ] A.7.5 Build multilingual NLP training data from Yaa Ghana NLP integration (Twi/Ewe/Ga/Hausa text processing results → low-resource language model fine-tuning data)
A.8 Euterpe Music & Aphrodite Content Data Collection#
- [ ] A.8.1 Build music preference data from Euterpe analytics (plays + skips + favorites + completion % + sharing → music generation quality ranking data)
- [ ] A.8.2 Build audio generation quality data from Euterpe genesis (generation prompts + parameters + outputs + user selections → audio generation preference data)
- [ ] A.8.3 Build content recommendation feedback from Aphrodite recommendation-engine (recommendations shown + clicked + ignored + engagement duration → recommendation model training data)
- [ ] A.8.4 Build content moderation training data from Aphrodite content-safety (content items + moderation decisions + appeals → content classifier training data)
A.9 Minerva/Metis Education Data Collection#
- [ ] A.9.1 Build lecture quality training data from Minerva lecture generation (generated lectures + student engagement metrics + assessment performance → lecture generation quality data)
- [ ] A.9.2 Build assessment effectiveness data from Minerva tutoring (questions + student answers + time to answer + follow-up needed → assessment quality training data)
- [ ] A.9.3 Build learning progression data from Minerva learning-system (student knowledge states from DKT + skill acquisition rates → adaptive learning model training data)
- [ ] A.9.4 Build VLM diagram quality data from Minerva VLM (diagram descriptions + generated diagrams + educational effectiveness → VLM fine-tuning data)
A.10 Cross-Domain Data Collection Infrastructure#
- [ ] A.10.1 Build centralized training data sink service that subscribes to all domain event buses and routes training-relevant events to the dataset registry with consistent schema (domain, data type, timestamp, quality score, format)
- [ ] A.10.2 Build privacy-first data collection pipeline with automatic PII detection and scrubbing (using Presidio/GLiNER) before any training data is stored
- [ ] A.10.3 Build consent management system that tracks user opt-in/opt-out for training data collection per domain and data type
- [ ] A.10.4 Build data collection rate limiter to prevent storage bloat (sample strategically: high-quality examples always stored, average examples sampled at configurable rate)
- [ ] A.10.5 Build training data deduplication service (near-duplicate detection using MinHash/SimHash) running continuously as data arrives
- [ ] A.10.6 Build data collection health dashboard showing ingest rates per domain, quality distribution, storage usage, and coverage gaps
- [ ] A.10.7 Build automated data quality scoring for all collected training data (multi-dimensional: completeness, consistency, diversity, toxicity, usefulness)
- [ ] A.10.8 Build data collection audit trail for compliance (what data was collected, when, from whom, for what purpose, retention policy applied)
A.11 Maya Game Engine — Combat, NPC, Economy, World Data#
The Maya domain is the single richest training data source in Oshun. Every gameplay tick generates behavioral, economic, and social telemetry.
- [ ] A.11.1 Build combat telemetry exporter from Maya combat system (DamageCalculated events: sourceKind, damageType, outcome, rawDamage, mitigatedDamage, armor/resistance reduction per encounter → combat balance training data)
- [ ] A.11.2 Build encounter analysis data collector from Maya combat (EncounterAnalyzed: per-combatant DPS, damage taken, healing, defeats + team-level dominantTactic + encounter difficulty tier + adaptiveBias → difficulty calibration training data)
- [ ] A.11.3 Build NPC combat AI decision logger from Maya combat AI (DecisionMade events: combatantId, behavior archetype, action, score, confidence, target, reasons[] trace → combat AI behavior prediction training data)
- [ ] A.11.4 Build combat balance feedback loop (weapon/armor/ability effectiveness metrics across all encounters → balance tuning datasets, mapping item stats to win rates)
- [ ] A.11.5 Build player skill inference data from Maya combat (damage patterns, decision frequency, reaction time, combo execution rate → player skill tier classification training data)
- [ ] A.11.6 Build NPC behavioral state training data from Maya NPC AI State (alert level progression, behavior state machine transitions: idle→patrol→chase→attack/flee, distance-based LOD decisions → NPC behavior prediction model training data)
- [ ] A.11.7 Build NPC memory and relationship data from Maya Souls system (episodic memories + emotional memories + relationship values per player -100 to +100 + reputation gossip + faction loyalty → social dynamics model training data)
- [ ] A.11.8 Build NPC emotional state training data from Maya Souls emotional systems (EmotionValence, Arousal, Dominance, MoodKind transitions, StressAcute/Chronic, TraumaHypervigilance, mood anchor persistence → emotion model training data)
- [ ] A.11.9 Build NPC dialogue quality data from Maya Souls conversation systems (conversation steering decisions, branching choices, dialogue history summaries, personality consistency checks → character dialogue fine-tuning data)
- [ ] A.11.10 Build economy transaction data from Maya economy system (transaction type, currency amount, balance_after, item rarity multipliers → economic modeling and fraud detection training data)
- [ ] A.11.11 Build quest progression data from Maya quest system (quest start/complete/abandon patterns, objective progress rates, XP curves, reward preferences → content recommendation and difficulty calibration training data)
- [ ] A.11.12 Build player analytics data from Maya analytics service (PlayerLogin/Logout, LevelUp, ItemPurchased, QuestStarted/Completed/Abandoned, PlayerKilled/Died, ZoneEntered, AchievementUnlocked, SessionStarted/Ended → retention prediction, churn detection, engagement forecasting)
- [ ] A.11.13 Build procedural world generation feedback data from Maya genesis systems (terrain biome parameters + flora distribution + urban layout results + player exploration patterns → generation quality optimization training data)
- [ ] A.11.14 Build multiplayer social graph data from Maya Nexus (party formation, guild membership, follow patterns, friend networks, block/mute actions → social recommendation and community health training data)
- [ ] A.11.15 Build matchmaking quality data from Maya multiplayer (player stats + team composition + match outcome + player satisfaction → matchmaking optimization training data)
A.12 Arete — Personal Development & Life Mastery#
- [ ] A.12.1 Build habit tracking data pipeline from Arete (habit completion streaks, streak analytics per habit, time-of-day patterns → habit formation prediction model training data)
- [ ] A.12.2 Build goal achievement data from Arete seven habits system (mission statements, goal alignment scores, emotional bank account deposits/withdrawals → goal attainment prediction training data)
- [ ] A.12.3 Build time management effectiveness data from Arete (time tracking, Q2 activity ratios, delegation reviews, weekly compass progress → productivity optimization training data)
- [ ] A.12.4 Build AI coaching quality data from Arete (coaching conversation transcripts + user progress outcomes → coaching response optimization training data)
A.13 Hestia — Culinary Intelligence#
- [ ] A.13.1 Build recipe interaction data from Hestia (recipe views, saves, completions, ratings, cooking session step timing, substitutions made → recipe recommendation and difficulty estimation training data)
- [ ] A.13.2 Build nutrition tracking data from Hestia (meal entries, daily nutrient tracking, goal progress, deficiency alerts, supplement recommendations → nutritional optimization model training data)
- [ ] A.13.3 Build allergen management data from Hestia (allergen detections, substitution suggestions used/rejected, dietary compliance → allergen prediction and substitution model training data)
- [ ] A.13.4 Build kitchen workflow data from Hestia professional (kitchen tickets, prep timing, station assignments, service flow analysis, bottleneck reports → kitchen operations optimization training data)
A.14 Demeter — Home Gardening#
- [ ] A.14.1 Build garden yield data from Demeter (harvest records, yield factors, growth tracking, germination rates, transplant success rates → yield prediction model training data)
- [ ] A.14.2 Build environmental sensor data from Demeter IoT (soil moisture, temperature, light timeseries data → environmental condition prediction training data)
- [ ] A.14.3 Build pest/disease incident data from Demeter (identification queries, observation journal entries with photos, health scores → pest/disease early warning model training data)
- [ ] A.14.4 Build sustainability metrics data from Demeter (carbon sequestration, water footprint, food miles, biodiversity metrics → sustainability impact prediction training data)
A.15 Lakshmi — Personal Finance Intelligence#
- [ ] A.15.1 Build transaction categorization data from Lakshmi (transactions, auto-categorization decisions, user corrections → categorization model improvement training data)
- [ ] A.15.2 Build investment decision data from Lakshmi (portfolio rebalancing actions, tax-loss harvesting events, Monte Carlo parameters → investment recommendation training data)
- [ ] A.15.3 Build behavioral finance data from Lakshmi (nudge response rates, budget adherence patterns, spending pattern → behavioral prediction model training data)
- [ ] A.15.4 Build financial goal tracking data from Lakshmi (goal definitions, progress tracking, debt payoff sequences → goal attainment prediction training data)
A.16 Mnemosyne — Humanistic Learning & Culture#
- [ ] A.16.1 Build spaced repetition data from Mnemosyne (review performance per knowledge item, interval adjustments, mastery levels → spaced repetition interval optimization training data)
- [ ] A.16.2 Build writing quality data from Mnemosyne (grammar checks, style suggestions, readability scores, peer reviews, rubric scores → writing quality model training data)
- [ ] A.16.3 Build assessment calibration data from Mnemosyne (item bank statistics, self-assessment accuracy, certification outcomes → assessment difficulty calibration training data)
- [ ] A.16.4 Build language learning data from Mnemosyne (morphological parsing results, vocabulary acquisition rates, CEFR/ILR level assessments → language proficiency model training data)
A.17 Tara & Meditation — Wellness Platforms#
- [ ] A.17.1 Build meditation session data pipeline from Tara/Meditation (session duration, completion rates, interruptions, pause patterns, guided meditation selection → session recommendation model training data)
- [ ] A.17.2 Build breathing exercise data from Meditation breathing system (breathing pattern telemetry, breathing exercise completion rates → breathing technique effectiveness training data)
- [ ] A.17.3 Build user engagement experiment data from Tara experiments (A/B experiment assignments, user context, experiment outcomes → feature effectiveness and personalization training data)
- [ ] A.17.4 Build content performance data from Tara metrics (content dropoff points, engagement duration, device-specific patterns → content optimization training data)
A.18 Nike & Shakti — Physical Wellness & Movement#
- [ ] A.18.1 Build exercise form analysis data from Nike (skeletal tracking: 65+ joint positions, rep counts, form correction events → exercise form quality model training data)
- [ ] A.18.2 Build workout progression data from Nike (workout program progression, fatigue indicators, completion rates → personalized program generation training data)
- [ ] A.18.3 Build yoga/martial arts technique data from Shakti (pose tracking, technique chains, form analysis scores → movement quality scoring model training data)
- [ ] A.18.4 Build movement pattern capture data from Shakti fitness video analysis (key moment detection, form comparison, coaching reviews → technique assessment model training data)
A.19 Aglaea — Fashion, Beauty & Personal Style#
- [ ] A.19.1 Build skin analysis data from Aglaea (150+ biomarker measurements, product recommendations, user satisfaction feedback → skincare recommendation model training data)
- [ ] A.19.2 Build fabric identification data from Aglaea (fiber characteristics, weave data, blend analysis results → fabric classification model training data)
- [ ] A.19.3 Build style preference data from Aglaea memory system (color preferences, brand affinities, style patterns, body type classifications → personal style recommendation training data)
- [ ] A.19.4 Build ingredient safety data from Aglaea scanner (ingredient risks, safety alerts, sensitivity profiles → ingredient safety prediction model training data)
A.20 Aja — Animation & Content Moderation#
- [ ] A.20.1 Build pose estimation ensemble data from Aja (outputs from 6+ backends: MediaPipe, MoveNet, OpenPose, ViTPose, YOLO-Pose, AlphaPose → ensemble model improvement training data)
- [ ] A.20.2 Build content moderation decision data from Aja (NSFW detection, violence detection, copyright detection, policy violations, appeal outcomes → content classifier improvement training data)
- [ ] A.20.3 Build animation blending quality data from Aja (blend nodes, transitions, body masks, blended frame quality → animation quality model training data)
- [ ] A.20.4 Build privacy protection effectiveness data from Aja (face/person detection accuracy, anonymization quality → privacy model improvement training data)
A.21 Calliope — AI Artist Creation & Management#
- [ ] A.21.1 Build artist persona effectiveness data from Calliope (psychological profiles, artistic philosophy, generated output quality, fan engagement metrics → persona optimization training data)
- [ ] A.21.2 Build creative consistency data from Calliope (symbol registry appearances, consistency issues detected, cultural authenticity validations → artistic consistency model training data)
- [ ] A.21.3 Build collaboration quality data from Calliope (collaboration agent results, co-creation session outcomes → collaboration effectiveness model training data)
A.22 Maat — DAO & Governance Intelligence#
- [ ] A.22.1 Build strategic analysis outcome data from Maat (Porter's/SWOT/BCG analysis parameters + downstream outcomes → strategic model calibration training data)
- [ ] A.22.2 Build financial modeling calibration data from Maat (DCF/IRR/NPV parameters + actual performance → financial model accuracy training data)
- [ ] A.22.3 Build multi-agent coordination data from Maat (agent decisions, coordination logs, simulation outcomes → governance agent optimization training data)
A.23 Industrial & Agriculture Domains (Asase, Cybele, Brigid, Freya, Saraswati)#
- [ ] A.23.1 Build crop intelligence data from Asase (seed performance, fertilizer outcomes, cold chain temperature logs, post-harvest loss measurements → agricultural prediction model training data)
- [ ] A.23.2 Build construction progress data from Cybele (CPM schedule adherence, earned value metrics, quality inspection results, safety incidents → construction prediction model training data)
- [ ] A.23.3 Build predictive maintenance data from Brigid (factory sensor streams, equipment telemetry, failure events, maintenance timing → predictive maintenance model training data)
- [ ] A.23.4 Build manufacturing quality data from Freya (OEE metrics, production costs, waste tracking, kaizen event outcomes → manufacturing optimization model training data)
- [ ] A.23.5 Build EV/drone/pharma telemetry from Saraswati (powertrain data, battery metrics, drone flight logs, pharmaceutical batch records → domain-specific prediction model training data)
A.24 Specialized Domains (Nyx, Uzume, Athena, Annapurna, Oya, Airmid, Seshat)#
- [ ] A.24.1 Build astronomical engagement data from Nyx (exploration paths, celestial object searches, quiz performance, event viewing sessions → content recommendation and educational effectiveness training data)
- [ ] A.24.2 Build performance production data from Uzume (lighting sequences, audio routing configs, quick change timing, costume effectiveness → production optimization training data)
- [ ] A.24.3 Build maker/workshop data from Athena (CAD iterations, CNC toolpath results, acoustic simulation parameters, quality measurements → manufacturing optimization training data)
- [ ] A.24.4 Build restaurant operations data from Annapurna (order patterns, robotic cooking telemetry, delivery routes, food waste metrics → demand forecasting and operations optimization training data)
- [ ] A.24.5 Build drone telemetry data from Oya (flight paths, human detection events, crowd density estimation, obstacle avoidance → autonomous flight model training data)
- [ ] A.24.6 Build botanical intelligence data from Airmid (plant identification results, drug-herb interaction queries, formulation outcomes, adverse event reports → botanical prediction model training data)
- [ ] A.24.7 Build spatial design data from Seshat (Feng Shui/Vastu analysis results, biophilic assessments, sustainability LCA data → design recommendation model training data)
A.25 Blockchain & Prediction Markets (Aje, Themis)#
- [ ] A.25.1 Build prediction market accuracy data from Aje (prediction records, Brier scores, calibration curves, forecaster rankings → forecasting model training data)
- [ ] A.25.2 Build trading pattern data from Aje (trade records, price history OHLCV, market regime identification → market prediction model training data)
- [ ] A.25.3 Build governance decision data from Themis (voting records across governance models, deliberation quality, governance health metrics → governance effectiveness model training data)
- [ ] A.25.4 Build originality detection data from Themis (copyright scan results, plagiarism detection outcomes, IP audit trails → content originality model training data)
A.26 Cross-Cutting Infrastructure Data Collection#
- [ ] A.26.1 Build structured log mining pipeline from shared logging (Pino JSON logs: method, url, statusCode, duration, contentLength per request → latency prediction, error detection, anomaly detection training data)
- [ ] A.26.2 Build Prometheus metrics export for ML from shared metrics (counters, histograms, gauges at 10-30s intervals → time-series forecasting, capacity planning, SLA prediction training data)
- [ ] A.26.3 Build OpenTelemetry trace analysis from shared tracing (distributed span hierarchies, cross-service latencies, critical path data → performance bottleneck detection model training data)
- [ ] A.26.4 Build job queue outcome data from shared queue/BullMQ (job name, status, priority, attempts, duration, error messages → job success prediction, resource forecasting training data)
- [ ] A.26.5 Build event bus pattern data from shared event-bus (Redis Streams: cross-domain event sequences with correlationIds → event causality analysis, user journey modeling training data)
- [ ] A.26.6 Build cache effectiveness data from shared cache (Redis hit/miss patterns by key prefix → cache invalidation prediction, working set analysis training data)
- [ ] A.26.7 Build database query performance data from shared database (query execution times, connection pool statistics → query optimization model training data)
- [ ] A.26.8 Build user behavior funnel data from Lilith analytics service (event_name, event_category, user_id, session_id, properties → user segmentation, churn prediction, feature adoption training data)
A.27 Yemaya Creative Studio & Bellona Build Engine#
- [ ] A.27.1 Build project activity pattern data from Yemaya (all user actions with before/after state changes, collaboration patterns → project success prediction, team effectiveness model training data)
- [ ] A.27.2 Build content safety check data from Yemaya (safety check results, content flags, age ratings, consent records, bias reports → content safety model training data)
- [ ] A.27.3 Build canon enforcement data from Yemaya (style consistency checks, character consistency validations → creative consistency model training data)
- [ ] A.27.4 Build asset pipeline data from Bellona (build queue timing, export success/failure rates, conversion quality metrics, error categorization → build prediction and quality gate training data)
- [ ] A.27.5 Build media processing data from Bellona (media pool operations, DaVinci render jobs, audio routing decisions, proxy generation metrics → media processing optimization training data)
A.28 Nisaba — Ancient Text & Scholarly Research#
- [ ] A.28.1 Build manuscript collation data from Nisaba (witness comparisons, variant units, readings, collation statistics → textual criticism model training data)
- [ ] A.28.2 Build lexicon and morphological data from Nisaba (morphological parsing results, dictionary lookups, cognate discoveries → ancient language NLP model training data)
- [ ] A.28.3 Build cross-tradition discovery data from Nisaba (parallel text identifications, influence graph data, attestation tracking → scholarly connection model training data)
Phase A Summary#
| Section | Domain(s) | Tasks | Primary ML Signal |
|---|---|---|---|
| A.1 | Iris | 10 | Conversation quality, preferences, RAG feedback |
| A.2 | Psyche | 7 | Emotion, expression, voice, behavior |
| A.3 | Isis | 8 | Generation quality, user selections |
| A.4 | Sophia | 6 | Search relevance, citation quality, RAG faithfulness |
| A.5 | Kuan Yin | 5 | Moderation decisions, appeals, emotional states |
| A.6 | Hathor | 5 | NPC dialogue, emotions, world simulation |
| A.7 | Veritas/Yaa | 5 | NER, fact-checking, bias detection |
| A.8 | Euterpe/Aphrodite | 4 | Music preferences, content recommendations |
| A.9 | Minerva/Metis | 4 | Lecture quality, assessment effectiveness |
| A.10 | Cross-domain infra | 8 | Data collection infrastructure |
| A.11 | Maya | 15 | Combat, NPC behavior, economy, social, world gen |
| A.12 | Arete | 4 | Habit formation, goal achievement |
| A.13 | Hestia | 4 | Recipe interactions, nutrition, kitchen ops |
| A.14 | Demeter | 4 | Garden yield, sensors, pests |
| A.15 | Lakshmi | 4 | Transactions, investments, behavioral finance |
| A.16 | Mnemosyne | 4 | Spaced repetition, writing, assessment |
| A.17 | Tara/Meditation | 4 | Session quality, experiments, content |
| A.18 | Nike/Shakti | 4 | Exercise form, movement analysis |
| A.19 | Aglaea | 4 | Skin analysis, fabric ID, style preferences |
| A.20 | Aja | 4 | Pose estimation, moderation, animation |
| A.21 | Calliope | 3 | Artist persona, consistency, collaboration |
| A.22 | Maat | 3 | Strategy, financial models, governance |
| A.23 | Industrial (5 domains) | 5 | Agriculture, construction, manufacturing, IoT |
| A.24 | Specialized (7 domains) | 7 | Astronomy, performance, drones, botanical |
| A.25 | Blockchain (2 domains) | 4 | Prediction markets, governance, originality |
| A.26 | Infrastructure | 8 | Logs, metrics, traces, queues, events |
| A.27 | Yemaya/Bellona | 5 | Project activity, build pipeline |
| A.28 | Nisaba | 3 | Manuscript collation, lexicon, cross-tradition |
| TOTAL | 45 domains | 155 |
Phase B: Centralized Dataset Management & Synthetic Data Generation#
B.1 Dataset Registry & Versioning#
- [ ] B.1.1 Design dataset registry schema (DatasetId, name, version, domain, modality, format, sample count, size bytes, license, quality score, created by, lineage parent, schema version)
- [ ] B.1.2 Implement dataset registry PostgreSQL storage with full CRUD operations and version history
- [ ] B.1.3 Build content-addressable storage backend for dataset artifacts (SHA-256 hashed chunks in S3/MinIO, delta-only storage between versions)
- [ ] B.1.4 Create dataset CLI tool (oshun-dataset create/push/pull/list/diff/merge/split/validate/export)
- [ ] B.1.5 Build dataset format converters: JSONL ↔ Parquet ↔ HuggingFace Datasets ↔ Alpaca ↔ ShareGPT ↔ OpenAI Chat ↔ CSV ↔ Arrow
- [ ] B.1.6 Implement dataset split management (train/val/test splits with configurable ratios, stratified splitting, contamination-preventing held-out set isolation)
- [ ] B.1.7 Build dataset card auto-generator (HuggingFace format: description, intended use, limitations, biases, training data sources, license, citation)
- [ ] B.1.8 Implement dataset diff tool (compare versions: added/removed/modified samples, distribution shift metrics, schema changes)
- [ ] B.1.9 Build dataset search and discovery (full-text search on descriptions + filter by domain/modality/license/quality + similarity search by task type)
- [ ] B.1.10 Implement dataset access control (RBAC: who can read/write/delete datasets, per-domain permissions, audit logging)
- [ ] B.1.11 Build dataset statistics dashboard (sample count over time, quality distribution, format breakdown, per-domain coverage, storage usage)
- [ ] B.1.12 Implement dataset garbage collection (auto-expire staging datasets after configurable TTL, archive old versions to Glacier)
- [ ] B.1.13 Build dataset export pipeline for external sharing (anonymization, license bundling, format conversion, validation)
- [ ] B.1.14 Implement dataset import from HuggingFace Hub (one-command import with automatic format detection and quality scoring)
- [ ] B.1.15 Build data mixing recipe system (specify domain ratios, curriculum schedules, upsampling/downsampling rules → produce training-ready mixed dataset)
B.2 Data Quality & Curation#
- [ ] B.2.1 Build multi-dimensional quality scorer (completeness, consistency, length distribution, diversity, toxicity, information density — scores 0-1 per dimension)
- [ ] B.2.2 Implement exact and near-duplicate detection pipeline (MinHash for text, perceptual hashing for images, chromaprint for audio — running on ingest and on-demand)
- [ ] B.2.3 Build data contamination detector (check if evaluation benchmark samples appear in training data — MMLU, HumanEval, etc.)
- [ ] B.2.4 Implement PII detection and scrubbing pipeline (Presidio for named entities, regex for structured PII, GLiNER for context-dependent PII — configurable per domain)
- [ ] B.2.5 Build license compliance checker (verify training data licenses, flag incompatible combinations, generate license summary for model cards)
- [ ] B.2.6 Implement toxicity and safety filtering (Llama Guard 4 or Granite Guardian for content classification, configurable thresholds per domain)
- [ ] B.2.7 Build data diversity analyzer (topic distribution, language distribution, source distribution, difficulty distribution — flag under-represented areas)
- [ ] B.2.8 Implement outlier detection and removal (statistical outliers by length, quality, embedding distance — configurable sensitivity)
- [ ] B.2.9 Build automated data quality reports (weekly summaries per domain: new data collected, quality trends, coverage gaps, flagged issues)
- [ ] B.2.10 Implement data lineage tracking (which source data → which processing steps → which training runs → which models, full DAG visualization)
B.3 Synthetic Data Generation#
- [ ] B.3.1 Build Magpie-style self-instruct pipeline (prompt aligned LLM with pre-query template only → generate diverse instruction-response pairs → quality filter → dataset, covering broader topic distribution than Alpaca/Evol-Instruct/UltraChat combined per ICLR 2025)
- [ ] B.3.2 Build Persona Hub pipeline (curate 1B+ diverse personas from web data → use each persona as seed for domain-specific instruction generation → personas as distributed carriers of world knowledge per Tencent 2024)
- [ ] B.3.3 Build GLAN-style taxonomy-driven generation (traverse human knowledge taxonomy: subject → discipline → field → generate comprehensive instruction coverage without seed tasks)
- [ ] B.3.4 Build Evol-Instruct pipeline (take seed instructions → evolve complexity through deepening, widening, complication, reasoning steps → produce difficulty-graded instruction data)
- [ ] B.3.5 Build synthetic preference pair generator (for each prompt: generate N responses with different models/temperatures → rank by quality → produce chosen/rejected pairs for DPO/SimPO/KTO)
- [ ] B.3.6 Build CoT-Self-Instruct pipeline (chain-of-thought reasoning to create high-quality synthetic prompts → self-filter for quality → both verifiable reasoning data and non-verifiable instruction data)
- [ ] B.3.7 Build Nemotron-style reward data generator (use multidimensional reward model to rate synthetic responses across helpfulness/harmlessness/honesty → produce reward model training data with 98%+ synthetic coverage)
- [ ] B.3.8 Build comparative knowledge distillation (CKD) pipeline (generate N responses per prompt → create N^2 pairwise comparisons → richer training signal without additional API cost)
- [ ] B.3.9 Build synthetic NER data generator for Ghanaian entities (generate Twi/Ewe/Ga/Hausa text with entity annotations using LLM + Ghana NLP knowledge base)
- [ ] B.3.10 Build synthetic code generation pipeline (generate programming problems → solutions → explanations → test cases for code model training)
- [ ] B.3.11 Build synthetic dialogue generator for NPC training (generate character-consistent multi-turn dialogues with personality constraints from Hathor personality models)
- [ ] B.3.12 Build diffusion-based synthetic image dataset generator (generate training images with controlled variation: style, composition, lighting, subject → diversity-controlled via class-conditional generation)
- [ ] B.3.13 Build TTS synthetic audio data pipeline (generate multi-speaker, multi-language audio training data using XTTS/Voxtral → with phoneme alignment for lip-sync training)
- [ ] B.3.14 Build adversarial/red-team data generator (automatically generate jailbreak attempts, prompt injections, harmful instructions per category taxonomy → red-team eval sets and safety training data)
- [ ] B.3.15 Build multi-lingual synthetic data pipeline for low-resource Ghanaian languages (Twi, Ewe, Ga, Hausa, Dagbani) using cross-lingual transfer + back-translation + Persona Hub)
- [ ] B.3.16 Build quality filter for synthetic data (decontamination check, deduplication, multi-dimensional quality scoring, reward model filtering → retain only top-k% samples)
- [ ] B.3.17 Build TreeSynth-style diversity-controlled generation (tree-structured subspace partitioning for systematic coverage of topic/difficulty/style space)
- [ ] B.3.18 Build Infinity-Instruct-style attribute-guided generation at scale (combine selection from existing data + synthesis of new data with attribute steering)
B.4 Human-in-the-Loop Annotation#
- [ ] B.4.1 Deploy self-hosted Label Studio instance for annotation backend (multi-type labeling with standardized output)
- [ ] B.4.2 Integrate Argilla for AI-assisted annotation (AI feedback suggestions, semantic search, quality filtering integration)
- [ ] B.4.3 Build preference comparison UI (side-by-side A/B interface: show two model responses, annotator picks preferred → DPO training data)
- [ ] B.4.4 Build thumbs-up/down feedback ingestion from all production endpoints (Iris, Psyche, Isis → standardized feedback events → preference data)
- [ ] B.4.5 Build annotation task queue with priority, assignment, and deadline management
- [ ] B.4.6 Implement inter-annotator agreement calculator (Cohen's kappa for 2 annotators, Fleiss' kappa for 3+, Krippendorff's alpha for mixed types)
- [ ] B.4.7 Build gold standard quality checks (inject known-answer questions at configurable rate, flag annotators below accuracy threshold)
- [ ] B.4.8 Create annotator calibration workflow (training phase with examples + certification test before production annotation)
- [ ] B.4.9 Build active learning loop (use model uncertainty to route most informative samples to annotators → maximize annotation budget efficiency)
- [ ] B.4.10 Implement RLHF data pipeline (feedback events → preference pair formatting → deduplication → quality filtering → training-ready DPO/KTO/SimPO datasets)
- [ ] B.4.11 Build red team dataset curation tool (organize adversarial prompts by harm category from Kuan Yin taxonomy, track coverage per category, prioritize under-covered areas)
- [ ] B.4.12 Build evaluation set management (maintain held-out eval sets per domain with contamination monitoring — alert if any eval sample appears in any training dataset)
- [ ] B.4.13 Build annotation analytics dashboard (throughput per annotator, quality scores, cost per annotation, task completion rates, agreement trends)
- [ ] B.4.14 Implement multi-annotator consensus resolution workflow (majority voting, adjudication for disagreements, escalation path)
- [ ] B.4.15 Build stepwise process reward annotation tool (annotators label each reasoning step as correct/incorrect/unnecessary → PRM training data per ThinkPRM approach)
Phase C: Production Model Serving (SOTA April 2026)#
C.1 LLM Serving Engines#
- [ ] C.1.1 Integrate vLLM v0.7+ as primary LLM serving engine (PagedAttention, continuous batching, optimized for Blackwell B200 — consistently fastest TTFT across concurrency levels)
- [ ] C.1.2 Build vLLM Docker image for RunPod deployment with multi-LoRA support (serve N fine-tuned adapters on single base model using S-LoRA/LoRAX pattern — 4x throughput scaling)
- [ ] C.1.3 Integrate SGLang v0.4+ for structured generation workloads (RadixAttention prefix caching, constrained JSON/code output, zero-overhead batch scheduler, cache-aware load balancer)
- [ ] C.1.4 Integrate TensorRT-LLM v1.2+ for NVIDIA-optimized inference (speculative decoding up to 3.6x, 1000 tok/s per user on Blackwell, 14x TTFT reduction on H100)
- [ ] C.1.5 Integrate llama.cpp with latest GGUF advances (imatrix quantization, AWQ-to-GGUF pipeline, lazy conversion, sharding via llama-gguf-split, speculative decoding, multimodal via libmtmd)
- [ ] C.1.6 Build model serving router that selects optimal backend per request (vLLM for general chat, SGLang for structured output, TensorRT-LLM for throughput-critical, llama.cpp for edge)
- [ ] C.1.7 Implement Ollama-compatible API endpoint for developer experience (local development parity with production serving)
- [ ] C.1.8 Build multi-tenant model isolation (per-customer LoRA adapters served via LoRAX pattern with adapter hot-loading and memory-efficient sharing)
C.2 Inference Optimization#
- [ ] C.2.1 Implement speculative decoding pipeline with EAGLE-3 (fuse hidden states from multiple layers → 2-6x speedup depending on model/batch)
- [ ] C.2.2 Build KV cache compression using Expected Attention and ClusterAttn (predict future attention patterns → compress KV cache → lower memory without quality loss)
- [ ] C.2.3 Implement prefix caching with LMCache integration (up to 15x throughput, 2x lower latency for shared-prefix workloads like RAG and chatbots)
- [ ] C.2.4 Build EXL3 quantization pipeline (ExLlamaV3 trellis-based encoding: 140GB model → <40GB VRAM, 120 tok/s on 24GB GPU, targeting 200+ tok/s on RTX 50-series)
- [ ] C.2.5 Implement quantization format conversion pipeline (HuggingFace → GGUF imatrix, AWQ, GPTQ, EXL3, MXFP4 — automated with quality validation per format)
- [ ] C.2.6 Build Flash Attention 3 integration (async + low-precision + paged KV + incoherent FP8 processing)
- [ ] C.2.7 Implement dynamic batching optimizer (group requests by sequence length, maximize GPU utilization per batch, token-level scheduling)
- [ ] C.2.8 Build model warmup and preloading system (pre-populate KV cache for system prompts, keep hot models in GPU memory, intelligent eviction)
- [ ] C.2.9 Implement graceful model swapping (zero-downtime: load new model version in background → swap atomically → unload old version)
- [ ] C.2.10 Build inference cost calculator per endpoint ($/token by model × backend × GPU × quantization level — real-time dashboard)
- [ ] C.2.11 Implement request priority queuing (interactive < 500ms target > batch < 5s target > background — with SLA monitoring)
- [ ] C.2.12 Build llm-d style Kubernetes-native KV cache-aware request routing (87% cache hit rate, 88% faster TTFT for warm cache hits)
C.3 Model Format Management#
- [ ] C.3.1 Build automated model conversion pipeline (HuggingFace SafeTensors → GGUF + AWQ + GPTQ + EXL3 + ONNX + TensorRT → quality validation for each format)
- [ ] C.3.2 Implement model format registry (track which formats exist for each model, quality metrics per format, storage locations)
- [ ] C.3.3 Build MXFP4 native quantization support (following GPT-OSS pattern for maximum efficiency on modern GPUs)
- [ ] C.3.4 Implement automated imatrix calibration for GGUF quantization (importance-matrix generation from representative prompts → higher quality quants)
- [ ] C.3.5 Build model sharding manager for large models (automatic split across multiple GPUs with optimal shard placement)
Phase D: Training & Fine-Tuning Infrastructure (SOTA April 2026)#
D.1 Training Frameworks Integration#
- [ ] D.1.1 Integrate TRL v1.0 as reference post-training library (75+ methods: DPO, SimPO, KTO, ORPO, CPO, GRPO, DAPO, RLOO, REINFORCE++, GKD, SDFT — stable API)
- [ ] D.1.2 Integrate Unsloth for single-GPU training (12x faster MoE, 7x longer context RL, 70% less VRAM, 150+ model support including Llama 4, Qwen3, DeepSeek V3, GPT-OSS)
- [ ] D.1.3 Integrate Axolotl v0.8+ for production-scale training (multimodal fine-tuning, QAT, sequence parallelism, GRPO, reward modeling, YAML-driven config)
- [ ] D.1.4 Integrate torchtune for PyTorch-native training (DoRA support, QAT, PPO, PyTorch 2.5 compile for 20-24% speedup)
- [ ] D.1.5 Integrate LLaMA-Factory v0.9+ for accessible training (LlamaBoard web UI, OFT support, Megatron-LM integration via MCoreAdapter)
- [ ] D.1.6 Build unified training launcher CLI (oshun-train --framework unsloth/axolotl/torchtune/trl --method lora/dpo/grpo --model llama4-scout --dataset <id> → manages the full lifecycle)
D.2 Advanced PEFT Methods (Beyond LoRA)#
- [ ] D.2.1 Implement DoRA training support (weight decomposition into magnitude + direction, NVIDIA ICML 2024 Oral — same as LoRA params, no inference overhead, better convergence)
- [ ] D.2.2 Implement VeRA training support (shared frozen random matrices across all layers + learnable scaling vectors — dramatically fewer params than LoRA)
- [ ] D.2.3 Implement Spectrum layer selection (SNR-based selection of highest signal-to-noise layers for fine-tuning — fewer layers updated, compute-efficient)
- [ ] D.2.4 Implement QAT (Quantization-Aware Training) pipeline via Axolotl (5x perplexity improvement vs post-training quantization — critical for edge deployment)
- [ ] D.2.5 Implement AdaLoRA with adaptive rank allocation per layer (SVD-based weight updates, automatic rank selection for optimal parameter budget)
- [ ] D.2.6 Implement multi-LoRA composition training (LoRA Soups: concatenation + optimal weighting, per COLING 2025 — 43% improvement over merge baselines)
D.3 RLVR & Advanced Alignment Training#
- [ ] D.3.1 Implement GRPO training pipeline (Group Relative Policy Optimization: sample 8-64 responses per prompt → compute group advantages → policy gradient without critic model — DeepSeek-R1's breakthrough method)
- [ ] D.3.2 Implement DAPO training pipeline (Dynamic Advantage PO: Clip-Higher + Dynamic Sampling + Token-level PG Loss + Overlong Reward Shaping — 50% fewer training steps than GRPO for equivalent quality)
- [ ] D.3.3 Implement RLVR infrastructure (verifiable reward functions: math correctness checker, code execution sandbox, proof verifier, format compliance validator — replacing human preference labels for reasoning)
- [ ] D.3.4 Implement SimPO training (reference-free preference optimization using average log probability as implicit reward — +6.4 AlpacaEval 2, +7.5 Arena-Hard vs DPO)
- [ ] D.3.5 Implement Process Reward Model (PRM) training (stepwise reward at every reasoning step, not just final answer — ThinkPRM approach with verbalized verification chains)
- [ ] D.3.6 Implement self-play training pipeline (SPIRAL: zero-sum game self-play → reasoning transfer across benchmarks, +10% on 8 benchmarks per arxiv)
- [ ] D.3.7 Implement online DPO/iterative DPO (generate new responses during training, recompute preferences → train beyond offline dataset quality)
- [ ] D.3.8 Implement RainbowPO training (unified framework combining 7 orthogonal DPO improvements — +28.74% LC Win Rate on AlpacaEval2 from offline data alone)
- [ ] D.3.9 Implement KTO training for binary feedback (Kahneman-Tversky: thumbs-up/down without pairwise comparisons, models human loss aversion — ideal when preference pairs unavailable)
- [ ] D.3.10 Implement multimodal alignment training via MM-DPO (120K fine-grained multimodal preference pairs across 8 dimensions with critique-based reward model)
D.4 Optimizers & Training Efficiency#
- [ ] D.4.1 Integrate Muon optimizer (orthogonalize moving gradient averages via Newton-Schulz iteration — 2x computational efficiency vs AdamW, ~50% cost reduction, 33% memory savings)
- [ ] D.4.2 Integrate GaLore 2 for memory-efficient training (gradient projection to lower-dimensional subspaces — 82.5% optimizer memory reduction, 63.3% total memory reduction, scaled to Llama 7B pre-training on 500B tokens)
- [ ] D.4.3 Integrate Liger Kernel (LinkedIn Triton kernels: fused RMSNorm, RoPE, SwiGLU, CrossEntropy — 20% throughput increase, 60% memory reduction, 80% more efficient post-training losses)
- [ ] D.4.4 Integrate FSDP2 via TorchTitan (per-parameter dim-0 sharding, 7% lower memory, deterministic usage, communication-free state dicts, built-in BF16/FP32 mixed precision)
- [ ] D.4.5 Integrate DeepSpeed ZeRO++ (4x communication reduction, 50%+ end-to-end training time reduction at scale)
- [ ] D.4.6 Implement NeMo Megatron Bridge for bidirectional HuggingFace ↔ Megatron format conversion (enables using any HuggingFace model with Megatron training, and vice versa)
- [ ] D.4.7 Build training memory profiler (predict peak VRAM for given model + batch size + sequence length + PEFT method + optimizer → right-size GPU allocation before launch)
D.5 MoE Training & Scaling#
- [ ] D.5.1 Implement MoE fine-tuning pipeline (Mixture of Experts: fine-tune individual experts or routing, supporting Llama 4 Scout/Maverick 16/128-expert architectures)
- [ ] D.5.2 Implement MoE expert parallelism via NeMo Megatron Core (distribute experts across GPUs with FP8 training for maximum throughput)
- [ ] D.5.3 Build MoErges/FrankenMoE pipeline (upcycle dense fine-tuned models into MoE architectures using mergekit-moe with learned router initialization)
- [ ] D.5.4 Implement selective expert fine-tuning (freeze base experts, fine-tune only domain-relevant experts → cheaper than full MoE fine-tuning)
D.6 Model Merging & Composition#
- [ ] D.6.1 Integrate mergekit for automated model merging (SLERP, TIES, DARE, Task Arithmetic, Passthrough/Frankenmerge, MoErges — yaml-driven configuration)
- [ ] D.6.2 Build automated merge quality evaluation (merge → run benchmark suite → compare against source models → accept/reject merge)
- [ ] D.6.3 Implement multi-LoRA routing system (LoRA-Mixer: coordinate modular LoRA experts through serial attention routing → domain-specific adapters activated per request)
- [ ] D.6.4 Build model merge search (try multiple merge strategies × weight ratios → evaluate each → select Pareto-optimal merge on quality/size tradeoff)
Phase E: Alignment & Safety Training (SOTA April 2026)#
- [ ] E.1 Integrate Llama Guard 4 as primary guardrail model (12B, natively multimodal text+image, MLCommons hazards taxonomy, dense architecture, single GPU)
- [ ] E.2 Integrate Granite Guardian for RAG hallucination detection (0.854 AUC — SOTA for hallucination, 0.871 for risk detection)
- [ ] E.3 Build Constitutional AI training pipeline with Oshun-specific principles (safety, helpfulness, honesty, privacy, fairness — per domain customization)
- [ ] E.4 Build red-team automation pipeline (auto-generate adversarial prompts → test model → classify failures → add to safety training → retrain → repeat)
- [ ] E.5 Implement RLAIF pipeline using Oshun's own models as judges (eliminate dependency on external APIs for preference labeling)
- [ ] E.6 Build jailbreak detection fine-tuning pipeline (collect jailbreak attempts from production → train detection model → deploy as input guardrail)
- [ ] E.7 Build prompt injection detection model (fine-tune classifier on injection attempts collected from Kuan Yin + synthetic generation)
- [ ] E.8 Implement safety benchmark continuous evaluation (TruthfulQA, BBQ, RealToxicityPrompts — run on every model version, alert on regressions)
- [ ] E.9 Build deepfake detection pipeline (integrate Deepfake-o-Meter framework with 18+ detection models across image/video/audio)
- [ ] E.10 Build NSFW content classifier fine-tuning pipeline for Aphrodite domain (domain-specific boundary calibration)
- [ ] E.11 Implement NeMo Guardrails integration (Content Safety NIM + Topic Control NIM + Jailbreak Detection NIM)
- [ ] E.12 Build multi-lingual safety evaluation for Ghanaian languages (safety benchmarks translated/adapted for Twi, Ewe, Ga, Hausa)
Phase F: Continuous Evaluation & Benchmarking#
F.1 LLM Evaluation Suite#
- [ ] F.1.1 Implement MMLU-Pro benchmark runner (multi-task language understanding, harder than MMLU, augmented options)
- [ ] F.1.2 Implement HumanEval / BigCodeBench runner (code generation evaluation with execution)
- [ ] F.1.3 Implement IFEval runner (instruction following with verifiable constraints — automated verification)
- [ ] F.1.4 Implement Arena-Hard-Auto evaluation (500 challenging prompts, LLM-as-judge scoring)
- [ ] F.1.5 Implement GPQA Diamond runner (PhD-level science reasoning, non-experts ~34%)
- [ ] F.1.6 Implement MATH-500 benchmark runner (mathematical reasoning)
- [ ] F.1.7 Implement LiveBench runner (contamination-free, monthly updated prompts across 6 categories)
- [ ] F.1.8 Implement SWE-bench Verified runner (real GitHub issue resolution)
- [ ] F.1.9 Implement WildBench runner (real user prompts from production)
- [ ] F.1.10 Integrate LightEval (HuggingFace lightweight evaluation library — unified evaluation across all benchmarks)
F.2 Domain-Specific Evaluation#
- [ ] F.2.1 Build RAG evaluation framework using RAGAS methodology (faithfulness, answer relevancy, context precision, context recall — per Sophia domain)
- [ ] F.2.2 Build embedding quality benchmark using MTEB subset (retrieval, classification, clustering, reranking — for Sophia and Iris embedding models)
- [ ] F.2.3 Build NER evaluation suite for Ghanaian entities (precision, recall, F1 per entity type — for Veritas domain)
- [ ] F.2.4 Build dialogue consistency evaluation for NPC conversations (personality adherence, memory consistency, emotional coherence — for Hathor domain)
- [ ] F.2.5 Build TTS quality evaluation suite (MOS prediction, speaker similarity, intelligibility — for Psyche and Veritas voice models)
- [ ] F.2.6 Build image generation quality evaluation (FID, CLIP score, aesthetic score, prompt alignment — for Isis domain)
- [ ] F.2.7 Build 3D generation quality evaluation (geometry accuracy, texture quality, PBR correctness — extending Isis 3d-generation-benchmarks)
- [ ] F.2.8 Build music generation evaluation (audio quality, genre accuracy, prompt alignment — for Euterpe domain)
- [ ] F.2.9 Build content moderation precision/recall evaluation (per harm category from Kuan Yin taxonomy)
- [ ] F.2.10 Build cross-lingual evaluation suite for Ghanaian languages (NLU, NLG, translation quality across Twi/Ewe/Ga/Hausa/Dagbani)
F.3 Evaluation Infrastructure#
- [ ] F.3.1 Build unified benchmark runner that executes all registered benchmarks against any model endpoint (scheduled: daily for production models, on-demand for new checkpoints)
- [ ] F.3.2 Build model leaderboard dashboard (per-domain quality tracking over time, cross-model comparison, Pareto front visualization on quality vs cost)
- [ ] F.3.3 Implement automated regression detection (compare new model version against baseline on all benchmarks → alert on statistically significant regressions)
- [ ] F.3.4 Build human evaluation framework (structured rating tasks, random sampling from production, Likert scales per dimension, aggregation with confidence intervals)
- [ ] F.3.5 Implement LLM-as-judge evaluation (use Claude/GPT as automated judges with calibrated prompts, cross-validate with human ratings)
- [ ] F.3.6 Build eval set contamination detector (check all training datasets against all eval sets — hash-based exact match + embedding-based near match)
- [ ] F.3.7 Build evaluation report generator (PDF/HTML per model version: benchmark scores, regressions, comparison charts, recommendations)
- [ ] F.3.8 Implement Oshun-specific benchmark ("Oshun-Bench": curated challenging prompts from all domains, monthly rotation for contamination resistance)
Phase G: MLOps, Experiment Management & Governance#
G.1 Experiment Tracking#
- [ ] G.1.1 Deploy self-hosted MLflow as experiment tracking backend (model registry, experiment tracking, artifact storage, model serving endpoints)
- [ ] G.1.2 Build experiment tracking SDK that wraps MLflow/W&B (log metrics, params, artifacts from any training framework — unified API regardless of backend)
- [ ] G.1.3 Implement model lineage graph (visualize: base model → fine-tune → merge → quantize → deploy chain as interactive DAG)
- [ ] G.1.4 Build model comparison dashboard (side-by-side: metrics, outputs on same prompts, cost, latency, memory — for deployment decisions)
- [ ] G.1.5 Implement experiment tagging and search (tag by domain, method, dataset, model family — full-text search across all experiments)
- [ ] G.1.6 Build training job dashboard (running, queued, completed, failed jobs — with cost, duration, GPU utilization per job)
G.2 Model Lifecycle Management#
- [ ] G.2.1 Build model drift detection service (monitor production model outputs: distribution shift in embeddings, confidence calibration, latency — using sliding window statistics)
- [ ] G.2.2 Implement automated retraining trigger (when drift exceeds threshold: queue retraining job with latest data → evaluate → promote if better)
- [ ] G.2.3 Build model A/B testing framework (split traffic by percentage, collect quality metrics per variant, auto-promote winner after significance threshold)
- [ ] G.2.4 Implement shadow deployment (run new model alongside production: same inputs, compare outputs, no user impact — assess before switching)
- [ ] G.2.5 Build one-click model rollback (revert to any previous model version: update routing → drain in-flight requests → swap → health check → confirm)
- [ ] G.2.6 Implement canary deployment for model updates (route 1% → 5% → 25% → 100% with automatic rollback on quality degradation)
- [ ] G.2.7 Build model deprecation workflow (announce sunset → warn consumers → provide migration path → archive → delete after retention period)
G.3 Model Governance & Compliance#
- [ ] G.3.1 Build model card auto-generator (training data summary, benchmark scores, known limitations, bias audit results, license, intended use — HuggingFace format)
- [ ] G.3.2 Create model approval workflow (model trained → automated eval → human review → staging deployment → production approval — with RBAC)
- [ ] G.3.3 Implement automated bias audit pipeline (test model across protected attributes: gender, race, age, nationality — flag statistically significant disparities)
- [ ] G.3.4 Build audit trail for all model operations (who trained what, when, with what data, what parameters, what results — immutable log)
- [ ] G.3.5 Implement license compliance tracker (base model licenses × training data licenses → compatibility check → flag issues before deployment)
- [ ] G.3.6 Build reproducibility system (pin: data version + code commit + config hash + random seeds → any experiment can be exactly reproduced)
- [ ] G.3.7 Implement model risk classification (tier models by risk: safety-critical, customer-facing, internal-only → different governance requirements per tier)
- [ ] G.3.8 Build compliance reporting dashboard (model inventory, governance status, audit trail summary, outstanding approvals, expired certifications)
Phase H: Edge & On-Device Deployment#
- [ ] H.1 Build ONNX export pipeline (PyTorch/HF models → ONNX with graph optimization, operator fusion, constant folding — automated with quality validation)
- [ ] H.2 Implement CoreML conversion pipeline (ONNX → CoreML for iOS/macOS — targeting Apple MLX with Neural Accelerator support on M5/A19 Pro)
- [ ] H.3 Build TFLite conversion pipeline (ONNX → TensorFlow Lite for Android — with INT8/INT4 quantization, targeting Snapdragon 8 Elite Gen 5 NPU)
- [ ] H.4 Create TensorRT compilation pipeline (ONNX → TensorRT for NVIDIA edge — targeting Jetson/embedded with INT8 calibration)
- [ ] H.5 Implement WebGPU inference runtime for browser deployment (shader-based ML inference using wgpu/WebGPU API — for Oshun web apps)
- [ ] H.6 Integrate MLC-LLM/WebLLM for browser-native LLM inference (cross-platform model compilation, consistently lower TTFT)
- [ ] H.7 Integrate ExecuTorch 1.0 for mobile deployment (50KB base footprint, 12+ hardware backends, 80%+ popular edge LLMs — used by Instagram/WhatsApp/Messenger)
- [ ] H.8 Implement model pruning pipeline (structured + unstructured pruning with iterative fine-tuning to recover accuracy)
- [ ] H.9 Build quantization-aware training (QAT) pipeline (train with quantization in the loop → 5x better perplexity than post-training quantization — via Axolotl)
- [ ] H.10 Implement knowledge distillation automation (teacher → student: select teacher, generate distillation data, train student, evaluate, iterate — end-to-end pipeline)
- [ ] H.11 Build target device profiler (benchmark model on target hardware: latency, memory, battery, thermal — per device class matrix)
- [ ] H.12 Create model size/latency/accuracy Pareto optimizer (given quality floor and latency ceiling → find optimal model configuration)
- [ ] H.13 Implement on-device model update mechanism (differential model updates: download only changed weights, verify integrity, apply atomically)
- [ ] H.14 Build fallback chain system (try on-device model → edge server → cloud endpoint — with quality-aware routing)
- [ ] H.15 Build edge model monitoring (collect on-device metrics → aggregate centrally → detect per-device quality issues)
- [ ] H.16 Implement small language model fine-tuning pipeline (target: Gemma 4 E2B/E4B, Phi-4 mini, Qwen3-4B, SmolLM2 → domain-specific SLMs for on-device)
Phase I: Domain-Specific Training Pipelines#
Each pipeline is end-to-end: data collection → preparation → training → evaluation → deployment → monitoring.
- [ ] I.1 Build Iris conversational model fine-tuning pipeline (flywheel data from A.1 → format for TRL → LoRA/QLoRA on Llama 4 Scout or Qwen3-8B → eval on Oshun-Bench → deploy via vLLM multi-LoRA)
- [ ] I.2 Build Iris embedding model fine-tuning pipeline (search feedback from A.4.1 → triplet mining with hard negatives → contrastive training on BGE-M3 or Qwen3-Embedding-8B → eval on MTEB subset → deploy to embedding service)
- [ ] I.3 Build Iris reranker fine-tuning pipeline (RAG relevance data from A.4.1 → train cross-encoder on Qwen3-Reranker or BGE-reranker → eval on domain retrieval → deploy)
- [ ] I.4 Build Veritas NER model training pipeline (Ghana entity data from A.7.1 + synthetic Ghanaian NER data from B.3.9 → fine-tune spaCy/BERT → eval per entity type → deploy to NLP service)
- [ ] I.5 Build Veritas bias detector training pipeline (bias data from A.7.3 → train DeBERTa classifier → eval on political coverage → deploy)
- [ ] I.6 Build Veritas Ghana language TTS pipeline (Ghana NLP audio + synthetic audio from B.3.13 → fine-tune Voxtral 4B or XTTS v2 for Twi/Ewe/Ga → eval MOS → deploy)
- [ ] I.7 Build Psyche voice cloning pipeline (voice samples → fine-tune XTTS v2 or Dia 1.6B for target voices → quality eval → deploy to voice engine)
- [ ] I.8 Build Psyche emotion recognition fine-tuning pipeline (emotion data from A.2.1 → fine-tune EfficientNet/ConvNeXt classifier → eval per emotion class → deploy to perception engine)
- [ ] I.9 Build Kuan Yin harm detector training pipeline (moderation data from A.5.1 + red-team data from B.3.14 → fine-tune Llama Guard 4 or Granite Guardian → eval per harm category → deploy)
- [ ] I.10 Build Nous safety classifier training pipeline (jailbreak + injection data → fine-tune classifier → red-team eval → deploy as guardrail)
- [ ] I.11 Build Isis image LoRA training pipeline (user-selected outputs from A.3.2 → curate style datasets → train SDXL/Flux LoRA via Unsloth → quality gate → publish to model registry)
- [ ] I.12 Build Isis video LoRA training pipeline (video selection data → train AnimateDiff/Wan 2.2 LoRA → eval → deploy)
- [ ] I.13 Build Sophia document embedding fine-tuning pipeline (search relevance data from A.4 → contrastive training on BGE-M3 → eval retrieval quality → deploy)
- [ ] I.14 Build Oya aerial vision training pipeline (aerial imagery → fine-tune YOLO26 or RF-DETR → quantize for Jetson → deploy to drone)
- [ ] I.15 Build Airmid plant identification training pipeline (plant images → fine-tune ConvNeXt V2 → quantize for mobile → deploy)
- [ ] I.16 Build Euterpe music model training pipeline (music preference data from A.8.1 → fine-tune ACE-Step or MusicGen → eval → deploy)
- [ ] I.17 Build Minerva lecture LLM training pipeline (educational data from A.9 → fine-tune Llama 4 Scout or Qwen3-8B → eval → deploy)
- [ ] I.18 Build Hathor NPC dialogue training pipeline (NPC dialogue data from A.6 → fine-tune small LLM for character consistency → eval personality adherence → deploy)
- [ ] I.19 Build Aphrodite content moderation training pipeline (moderation data from A.8.4 → fine-tune NSFW detector → eval precision/recall → deploy)
- [ ] I.20 Build Themis originality model training pipeline (content fingerprints → train similarity model → eval detection rate → deploy)
- [ ] I.21 Build cross-domain transfer learning pipeline (shared learned representations from high-resource domains → bootstrap low-resource domain models)
- [ ] I.22 Build automated pipeline orchestration (when new training data exceeds threshold → trigger pipeline → evaluate → promote if better → alert on failure)
Phase J: GPU Infrastructure & Training Clusters#
- [ ] J.1 Design multi-cloud GPU orchestration layer (RunPod + AWS + GCP GPUs as unified pool — abstract provider, expose capacity/cost/availability)
- [ ] J.2 Build training cluster provisioner (spin up multi-GPU training clusters on demand: specify GPU type × count × hours → provision → configure NCCL/Gloo → return endpoint)
- [ ] J.3 Implement spot/preemptible instance manager with checkpointing (use 60-90% cheaper spot instances → checkpoint every N steps → auto-resume on preemption)
- [ ] J.4 Build GPU memory profiler (predict peak VRAM for model × batch size × sequence length × method → recommend GPU type before launching)
- [ ] J.5 Build training cost estimator (predict $/run from: model size × dataset size × epochs × GPU type × method → approve/reject before launch)
- [ ] J.6 Implement queue-based training scheduler with priority and fairness (domain priority weights, max concurrent jobs per domain, preemption for urgent jobs)
- [ ] J.7 Create distributed training launcher (oshun-train-distributed --nodes 4 --gpus-per-node 8 --model <id> → configure NCCL, launch, monitor — one command)
- [ ] J.8 Build GPU health monitoring dashboard (temperature, utilization, ECC errors, memory, clock speed per GPU — alert on anomalies)
- [ ] J.9 Implement auto-scaling for inference (monitor request queue depth → scale GPU workers up/down → drain before scale-down)
- [ ] J.10 Build GPU sharing for small models (MIG partitioning, MPS time-sharing for models that don't need full GPU)
- [ ] J.11 Build training cost dashboard (per-domain $/month, per-model $/run, per-experiment breakdown, trending, budget alerts)
- [ ] J.12 Implement GPU utilization alerting (underutilized GPU > 30min → suggest smaller GPU or sharing; overloaded → suggest scaling)
Phase K: SOTA Model Integration (April 2026 Open-Weight Models)#
These tasks integrate the latest open-weight models for self-hosting.
K.1 Foundation LLMs#
- [ ] K.1.1 Build Llama 4 Scout serving pipeline (109B total/17B active, 16 experts, 10M token context — vLLM with MoE expert parallelism, GGUF quantization for smaller GPUs)
- [ ] K.1.2 Build Llama 4 Maverick serving pipeline (400B total/17B active, 128 experts, 1M context — multi-GPU TensorRT-LLM, RunPod A100 cluster)
- [ ] K.1.3 Build Qwen 3.5-397B serving pipeline (397B MoE, native multimodal text+image+video, agentic — vLLM/SGLang with structured output)
- [ ] K.1.4 Build DeepSeek V3.1 serving pipeline (unified thinking/non-thinking modes via chat template — MIT license, vLLM)
- [ ] K.1.5 Build Mistral Large 3 serving pipeline (675B total/41B active, Apache 2.0, 256K context — TensorRT-LLM for throughput)
- [ ] K.1.6 Build Gemma 4 serving pipeline (26B MoE + 31B dense, native video/image, Apache 2.0, 256K context — vLLM)
- [ ] K.1.7 Build GPT-OSS-120B serving pipeline (117B total/5.1B active, Apache 2.0, native MXFP4 — single H100 GPU)
- [ ] K.1.8 Build GLM-5 serving pipeline (744B total/40B active, MIT license, trained on Huawei Ascend — #1 open-weight on BenchLM)
- [ ] K.1.9 Build small LLM serving for edge (Gemma 4 E2B/E4B, Phi-4 mini 3.8B, Qwen3-4B — llama.cpp/ExecuTorch/MLX)
- [ ] K.1.10 Build Qwen3-Coder serving for code tasks (480B/30B variants, 256K context, reinforcement learning on SWE-Bench)
K.2 Embedding & Reranking Models#
- [ ] K.2.1 Deploy Qwen3-Embedding-8B as primary open-source embedding model (70.58 multilingual MTEB — top open-source, Apache 2.0)
- [ ] K.2.2 Deploy Jina Embeddings v4 for multimodal embedding (3.8B, text+image+PDF, single/multi-vector, 3 task-specific LoRA adapters)
- [ ] K.2.3 Deploy Qwen3-Reranker-8B as primary reranker (69.02 multilingual MTEB ranking — SOTA open-source)
- [ ] K.2.4 Deploy Jina ColBERT v2 for late-interaction retrieval (89 languages, 7x GPU speedup via PLAID engine)
- [ ] K.2.5 Deploy BGE-M3 as fallback multilingual embedding (proven, widely tested baseline)
K.3 Vision Models#
- [ ] K.3.1 Deploy Qwen3-VL-235B for vision understanding (rivals Gemini-2.5-Pro, 2D/3D grounding, video, OCR, agentic)
- [ ] K.3.2 Deploy InternVL3.5 as alternative VLM (InternViT + Qwen3 backbone, strong on MMMU)
- [ ] K.3.3 Deploy YOLO26 for object detection (NMS-free, 43% faster, 5 tasks: detection/segmentation/classification/pose/OBB)
- [ ] K.3.4 Deploy RF-DETR for high-accuracy detection (first real-time >60 AP on COCO, DINOv2 backbone)
- [ ] K.3.5 Deploy SAM 2.1 for segmentation (improved small object/occlusion handling, Apache 2.0)
- [ ] K.3.6 Deploy Chandra OCR 2 for document reading (5B, outperforms GPT-4o on most benchmarks)
- [ ] K.3.7 Deploy PaddleOCR-VL 7B for document parsing (92.86 OmniDocBench, outperforms GPT-5.4)
K.4 Image Generation Models#
- [ ] K.4.1 Deploy FLUX.2 [klein] 4B for fast image generation (Apache 2.0, <1s generation at 1024px, 4 inference steps)
- [ ] K.4.2 Deploy FLUX.2 [dev] 32B for high-quality generation (flow matching transformer, multi-reference image support)
- [ ] K.4.3 Deploy Stable Diffusion 3.5 Medium for consumer hardware generation (2.5B, MMDiT-X, 4-step turbo)
- [ ] K.4.4 Integrate ControlNet + IP-Adapter for multi-control generation (22M additional params for image prompting)
K.5 Video Generation Models#
- [ ] K.5.1 Deploy Wan 2.2 for video generation (27B MoE, first open MoE video model, Apache 2.0, text-to-video + image-to-video)
- [ ] K.5.2 Deploy LTX-2.3 for audio-video generation (22B, native 4K, 50FPS, 20s videos, synchronized audio, Apache 2.0)
- [ ] K.5.3 Deploy HunyuanVideo for cinematic video (13B, beats Runway Gen-3)
- [ ] K.5.4 Deploy CogVideoX 1.5 for accessible video generation (5B, 3D VAE, 10s at 16fps)
K.6 Audio & Speech Models#
- [ ] K.6.1 Deploy Voxtral TTS 4B as primary TTS (9 languages, 90ms TTFA, 3s voice cloning, single 16GB GPU, Apache 2.0)
- [ ] K.6.2 Deploy Dia 1.6B for dialogue TTS (ultra-realistic, nonverbal cues: laughs/sighs/screams, Apache 2.0)
- [ ] K.6.3 Deploy Orpheus TTS 3B for emotional TTS (human-like emotion, Llama-based)
- [ ] K.6.4 Deploy Kokoro 82M as lightweight TTS (54 voices, 8 languages, <1GB FP16, RTF ~0.03, Apache 2.0)
- [ ] K.6.5 Deploy Whisper Large V3 Turbo for STT (5.4x faster than V3, close to V2 accuracy)
- [ ] K.6.6 Deploy Cohere Transcribe 2B for multilingual ASR (Apache 2.0, tops Open ASR leaderboard)
- [ ] K.6.7 Deploy ACE-Step v1.5 for music generation (commercial-grade on consumer hardware, full songs in seconds)
- [ ] K.6.8 Deploy YuE for long-form music with lyrics (5min music with lyrical alignment, Apache 2.0)
K.7 3D Generation Models#
- [ ] K.7.1 Deploy Hunyuan3D 2.1 for production 3D (two-stage DiT: geometry then texture, PBR materials, mini variant at 12GB VRAM, Apache 2.0)
- [ ] K.7.2 Integrate UniRig for automatic skeletal rigging (SIGGRAPH 2025, autoregressive with bone-point cross-attention, Hugging Face checkpoint)
- [ ] K.7.3 Integrate Puppeteer for video-guided animation (skeleton prediction → skinning → video motion transfer, MIT License)
- [ ] K.7.4 Integrate gsplat for CUDA-accelerated Gaussian Splatting (growing feature set beyond SIGGRAPH paper)
- [ ] K.7.5 Support Khronos KHR_gaussian_splatting glTF extension (February 2026 standard for cross-platform GS)
K.8 Safety Models#
- [ ] K.8.1 Deploy Llama Guard 4 12B as primary content safety (multimodal text+image, MLCommons taxonomy)
- [ ] K.8.2 Deploy Granite Guardian for RAG safety (hallucination detection 0.854 AUC)
- [ ] K.8.3 Integrate Deepfake-o-Meter for deepfake detection (18 models across image/video/audio)
- [ ] K.8.4 Deploy ShieldGemma for additional content moderation (2B/9B/27B variants based on Gemma 2)
Summary Statistics#
| Phase | Tasks | Category |
|---|---|---|
| A — Data Flywheel (all 45 domains) | 155 | Passive training data collection |
| B — Dataset Management & Synthetic Data | 58 | Data infrastructure |
| C — Production Model Serving | 25 | Inference optimization |
| D — Training Infrastructure | 27 | Training frameworks & techniques |
| E — Alignment & Safety Training | 12 | Safety & alignment |
| F — Continuous Evaluation | 28 | Benchmarking & quality |
| G — MLOps & Governance | 21 | Operations & compliance |
| H — Edge Deployment | 16 | On-device inference |
| I — Domain Training Pipelines | 22 | Per-domain end-to-end |
| J — GPU Infrastructure | 12 | Compute management |
| K — SOTA Model Integration | 40 | Latest open-weight models |
| TOTAL NEW TASKS | 416 |
Combined with the ~540 existing unchecked tasks in TODOS.md (Phases 47, 43, 53), the total work for full ML sovereignty is ~956 tasks.
Data Flywheel Coverage (Phase A)#
The data flywheel now covers ALL 45 named domains in the Oshun monorepo:
| Tier | Domains | Data Volume | Tasks |
|---|---|---|---|
| Tier 1 — Core AI | Iris, Psyche, Isis, Sophia, Kuan Yin, Hathor, Nous | Highest (direct ML training signal) | 41 |
| Tier 2 — Content & Media | Veritas/Yaa, Euterpe, Aphrodite, Minerva/Metis, Calliope, Yemaya, Bellona | High (user-generated content + feedback) | 24 |
| Tier 3 — Game Engine | Maya (combat, NPC, economy, social, world gen, analytics) | Massive (every gameplay tick) | 15 |
| Tier 4 — Consumer Apps | Arete, Hestia, Demeter, Lakshmi, Mnemosyne, Tara/Meditation, Nike/Shakti, Aglaea, Nyx | High (continuous user interaction) | 36 |
| Tier 5 — Industrial/Ops | Maat, Asase, Cybele, Brigid, Freya, Saraswati, Annapurna, Athena, Uzume | Medium (IoT + operational data) | 12 |
| Tier 6 — Specialized | Aja, Aje, Themis, Nisaba, Oya, Airmid, Seshat, Galatea | Medium (domain-specific signals) | 15 |
| Tier 7 — Infrastructure | Shared event bus, logging, metrics, tracing, queue, cache, analytics | Cross-cutting (system telemetry) | 12 |
| TOTAL | 45 domains | 155 |
No domain is left uncovered. Every interaction in every domain produces passive training data artifacts.
Recommended Priority Order#
Wave 1 (Foundation — blocks everything else):
- A.10 + A.26 (cross-domain data collection infrastructure + infrastructure telemetry — 16 tasks)
- B.1 (dataset registry — 15 tasks)
- C.1 (vLLM/SGLang serving — 8 tasks)
- K.1 (deploy foundation LLMs — 10 tasks)
- Complete existing Phase 47.5.4 (training data pipeline — 20 tasks)
Wave 2 (Core domain flywheels + training):
- A.1-A.9 (core AI domain data collectors: Iris, Psyche, Isis, Sophia, Kuan Yin, Hathor, Veritas, Euterpe/Aphrodite, Minerva — 49 tasks)
- A.11 (Maya game engine flywheel — 15 tasks, highest volume)
- B.3 (synthetic data generation — 18 tasks)
- D.1-D.3 (training frameworks + GRPO/DAPO/RLVR — 22 tasks)
- F.1-F.2 (evaluation suites — 20 tasks)
- G.1 (experiment tracking — 6 tasks)
- Complete existing Phase 47.6 (safety framework — 60 tasks)
Wave 3 (Full domain coverage + serving):
- A.12-A.28 (remaining 33 domain flywheels — 91 tasks)
- I (domain-specific training pipelines — 22 tasks)
- K.2-K.8 (deploy all model types — 30 tasks)
- B.4 (annotation platform — 15 tasks)
- E (alignment training — 12 tasks)
- C.2-C.3 (inference optimization + format management — 17 tasks)
Wave 4 (Scale, edge & governance):
- H (edge deployment — 16 tasks)
- J (GPU infrastructure — 12 tasks)
- G.2-G.3 (lifecycle + governance — 15 tasks)
- D.4-D.6 (advanced optimizers, MoE, merging — 15 tasks)
- B.2 (data quality — 10 tasks)
- F.3 (evaluation infrastructure — 8 tasks)
- Complete existing Phases 47.7-47.16, 43, 53
Each wave builds on the previous. Wave 1 is prerequisite for all others.
Data Flywheel Implementation Strategy#
The data flywheel should be implemented in three layers:
-
Collection Layer (Wave 1-2): Wire up event listeners at each domain touchpoint. Each collector subscribes to existing domain events (no new events needed — the events already exist in the system) and routes training-relevant data to the centralized dataset registry.
-
Processing Layer (Wave 2-3): PII scrubbing, deduplication, quality scoring, format conversion. Each domain's raw data becomes training-ready datasets automatically.
-
Feedback Layer (Wave 3-4): Close the loop — training data improves models, improved models serve users better, better user interactions produce higher-quality training data. The system continuously improves just by being used.