Historical task source. The Eve task board holds current task state.
Phase 93: ML Sovereignty — Edge & On-Device Deployment#
Model compilation, optimization, and deployment pipelines for mobile, browser, embedded, and edge targets.
93.1 Model Conversion & Edge Deployment#
93.1.1 Platform-Specific Pipelines#
- 93.1.1.1 Build ONNX export pipeline (PyTorch/HF → ONNX with graph optimization, operator fusion, constant folding — automated with quality validation)
- 93.1.1.2 Implement CoreML conversion pipeline (ONNX → CoreML for iOS/macOS — targeting Apple MLX with Neural Accelerator on M5/A19 Pro)
- 93.1.1.3 Build TFLite conversion pipeline (ONNX → TensorFlow Lite for Android — INT8/INT4 quantization targeting Snapdragon 8 Elite Gen 5 NPU)
- 93.1.1.4 Create TensorRT compilation pipeline (ONNX → TensorRT for NVIDIA edge — Jetson/embedded with INT8 calibration)
- 93.1.1.5 Implement WebGPU inference runtime for browser deployment (shader-based ML inference using wgpu/WebGPU API)
- 93.1.1.6 Integrate MLC-LLM/WebLLM for browser-native LLM inference (cross-platform model compilation)
- 93.1.1.7 Integrate ExecuTorch 1.0 for mobile deployment (50KB base footprint, 12+ hardware backends, 80%+ popular edge LLMs)
- 93.1.1.8 Implement model pruning pipeline (structured + unstructured pruning with iterative fine-tuning to recover accuracy)
93.1.2 Edge Optimization & Monitoring#
- 93.1.2.1 Build quantization-aware training pipeline (train with quantization in the loop → 5x better than post-training quantization — via Axolotl)
- 93.1.2.2 Implement knowledge distillation automation (teacher → student: select teacher, generate distillation data, train student, evaluate)
- 93.1.2.3 Build target device profiler (benchmark on target hardware: latency, memory, battery, thermal — per device class)
- 93.1.2.4 Create model size/latency/accuracy Pareto optimizer (given quality floor and latency ceiling → optimal config)
- 93.1.2.5 Implement on-device model update mechanism (differential updates: download only changed weights, verify, apply atomically)
- 93.1.2.6 Build fallback chain system (try on-device → edge server → cloud — with quality-aware routing)
- 93.1.2.7 Build edge model monitoring (collect on-device metrics → aggregate centrally → detect per-device issues)
- 93.1.2.8 Implement small language model fine-tuning pipeline (target: Gemma 4 E2B/E4B, Phi-4 mini, Qwen3-4B → domain-specific SLMs)
93.1.3 Platform-Specific & Advanced Edge Optimization#
- 93.1.3.1 Build dedicated Apple MLX inference integration (native MLX model format conversion, mlx-lm serving, Apple Foundation Models framework integration, M5 Neural Accelerator optimization — not just CoreML conversion)
- 93.1.3.2 Integrate Qualcomm AI Engine Direct / QNN SDK for direct NPU inference on Snapdragon (HTP access with AOT compilation, up to 12x faster than CPU, INT8/INT4 on Hexagon)
- 93.1.3.3 Integrate MediaTek NeuroPilot SDK for on-device inference on MediaTek-powered Android devices (LiteRT NeuroPilot Accelerator, up to 12x faster than CPU)
- 93.1.3.4 Define model size/memory/latency budget specifications per device class (watch < 50MB, phone < 500MB, tablet < 2GB, laptop < 8GB — with quality floor per class)
- 93.1.3.5 Build offline-first inference mode ensuring zero-network-dependency operation (model + tokenizer + vocabulary bundled, self-contained inference, graceful degradation)
- 93.1.3.6 Build model encryption and IP protection for on-device deployment (AES encryption at rest, decryption-in-memory only, obfuscation, anti-extraction measures, CoreML encryption support)
- 93.1.3.7 Build A/B testing framework for on-device vs cloud model inference (split traffic, compare quality/latency/cost, auto-promote based on parity threshold)
- 93.1.3.8 Build battery-aware inference scheduler for mobile (throttle or defer non-critical inference below battery threshold, thermal-aware frequency scaling)
- 93.1.3.9 Build edge model deployment CI/CD pipeline (compile → test on target device profiles → package → distribute via OTA → verify installation → monitor)