Historical task source. The Eve task board holds current task state.
Phase 108: Kalika ML Sovereignty — Data Flywheel, Custom Models, Knowledge Graph Infrastructure & Training Pipelines#
Implements the self-improving AI backbone that makes Kalika smarter with every user interaction. The core insight: formal verification provides a perfect, automated reward signal — unlike NLP where human annotation is subjective, in mathematics a proof either type-checks or it doesn't. This enables a flywheel where user-verified computations become training data, models improve, better suggestions attract more users, and the cycle compounds. Informed by Terence Tao's vision of AI as "mathematical sounding board", AlphaProof's RLVF architecture, and LeanDojo's retrieval-augmented proving.
108.1 Training Data Infrastructure#
108.1.1 Data Ingestion Pipeline#
- 108.1.1.1 Ingest and preprocess Proof-Pile-2 (~55B tokens): arXiv math/physics papers, OpenWebMath, AlgebraicStack (Lean, Isabelle, Coq, Agda code) — clean LaTeX, normalize equation formatting, extract (problem, solution) pairs, tag by mathematical domain (MSC classification), store in Parquet format with metadata
- 108.1.1.2 Ingest formal proof corpora: Lean 4 Mathlib (~190K theorems — extract tactic_state/tactic pairs following LeanDojo's methodology), Coq MathComp (~800K lines), Isabelle AFP (~800 entries), Metamath set.mm (~40K theorems) — parse proof scripts into structured training data with intermediate proof states
- 108.1.1.3 Ingest arXiv LaTeX source for physics: download and process hep-th, hep-ph, gr-qc, math-ph, math.AG, math.NT, math.DG categories (~500K papers) — extract equations, theorem statements, proof sketches, and figure references, preserving LaTeX structure for equation-aware training
- 108.1.1.4 Ingest structured mathematical databases: LMFDB (elliptic curves, modular forms, L-functions, number fields), OEIS (~370K integer sequences with formulas), Stacks Project (~7,400 tagged results with dependency graph), KnotInfo/LinkInfo (knot invariants), PDG (particle data) — convert to unified JSON/Parquet format
- 108.1.1.5 Build synthetic data generation pipeline: using Kalika's CAS engine, generate unlimited (expression, simplified_form) pairs, (integral, antiderivative) pairs, (ODE, solution) pairs, (matrix, eigenvalues) pairs — with verified correctness (CAS computes both directions), varying difficulty from undergraduate to research level, tagged by domain
108.1.2 Data Quality & Filtering#
- 108.1.2.1 Train math-content classifier: fastText model distinguishing mathematical content from non-mathematical web text (trained on positive examples from arXiv/MathOverflow and negative examples from general web) — for filtering large web crawls to extract math-relevant content (matching OpenWebMath's approach)
- 108.1.2.2 Implement quality scoring pipeline: for each training document, compute quality score based on: LaTeX well-formedness, equation density, citation presence, author reputation (h-index from Semantic Scholar), publication venue, and a trained quality model — higher-quality documents get higher sampling weight during training
- 108.1.2.3 Implement deduplication: exact and near-duplicate removal using MinHash/LSH on document content, with special handling for mathematical content (normalize LaTeX, ignore formatting differences, detect semantically equivalent but syntactically different expressions)
- 108.1.2.4 Implement domain balancing: ensure training data covers all major mathematical domains (algebra, analysis, topology, number theory, geometry, combinatorics, logic, physics) in proportion to their research volume, with upsampling of underrepresented domains (e.g., category theory, algebraic geometry have fewer online resources than calculus)
108.2 Knowledge Graph Infrastructure#
108.2.1 Graph Database & Ontology#
- 108.2.1.1 Deploy Neo4j graph database for Kalika knowledge graph (via
Docker profile
graph): configure for ~10M nodes and ~100M edges, with full-text search index on theorem statements, vector index (Neo4j 5.11+) for embedding-based semantic search, and Cypher query interface — with PostgreSQL fallback using Apache AGE extension for deployments that can't run Neo4j - 108.2.1.2 Implement mathematical ontology schema: entity types (Concept, Theorem, Definition, Conjecture, Proof, Algorithm, MathematicalObject, PhysicalTheory, Experiment, Person, Paper), relationship types (proves, depends_on, generalizes, specializes, is_instance_of, defined_in, cited_by, contradicts, is_equivalent_to, is_dual_to, computes, experimentally_validates), property types (difficulty_rating, domain_tags, formal_status, confidence_level, year_discovered, discoverer)
- 108.2.1.3 Seed knowledge graph from structured databases: import LMFDB entities and relationships (elliptic curves linked to modular forms via modularity theorem, L-functions linked to number fields), Stacks Project dependency graph (7,400 results with explicit dependencies), MSC classification tree (6,000 categories), OEIS cross-references, nLab concept links — providing a rich initial graph
- 108.2.1.4 Implement theorem dependency extraction from Mathlib: parse
Lean 4 Mathlib source to extract which theorems depend on which other
theorems (from
importand#checkusage), creating a complete dependency DAG of ~190K formalized results — the most precise theorem dependency graph in existence - 108.2.1.5 Implement automatic knowledge graph expansion: when users prove new results or verify computations in Kalika, automatically add them to the knowledge graph with full provenance (who proved it, when, using what tools, what dependencies) — the knowledge graph grows with every user interaction
108.2.2 Graph-Based Reasoning#
- 108.2.2.1 Implement graph-based RAG: for any mathematical query, traverse the knowledge graph from identified concepts to find relevant context (not just vector similarity but structural relevance — "what theorems are prerequisites for X?", "what results depend on Y?"), combine graph-retrieved context with vector-retrieved context for augmented generation
- 108.2.2.2 Implement prerequisite computation: given a target theorem or concept, traverse the dependency graph backward to compute the complete set of prerequisites, ordered by dependency (learn A before B before C) — for educational use (Metis integration) and for AI agents (know what tools/theorems are available)
- 108.2.2.3 Implement analogy detection: use graph structure to find analogies between mathematical domains — if concept A in topology has a similar neighborhood structure to concept B in algebra, surface this as a potential analogy. Use GNN-based node embedding to detect structural similarity beyond explicit edges
- 108.2.2.4 Implement knowledge gap identification: compare the knowledge graph against the MSC classification to identify areas with few formalized results, connecting to the Lean Mathlib roadmap — guide both human formalization efforts and AI exploration agent priorities
108.3 Math-Specialized Embedding Model#
- 108.3.1.1 Train math embedding model: fine-tune a base embedding model (E5-Mistral or GTE-Qwen2) on mathematical similarity data sourced from: Mathlib dependency pairs (theorem, its dependency), StackExchange Math duplicate questions, arXiv citation pairs (paper, cited paper), OEIS cross-reference pairs — producing embeddings where mathematically related content is close in vector space
- 108.3.1.2 Implement LaTeX-aware embedding preprocessing: before embedding, normalize LaTeX (canonicalize variable names, expand macros, standardize notation), convert inline math to a textual description for the embedding model, and create separate embeddings for the natural language context and the mathematical content — addressing the challenge that syntactically different LaTeX can be semantically identical
- 108.3.1.3 Implement proof-state embedding model (matching LeanDojo's retriever): train a model that embeds Lean 4 proof states (goal type + hypotheses) into a vector space where relevant premises (theorems from Mathlib that help close the goal) are nearest neighbors — the core retrieval component for AI-assisted theorem proving
- 108.3.1.4 Deploy embedding model as service: run the trained embedding model via ONNX Runtime or vLLM for low-latency inference (<50ms per embedding), with batch processing for bulk indexing, and integration with Qdrant vector database for similarity search across the entire knowledge base
108.4 Custom Model Training#
108.4.1 Continued Pretraining#
- 108.4.1.1 Perform continued pretraining of base LLM on math corpus: starting from a strong code-capable base (Qwen2.5-72B-Instruct or Llama-3.1-70B as available via @shared/ai or self-hosted), continue pretraining on ~50B tokens of math content (Proof-Pile-2 + arXiv + synthetic data) with learning rate 1e-5, linear warmup, and 10-20% general text mixing to prevent catastrophic forgetting — producing Kalika's foundation math model
- 108.4.1.2 Implement domain-specific tokenizer extension: add
math-specific tokens to the base tokenizer (common LaTeX commands as
single tokens:
\frac,\int,\sum,\partial,\nabla, common symbol sequences, Lean tactic names) to reduce token count for mathematical text by ~20-30%, improving both efficiency and reasoning quality - 108.4.1.3 Train multiple model sizes for routing: 7B (fast, for simple queries and autocomplete), 34B (medium, for standard computations and undergraduate proofs), 72B+ (large, for research-level reasoning) — all sharing the math-specialized tokenizer and continued pretraining, with the larger models being more capable but more expensive
108.4.2 Instruction Tuning & Alignment#
- 108.4.2.1 Create math instruction tuning dataset: compile ~100K high-quality (instruction, response) pairs covering: symbolic computation ("Simplify this expression..."), proof generation ("Prove that..."), explanation ("Explain why..."), translation ("Convert this to Lean 4..."), tool use ("Use the CAS to compute..."), literature search ("Find papers about..."), and multi-step research tasks — with verified responses (CAS-checked, formally verified, or expert-validated)
- 108.4.2.2 Implement mixed chain-of-thought and program-of-thought training: instructions that mix natural language reasoning with CAS tool calls and code execution, teaching the model when to reason symbolically, when to compute numerically, and when to invoke formal verification — matching MAmmoTH's hybrid approach but with Kalika-specific tools
- 108.4.2.3 Implement RLVF (Reinforcement Learning from Verification Feedback): use Lean 4 formal verification as the reward signal — the model generates candidate proofs, Lean type-checks them, success = reward, failure + error message = negative reward with diagnostic information. This is the key training innovation: formal provers provide perfect, automated, scalable reward signals, unlike RLHF where human annotation is the bottleneck
- 108.4.2.4 Implement Constitutional AI for mathematical rigor: define mathematical "constitution" (never claim a result is proven without a complete proof, distinguish rigorous proof from heuristic argument from numerical evidence, never hallucinate citations, always state assumptions) — generate (rigorous, sloppy) response pairs, train the model to prefer rigor
- 108.4.2.5 Train domain-specific reward model: score mathematical outputs on correctness (verified by CAS/prover when possible), clarity (compared to high-quality textbook proofs), efficiency (proof length, computation steps), and relevance (addresses the actual question) — used for rejection sampling and reinforcement learning
108.4.3 Tool-Augmented Training#
- 108.4.3.1 Create tool-use training data: ~20K examples showing the model deciding when to call Kalika CAS (for computation), Lean 4 (for formal verification), Python (for numerical experiments), arXiv API (for literature search), and the knowledge graph (for theorem lookup) — each example includes the tool call, the tool response, and the model's interpretation of the result
- 108.4.3.2 Implement tool-use fine-tuning: train the model to emit
special tokens for tool calls (
<cas>integrate(sin(x)^2, x)</cas>,<lean>exact?</lean>,<search>arXiv:Calabi-Yau moduli stabilization</search>), parse tool responses, and continue reasoning — matching Toolformer but with Kalika-specific tools - 108.4.3.3 Implement CAS-in-the-loop training: during training, the model can actually call the CAS engine to verify intermediate computation steps, using the verification result to guide further generation — the model learns to self-correct by checking its work
108.5 Data Flywheel#
108.5.1 User Interaction Data Collection#
- 108.5.1.1 Implement computation logging (privacy-preserving): log all CAS computations with their inputs, outputs, and verification status — with user consent, anonymization (strip variable names that might identify research topics), and opt-out. Aggregate patterns (e.g., "integration of hypergeometric functions is requested 500x/day") without storing individual expressions
- 108.5.1.2 Implement preference data collection: when AI suggests multiple proof strategies or simplification approaches, record which one the user selects — this is implicit preference data for DPO/RLHF training. When the user corrects an AI suggestion, record the (original, corrected) pair
- 108.5.1.3 Implement formal proof contribution pipeline: when a user formally verifies a result in Lean 4, offer to contribute the proof to Kalika's verified knowledge base (opt-in). Accepted proofs become: (1) training data for the proof model, (2) nodes in the knowledge graph, (3) cached results for future users. This is the purest form of the flywheel: every verified proof makes the platform permanently better
108.5.2 Continuous Improvement#
- 108.5.2.1 Implement weekly model refresh pipeline: aggregate the week's verified computations, accepted proofs, and preference data → augment the training dataset → fine-tune the current model (LoRA, 1 epoch, on the new data) → evaluate on held-out benchmark → if improved, deploy → rollback if degraded
- 108.5.2.2 Implement A/B testing for model versions: when a new model version is trained, deploy it to a random subset of users, measure: suggestion acceptance rate, verification success rate, user satisfaction (explicit thumbs up/down), and session length — only promote to full deployment if metrics improve
- 108.5.2.3 Implement synthetic data generation from user patterns: identify frequently requested computation types where the model underperforms → generate targeted synthetic training data for those types (using CAS to produce verified examples) → retrain — closing specific capability gaps driven by user demand
108.6 Model Serving & Routing#
- 108.6.1.1 Implement model router: a lightweight classifier that examines the input and routes to the appropriate model — simple arithmetic/autocomplete → 7B model (or direct CAS call, cheapest), undergraduate-level proofs → 34B model with tool augmentation, research-level reasoning → 72B+ model with extensive search, formal proof search → specialized Lean model + MCTS tree search
- 108.6.1.2 Implement prefix caching: mathematical proofs have long shared prefixes (theorem statement, definitions, prior proof steps) — cache the KV cache for these prefixes across requests to reduce latency by 50-80% for interactive proof assistance (matching vLLM's prefix caching)
- 108.6.1.3 Implement speculative decoding for math: use the 7B model as a draft model to predict 3-5 tokens ahead, verified by the larger model — effective for LaTeX formatting, variable reuse, and standard proof patterns but with fallback to full computation for novel mathematical content
- 108.6.1.4 Implement batch conjecture testing infrastructure: for FunSearch-style conjecture generation (Phase 104), support evaluation of millions of candidate conjectures with coarse → medium → fine filtering pipeline (small model rejects obviously false → medium model attempts proof sketch → large model + formal prover for survivors), using spot/preemptible GPU instances for cost efficiency
108.7 Tests & Benchmarks#
- 108.7.1.1 Create Kalika math benchmark: 500 problems spanning all mathematical domains covered by Kalika (algebra, analysis, topology, number theory, physics), at difficulty levels from undergraduate to research — measure: correctness, solution completeness, tool usage appropriateness, verification success rate. Run against every model version to track improvement
- 108.7.1.2 Evaluate on external benchmarks: miniF2F (formal theorem proving), MATH (competition math), GPQA-physics (graduate physics), FrontierMath subset (research-level) — track Kalika model performance relative to frontier models (Claude, GPT-4o, Gemini) and specialized math models (DeepSeek-Math, Llemma, InternLM-Math)
- 108.7.1.3 Measure flywheel metrics: weekly growth in knowledge graph size (nodes, edges), verified proof count, computation cache hit rate, model suggestion acceptance rate, user retention — the quantitative health of the self-improving cycle