## Phase 108: Kalika ML Sovereignty — Data Flywheel, Custom Models, Knowledge Graph Infrastructure & Training Pipelines

Implements the self-improving AI backbone that makes Kalika smarter with every
user interaction. The core insight: **formal verification provides a perfect,
automated reward signal** — unlike NLP where human annotation is subjective, in
mathematics a proof either type-checks or it doesn't. This enables a flywheel
where user-verified computations become training data, models improve, better
suggestions attract more users, and the cycle compounds. Informed by Terence
Tao's vision of AI as "mathematical sounding board", AlphaProof's RLVF
architecture, and LeanDojo's retrieval-augmented proving.

---

### 108.1 Training Data Infrastructure

#### 108.1.1 Data Ingestion Pipeline

- [x] **108.1.1.1** Ingest and preprocess Proof-Pile-2 (~55B tokens): arXiv
      math/physics papers, OpenWebMath, AlgebraicStack (Lean, Isabelle, Coq,
      Agda code) — clean LaTeX, normalize equation formatting, extract (problem,
      solution) pairs, tag by mathematical domain (MSC classification), store in
      Parquet format with metadata
- [x] **108.1.1.2** Ingest formal proof corpora: Lean 4 Mathlib (~190K theorems
      — extract tactic_state/tactic pairs following LeanDojo's methodology), Coq
      MathComp (~800K lines), Isabelle AFP (~800 entries), Metamath set.mm (~40K
      theorems) — parse proof scripts into structured training data with
      intermediate proof states
- [x] **108.1.1.3** Ingest arXiv LaTeX source for physics: download and process
      hep-th, hep-ph, gr-qc, math-ph, math.AG, math.NT, math.DG categories
      (~500K papers) — extract equations, theorem statements, proof sketches,
      and figure references, preserving LaTeX structure for equation-aware
      training
- [x] **108.1.1.4** Ingest structured mathematical databases: LMFDB (elliptic
      curves, modular forms, L-functions, number fields), OEIS (~370K integer
      sequences with formulas), Stacks Project (~7,400 tagged results with
      dependency graph), KnotInfo/LinkInfo (knot invariants), PDG (particle
      data) — convert to unified JSON/Parquet format
- [x] **108.1.1.5** Build synthetic data generation pipeline: using Kalika's CAS
      engine, generate unlimited (expression, simplified_form) pairs, (integral,
      antiderivative) pairs, (ODE, solution) pairs, (matrix, eigenvalues) pairs
      — with verified correctness (CAS computes both directions), varying
      difficulty from undergraduate to research level, tagged by domain

#### 108.1.2 Data Quality & Filtering

- [x] **108.1.2.1** Train math-content classifier: fastText model distinguishing
      mathematical content from non-mathematical web text (trained on positive
      examples from arXiv/MathOverflow and negative examples from general web) —
      for filtering large web crawls to extract math-relevant content (matching
      OpenWebMath's approach)
- [x] **108.1.2.2** Implement quality scoring pipeline: for each training
      document, compute quality score based on: LaTeX well-formedness, equation
      density, citation presence, author reputation (h-index from Semantic
      Scholar), publication venue, and a trained quality model — higher-quality
      documents get higher sampling weight during training
- [x] **108.1.2.3** Implement deduplication: exact and near-duplicate removal
      using MinHash/LSH on document content, with special handling for
      mathematical content (normalize LaTeX, ignore formatting differences,
      detect semantically equivalent but syntactically different expressions)
- [x] **108.1.2.4** Implement domain balancing: ensure training data covers all
      major mathematical domains (algebra, analysis, topology, number theory,
      geometry, combinatorics, logic, physics) in proportion to their research
      volume, with upsampling of underrepresented domains (e.g., category
      theory, algebraic geometry have fewer online resources than calculus)

---

### 108.2 Knowledge Graph Infrastructure

#### 108.2.1 Graph Database & Ontology

- [x] **108.2.1.1** Deploy Neo4j graph database for Kalika knowledge graph (via
      Docker profile `graph`): configure for ~10M nodes and ~100M edges, with
      full-text search index on theorem statements, vector index (Neo4j 5.11+)
      for embedding-based semantic search, and Cypher query interface — with
      PostgreSQL fallback using Apache AGE extension for deployments that can't
      run Neo4j
- [x] **108.2.1.2** Implement mathematical ontology schema: entity types
      (Concept, Theorem, Definition, Conjecture, Proof, Algorithm,
      MathematicalObject, PhysicalTheory, Experiment, Person, Paper),
      relationship types (proves, depends_on, generalizes, specializes,
      is_instance_of, defined_in, cited_by, contradicts, is_equivalent_to,
      is_dual_to, computes, experimentally_validates), property types
      (difficulty_rating, domain_tags, formal_status, confidence_level,
      year_discovered, discoverer)
- [x] **108.2.1.3** Seed knowledge graph from structured databases: import LMFDB
      entities and relationships (elliptic curves linked to modular forms via
      modularity theorem, L-functions linked to number fields), Stacks Project
      dependency graph (7,400 results with explicit dependencies), MSC
      classification tree (6,000 categories), OEIS cross-references, nLab
      concept links — providing a rich initial graph
- [x] **108.2.1.4** Implement theorem dependency extraction from Mathlib: parse
      Lean 4 Mathlib source to extract which theorems depend on which other
      theorems (from `import` and `#check` usage), creating a complete
      dependency DAG of ~190K formalized results — the most precise theorem
      dependency graph in existence
- [x] **108.2.1.5** Implement automatic knowledge graph expansion: when users
      prove new results or verify computations in Kalika, automatically add them
      to the knowledge graph with full provenance (who proved it, when, using
      what tools, what dependencies) — the knowledge graph grows with every user
      interaction

#### 108.2.2 Graph-Based Reasoning

- [x] **108.2.2.1** Implement graph-based RAG: for any mathematical query,
      traverse the knowledge graph from identified concepts to find relevant
      context (not just vector similarity but structural relevance — "what
      theorems are prerequisites for X?", "what results depend on Y?"), combine
      graph-retrieved context with vector-retrieved context for augmented
      generation
- [x] **108.2.2.2** Implement prerequisite computation: given a target theorem
      or concept, traverse the dependency graph backward to compute the complete
      set of prerequisites, ordered by dependency (learn A before B before C) —
      for educational use (Metis integration) and for AI agents (know what
      tools/theorems are available)
- [x] **108.2.2.3** Implement analogy detection: use graph structure to find
      analogies between mathematical domains — if concept A in topology has a
      similar neighborhood structure to concept B in algebra, surface this as a
      potential analogy. Use GNN-based node embedding to detect structural
      similarity beyond explicit edges
- [x] **108.2.2.4** Implement knowledge gap identification: compare the
      knowledge graph against the MSC classification to identify areas with few
      formalized results, connecting to the Lean Mathlib roadmap — guide both
      human formalization efforts and AI exploration agent priorities

---

### 108.3 Math-Specialized Embedding Model

- [x] **108.3.1.1** Train math embedding model: fine-tune a base embedding model
      (E5-Mistral or GTE-Qwen2) on mathematical similarity data sourced from:
      Mathlib dependency pairs (theorem, its dependency), StackExchange Math
      duplicate questions, arXiv citation pairs (paper, cited paper), OEIS
      cross-reference pairs — producing embeddings where mathematically related
      content is close in vector space
- [x] **108.3.1.2** Implement LaTeX-aware embedding preprocessing: before
      embedding, normalize LaTeX (canonicalize variable names, expand macros,
      standardize notation), convert inline math to a textual description for
      the embedding model, and create separate embeddings for the natural
      language context and the mathematical content — addressing the challenge
      that syntactically different LaTeX can be semantically identical
- [x] **108.3.1.3** Implement proof-state embedding model (matching LeanDojo's
      retriever): train a model that embeds Lean 4 proof states (goal type +
      hypotheses) into a vector space where relevant premises (theorems from
      Mathlib that help close the goal) are nearest neighbors — the core
      retrieval component for AI-assisted theorem proving
- [x] **108.3.1.4** Deploy embedding model as service: run the trained embedding
      model via ONNX Runtime or vLLM for low-latency inference (<50ms per
      embedding), with batch processing for bulk indexing, and integration with
      Qdrant vector database for similarity search across the entire knowledge
      base

---

### 108.4 Custom Model Training

#### 108.4.1 Continued Pretraining

- [x] **108.4.1.1** Perform continued pretraining of base LLM on math corpus:
      starting from a strong code-capable base (Qwen2.5-72B-Instruct or
      Llama-3.1-70B as available via @shared/ai or self-hosted), continue
      pretraining on ~50B tokens of math content (Proof-Pile-2 + arXiv +
      synthetic data) with learning rate 1e-5, linear warmup, and 10-20% general
      text mixing to prevent catastrophic forgetting — producing Kalika's
      foundation math model
- [x] **108.4.1.2** Implement domain-specific tokenizer extension: add
      math-specific tokens to the base tokenizer (common LaTeX commands as
      single tokens: `\frac`, `\int`, `\sum`, `\partial`, `\nabla`, common
      symbol sequences, Lean tactic names) to reduce token count for
      mathematical text by ~20-30%, improving both efficiency and reasoning
      quality
- [x] **108.4.1.3** Train multiple model sizes for routing: 7B (fast, for simple
      queries and autocomplete), 34B (medium, for standard computations and
      undergraduate proofs), 72B+ (large, for research-level reasoning) — all
      sharing the math-specialized tokenizer and continued pretraining, with the
      larger models being more capable but more expensive

#### 108.4.2 Instruction Tuning & Alignment

- [x] **108.4.2.1** Create math instruction tuning dataset: compile ~100K
      high-quality (instruction, response) pairs covering: symbolic computation
      ("Simplify this expression..."), proof generation ("Prove that..."),
      explanation ("Explain why..."), translation ("Convert this to Lean 4..."),
      tool use ("Use the CAS to compute..."), literature search ("Find papers
      about..."), and multi-step research tasks — with verified responses
      (CAS-checked, formally verified, or expert-validated)
- [x] **108.4.2.2** Implement mixed chain-of-thought and program-of-thought
      training: instructions that mix natural language reasoning with CAS tool
      calls and code execution, teaching the model when to reason symbolically,
      when to compute numerically, and when to invoke formal verification —
      matching MAmmoTH's hybrid approach but with Kalika-specific tools
- [x] **108.4.2.3** Implement **RLVF (Reinforcement Learning from Verification
      Feedback)**: use Lean 4 formal verification as the reward signal — the
      model generates candidate proofs, Lean type-checks them, success = reward,
      failure + error message = negative reward with diagnostic information.
      This is the key training innovation: formal provers provide perfect,
      automated, scalable reward signals, unlike RLHF where human annotation is
      the bottleneck
- [x] **108.4.2.4** Implement Constitutional AI for mathematical rigor: define
      mathematical "constitution" (never claim a result is proven without a
      complete proof, distinguish rigorous proof from heuristic argument from
      numerical evidence, never hallucinate citations, always state assumptions)
      — generate (rigorous, sloppy) response pairs, train the model to prefer
      rigor
- [x] **108.4.2.5** Train domain-specific reward model: score mathematical
      outputs on correctness (verified by CAS/prover when possible), clarity
      (compared to high-quality textbook proofs), efficiency (proof length,
      computation steps), and relevance (addresses the actual question) — used
      for rejection sampling and reinforcement learning

#### 108.4.3 Tool-Augmented Training

- [x] **108.4.3.1** Create tool-use training data: ~20K examples showing the
      model deciding when to call Kalika CAS (for computation), Lean 4 (for
      formal verification), Python (for numerical experiments), arXiv API (for
      literature search), and the knowledge graph (for theorem lookup) — each
      example includes the tool call, the tool response, and the model's
      interpretation of the result
- [x] **108.4.3.2** Implement tool-use fine-tuning: train the model to emit
      special tokens for tool calls (`<cas>integrate(sin(x)^2, x)</cas>`,
      `<lean>exact?</lean>`,
      `<search>arXiv:Calabi-Yau moduli stabilization</search>`), parse tool
      responses, and continue reasoning — matching Toolformer but with
      Kalika-specific tools
- [x] **108.4.3.3** Implement CAS-in-the-loop training: during training, the
      model can actually call the CAS engine to verify intermediate computation
      steps, using the verification result to guide further generation — the
      model learns to self-correct by checking its work

---

### 108.5 Data Flywheel

#### 108.5.1 User Interaction Data Collection

- [x] **108.5.1.1** Implement computation logging (privacy-preserving): log all
      CAS computations with their inputs, outputs, and verification status —
      with user consent, anonymization (strip variable names that might identify
      research topics), and opt-out. Aggregate patterns (e.g., "integration of
      hypergeometric functions is requested 500x/day") without storing
      individual expressions
- [x] **108.5.1.2** Implement preference data collection: when AI suggests
      multiple proof strategies or simplification approaches, record which one
      the user selects — this is implicit preference data for DPO/RLHF training.
      When the user corrects an AI suggestion, record the (original, corrected)
      pair
- [x] **108.5.1.3** Implement formal proof contribution pipeline: when a user
      formally verifies a result in Lean 4, offer to contribute the proof to
      Kalika's verified knowledge base (opt-in). Accepted proofs become: (1)
      training data for the proof model, (2) nodes in the knowledge graph, (3)
      cached results for future users. This is the purest form of the flywheel:
      every verified proof makes the platform permanently better

#### 108.5.2 Continuous Improvement

- [x] **108.5.2.1** Implement weekly model refresh pipeline: aggregate the
      week's verified computations, accepted proofs, and preference data →
      augment the training dataset → fine-tune the current model (LoRA, 1 epoch,
      on the new data) → evaluate on held-out benchmark → if improved, deploy →
      rollback if degraded
- [x] **108.5.2.2** Implement A/B testing for model versions: when a new model
      version is trained, deploy it to a random subset of users, measure:
      suggestion acceptance rate, verification success rate, user satisfaction
      (explicit thumbs up/down), and session length — only promote to full
      deployment if metrics improve
- [x] **108.5.2.3** Implement synthetic data generation from user patterns:
      identify frequently requested computation types where the model
      underperforms → generate targeted synthetic training data for those types
      (using CAS to produce verified examples) → retrain — closing specific
      capability gaps driven by user demand

---

### 108.6 Model Serving & Routing

- [x] **108.6.1.1** Implement model router: a lightweight classifier that
      examines the input and routes to the appropriate model — simple
      arithmetic/autocomplete → 7B model (or direct CAS call, cheapest),
      undergraduate-level proofs → 34B model with tool augmentation,
      research-level reasoning → 72B+ model with extensive search, formal proof
      search → specialized Lean model + MCTS tree search
- [x] **108.6.1.2** Implement prefix caching: mathematical proofs have long
      shared prefixes (theorem statement, definitions, prior proof steps) —
      cache the KV cache for these prefixes across requests to reduce latency by
      50-80% for interactive proof assistance (matching vLLM's prefix caching)
- [x] **108.6.1.3** Implement speculative decoding for math: use the 7B model as
      a draft model to predict 3-5 tokens ahead, verified by the larger model —
      effective for LaTeX formatting, variable reuse, and standard proof
      patterns but with fallback to full computation for novel mathematical
      content
- [x] **108.6.1.4** Implement batch conjecture testing infrastructure: for
      FunSearch-style conjecture generation (Phase 104), support evaluation of
      millions of candidate conjectures with coarse → medium → fine filtering
      pipeline (small model rejects obviously false → medium model attempts
      proof sketch → large model + formal prover for survivors), using
      spot/preemptible GPU instances for cost efficiency

---

### 108.7 Tests & Benchmarks

- [x] **108.7.1.1** Create Kalika math benchmark: 500 problems spanning all
      mathematical domains covered by Kalika (algebra, analysis, topology,
      number theory, physics), at difficulty levels from undergraduate to
      research — measure: correctness, solution completeness, tool usage
      appropriateness, verification success rate. Run against every model
      version to track improvement
- [x] **108.7.1.2** Evaluate on external benchmarks: miniF2F (formal theorem
      proving), MATH (competition math), GPQA-physics (graduate physics),
      FrontierMath subset (research-level) — track Kalika model performance
      relative to frontier models (Claude, GPT-4o, Gemini) and specialized math
      models (DeepSeek-Math, Llemma, InternLM-Math)
- [x] **108.7.1.3** Measure flywheel metrics: weekly growth in knowledge graph
      size (nodes, edges), verified proof count, computation cache hit rate,
      model suggestion acceptance rate, user retention — the quantitative health
      of the self-improving cycle
