sophia library

A one-line namespace re-export (oshun-client/src/index.ts is export * from '@sophia/client';). It exists to expose the Sophia client SDK under the platform-wide @oshun/* scope so cross-domain consumers can import it without reaching into the @sophia/* scope. No logic of its own — intentionally a thin alias, honestly described as such.

The TypeScript client SDK for Sophia's HTTP services (client/src). A unified SophiaClient aggregates SearchClient, IngestionClient, and KnowledgeGraphClient, each with fluent builders (SearchQueryBuilder, GraphQueryBuilder) and a shared HttpClient with retries/error handling (SophiaClientError). The branded ID types and request/response shapes mirror a generated OpenAPI surface (client/src/generated/openapi.ts, ~1.7K lines). createLocalSophiaClient targets local dev. This is the canonical way other domains call Sophia over the wire.

The Sophia domain-schema foundation (schemas/src). Zod schemas and branded ID types for research documents, citations, claims, knowledge packs, entities, and relations, plus a typed error surface (errors.ts) and ID helpers (ids.ts). Each domain object gets its own file (citation.ts, claim.ts, entity.ts, relation.ts, research-document.ts, knowledge-pack.ts), re-exported from index.ts. This is the shared vocabulary the rest of the area validates against.

The Prisma data-access library (database/src). It is a thin, honest wrapper over a generated Prisma client: getSophiaClient is a lazily-initialized singleton, createSophiaClient makes fresh connections, and both fail loud — they throw "Prisma client not generated. Run nx run @sophia/database:prisma:generate first." rather than pretend a client exists. The generated client under database/src/generated/ is a build artifact, so the checked-in source is the client accessors plus model/enum re-exports.

The vector-store abstraction (vectordb/src). A common store interface with four backends — an in-memory memory-store (~619 LOC), and real adapters for milvus (~850 LOC), pinecone (~819 LOC), and qdrant (~842 LOC) — plus collection schemas (e.g. KNOWLEDGE_CHUNKS_SCHEMA), similarity utilities, batch helpers, and a health module. Supports upsert, vector search with structured filters, and hybrid search; createMemoryStore for dev, createMilvusStoreFromEnv for production. Genuine multi-provider adapters.

The advanced research-agent framework (agents/src), ported from Metis/Minerva. It composes a full research pipeline: QueryDecomposer (with scientific/simple variants), AdvancedResearchAgent over a pluggable SearchProvider (InMemorySearchProvider included), and a verification suite — FactCheckingAgent, ContradictionDetector, TemporalConsistencyChecker, StatisticalValidator, MethodologyEvaluator — all wired together by ResearchOrchestrator. Each agent ships strict/lenient/quick factory variants (agents/src/verification/*, ~3.4K LOC; the whole @sophia/agents lib is ~6.6K), and the public surface is the barrel in agents/src/index.ts.

Citation analysis ported from the Metis CitationAnalyzer (citation-analysis/src). Four analyzers: SourceQualityAnalyzer (with STEM/humanities variants, multi-dimensional QualityDimension scoring and SourceTier classification), CitationNetworkAnalyzer (authority scores, co-citation, bibliographic coupling, clustering), CitationLinter (rule-based lint with severity levels), and ScholarlyConsensusAnalyzer (per-claim stance aggregation, agreement matrices, contradictions, temporal trends). Real, config-driven implementations under source-quality/, network/, linters/, and consensus/.

A small, focused cross-domain library (concordia-knowledge/src, precedent-lookup.ts, ~142 lines) implementing §179.7: Sophia-backed precedent lookup for Concordia cases. It defines PrecedentQuerySchema / PrecedentMatchSchema (Zod, with a fixed useCaseClass enum and ISO-country jurisdiction validation) and a deterministic, documented bounded-heuristic scorePrecedent (a weighted average of jurisdiction match, family overlap, and keyword overlap, each in [0,1]) plus rankPrecedents. The determinism is intentional so "audit reproducibility holds across runs."

The educational-asset corpus (corpus/src): EducationalImageCorpus, a DiagramTemplateLibrary, and CorpusMaintenanceManager (each ~1.1–1.3K LOC). It manages curated educational images and diagram templates plus the maintenance operations over that collection. Substantial real implementations behind the types.ts surface; the barrel re-exports all three modules.

Cultural-sensitivity, localization, and accessibility for Sophia content (cultural-research/src). The largest single file in the area (consultant/cultural-consultant.ts, ~4.6K LOC) plus a ~3K-LOC LocalizationManager, a ~1.7K-LOC AccessibilityManager, and a CulturalDatabase. The AI analysis is structured around a CulturalAIProvider interface (sensitivity checks, suggestions, representation analysis) with real orchestration logic and Mock*Provider implementations (MockCulturalAIProvider, etc.) plus createMock* factories for local development — an honest provider seam, not fabricated AI output.

Real, model-backed embeddings (embeddings/src). The header is explicit that this replaced a former SHA-256 hash "embedding" stub: TransformersTextBackend runs sentence-transformers/all-MiniLM-L6-v2 in-process via onnxruntime through @huggingface/transformers (384-d, mean-pooled, L2-normalized so dot product == cosine), with no hash fallback — it throws NotConfiguredError when the model can't load. Also ships an image-embedder, multimodal-embedder, a clip-multimodal-backend, a dense-retriever, and an embedding-cache. This is a genuine fail-loud inference seam.

The quality-gate library (evaluation/src). RetrievalEvaluator (faithfulness/relevancy, precision/recall, ~1K LOC), CitationChecker (quote accuracy, entailment, attribution), and GroundednessScorer (claim extraction + evidence matching + hybrid semantic/lexical scoring against a configurable threshold). Plus benchmark runners — sophia-release-gate-benchmark (~1.1K LOC) and veritas-explainer-benchmark — an answer-regrounding module, and an ingestion-quality evaluator with remediation/routing. The AI/semantic steps sit behind AIEvaluationProvider/SemanticSimilarityProvider interfaces.

The indexing pipeline (indexing/src): document chunking → embedding → vector index. Chunking ships multiple strategies over a base-chunker; the embedding/ module has providers, a cache, a service, and types, while a parallel embeddings/ module adds audio/text/multimodal embedders, a backend abstraction, and a cutover path; graphs/ carries an entity-resolver and graph-visualization; and index/ has the builder, similarity, and types. createIndexBuilder indexes a document and exposes search. Large, real implementations (entity-resolver ~621 LOC, graph-visualization ~888 LOC).

The six-stage ingestion pipeline (ingestion/src, §9.9) with provenance, idempotency, and tenant isolation. The standout is a rich adapters/ registry: peer-reviewed PDF, news-article HTML, RSS/Atom feed, audio- and video-transcript, IIIF manuscript, LMS SCORM/xAPI, tenant BYOM bundle, and a structured-api-base, all behind a typed registry with a release-gate. Plus chunking/ (policy + strategies), enrichment/ (classifiers + pipeline), a lifecycle/ subsystem (cascade, retirement, scheduler, claim-diff), a pipeline/ with events, provenance/, and tenant/isolation. Real ingestion machinery, not placeholders.

A directed labeled property-graph engine (knowledge-graph/src), documented in detail in its barrel. GraphStore (CRUD, adjacency, filtering, events); GraphTraverser (BFS, DFS, Dijkstra shortest path, all-paths, subgraph, neighborhood); graph algorithms (degree/betweenness/closeness/PageRank/ eigenvector centrality, LPA + Louvain community detection, node similarity); ConceptGraph (prerequisite DAG with topological sort, cycle detection, critical path, level assignment); a gap-analyzer; learning-path generation (path-optimizer); a fluent query-builder; graph-aware retrieval; and an NLP relationship-mapper. Genuine algorithm implementations, not stubs.

Research-methodology frameworks and citation integrity (theory/src). CitationManager (~2.6K LOC: claim detection, multi-style citation formatting — chicago-notes and others — verification, and a graph provider) and ResearchManager (~1.8K LOC: projects, hypotheses, methodology). It integrates the @kalika/citations package via kalika-citation-storage.ts (InMemoryCitationDatabase + storage provider), so Sophia's citation store is backed by the shared Kalika citation engine.

Phase 85–86 flywheel producer for knowledge/research (libs/sophia/training-data/src): SophiaTrainingDataPipeline normalizes thirteen SophiaTrainingKind signals — search-relevance, citation-verification, rag-quality, chunking-quality, knowledge-graph-extraction, source-credibility, prediction-calibration, theory-evaluation, and peers — into SophiaTrainingRecords gated by an explicit SophiaGovernanceGrant before reaching the SophiaTrainingSink.

The claim-verification pipeline suite (verification/src). Independent pipelines for fact-checking (fact-checking-pipeline, plus a fact-check-loop ~1.2K LOC), contradiction handling (contradiction-loop + contradiction-pipeline), methodology, statistical, and temporal verification, and an unsupported-claim-loop (~1.1K LOC). Orchestration is real: job-envelope (SophiaGroundingJobEnvelope) and a publication-gate that runs concrete checks (claim-density, citation-integrity, contradiction/unsupported backlog, evaluation-thresholds, reviewer-signoff) and hashes evidence with SHA-256 (@noble/hashes) — a genuine block/pass gate, not a rubber stamp.

The web-acquisition layer (crawling/src). Five real subsystems: RateLimiter (token-bucket with domain tiers, configurable backoff strategies, polite vs aggressive presets), the crawl frontier (URL normalization, tracking-param stripping, a hand-rolled Bloom filter with optimal-size calculation, priority sorting), file detection (magic-byte signatures, format/category classification), compliance checking (~1.4K LOC: robots.txt directives, license/ToS, data-usage policy), and URL-pattern classification (including curriculum-seed source types). Domain-specific crawling logic, not a thin fetch wrapper.

Type-safe domain-event publishing for Sophia (event-publisher/src). A SophiaEventPublisher (singleton accessor + factory + reset for tests) emits the full Sophia event taxonomy as typed payloads: document ingested/updated/deleted, index updated/rebuilt, search performed, entity extracted, relation discovered, citation created. The payload types in types.ts are the contract other domains subscribe to on the bus.

Predictive analytics for streaming platforms (predictions/src). Three modules: revenue forecasting (forecast/revenue.ts, ~644 LOC — time-series with configurable confidence level, minimum data points, seasonality detection, peak periods, risk factors, recommendations), a schedule optimizer (~725 LOC), and a content-performance predictor (~1K LOC). Statistical forecasting methods over TimeSeries/ForecastResult types in types.ts. Note this is the most adjacent-to-streaming library in the area rather than core research tooling.

A larger, standalone citation-network library (citation-graph/src, ~6K LOC of implementation across five modules). CitationGraphBuilder constructs the graph (DOI parsing, author/title normalization, dedup, field/year filtering); NetworkAnalyzer computes degree/in/out, betweenness, closeness, PageRank (damping/convergence configurable), clustering coefficient, density, diameter, reciprocity, and connected components; CoCitationAnalyzer builds co-citation and coupling matrices and clusters by similarity; TemporalAnalyzer computes citation half-life, velocity, growth, bursts, dormant/emerging papers; and AnomalyDetector flags citation rings, spikes, self-citation excess, orphans, and reciprocal-citation patterns. The exported helpers are real graph algorithms, not wrappers.

Source-trust analysis (credibility/src). Five analyzers: source-credibility (authority levels, accuracy results, citation metrics, peer-review verification), quality-assessment (~1.2K LOC, multi-dimension quality scoring with issue severity/category), claim-extraction (typed claims from text), bias-detection (bias type/impact indicators), and contradiction-resolution (contradiction typing with resolution strategies and consensus levels). The types.ts surface defines the branded IDs and config types; implementations are real scoring logic.

Document parsing and normalization (document-parser/src). Seven modules: format-detection (extension + content heuristics, SupportedFormat), text-extraction (with language detection), structure-analysis (headers, sections), metadata-extraction (~885 LOC), citation-extraction (~881 LOC: typed citations, bibliography parsing in multiple styles, validation/issues), content-chunking, and a top-level document-processor that ties them into a DocumentProcessingResult. Real parsing logic with branded IDs from types.ts.

The educational knowledge base (knowledge-base/src). Five large modules: entity-extraction (~1.8K LOC), relationship-extraction (~1.1K LOC), standards-mapping (~1.4K LOC: curriculum standard alignment, coverage analysis), coherence-analysis (~1K LOC: prerequisite/progression analysis, coherence scoring/issues), and a knowledge-store (~952 LOC) with query results and statistics. Branded KBEntityId/KBRelationshipId/StandardId types anchor the domain in types.ts.

The end-to-end research orchestration engine (research-engine/src, ~7.5K LOC). Five large modules: query-planning (~1.6K LOC: decomposition, strategies), knowledge-extraction (~1.7K LOC: facts, entities, relationships), fact-verification (~1.3K LOC: verification results, consistency, contradictions), source-scoring (~1K LOC), and research-orchestration (~1.2K LOC) tying them into ResearchSession/ResearchResult. Overlaps conceptually with @sophia/agents but is the lower-level engine; both are real.

The educational semantic-search stack (semantic-search/src), the largest multi-module library here. BM25 sparse retrieval (configurable k1/b, field weights, stop-words, highlights), hierarchical and hybrid retrieval (with reciprocal-rank fusion), multi-factor result-ranking, a two-layer search-cache, search-config, a ~1.4K-LOC search-orchestration, structured and tool-mediated retrieval, retrieval-labels (provenance/source- scope enforcement), source-set-invalidation, and an iris-adapter for the Iris assistant. The educational-search module alone is ~2K LOC. Real IR implementations.

Curriculum-vs-trend analysis (trend-curriculum/src). TrendMapper (~1.6K LOC, mapping learning objectives to trend signals, surfacing unmapped trends and orphaned objectives), a CurriculumProfiler (field/Bloom distributions, coverage gaps, skew analysis), a RelevanceScorer (weighted relevance breakdown), an objective-trend analyzer, and a CurriculumRecommender (~1.7K LOC producing recommendation reports). Typed events and branded IDs in types.ts; real scoring and mapping logic.