# Aja — Systems Deep Dive

> The `apps/aja/` area: the deployable surface of Aja, the video-to-animation
> motion domain — a `commander` CLI, a Hono motion-AI service, and a Fastify
> pipeline-orchestration service that turn reference videos into exported
> skeletal-animation files.

## What this area is

Aja is Oshun's **motion / video-to-animation** domain. The heavy lifting —
motion file parsing/encoding, pose math, IK, retargeting, quality scoring —
lives in `libs/aja/*` packages (`@aja/motion-formats`,
`@aja/motion-pipeline-sdk`, `@aja/motion-processing`, `@aja/motion-quality`,
`@aja/skeleton-mapping`, `@aja/optimization-ik`, and the ML-oriented
`@aja/pose-lifting` / `@aja/human-mesh-recovery` / `@aja/neural-retargeting`
libraries). The `apps/aja/` directory is the thin **deployable layer** on top of
those libraries: the three registered Nx projects here are the things you
actually run — a command-line client, an HTTP motion-AI service, and a pipeline
orchestrator.

There are exactly three tracked Nx projects in this area (one `project.json`
each): `@aja/cli` (`apps/aja/cli`), `aja-svc-motion-ai`
(`apps/aja/svc-motion-ai`), and `@lilith/svc-motion-pipeline`
(`apps/aja/svc-motion-pipeline`). Two further directories sit alongside them but
are **not** Nx projects: `apps/aja/docs/` is Markdown reference material, and
`apps/aja/svc-reference-video/` carries a `package.json` and real `src/` but has
**no `project.json`**, so Nx does not register it and it is out of scope for
this catalog.

The three projects compose along the obvious axis. `@lilith/svc-motion-pipeline`
is the orchestration server: it owns jobs, pipeline configurations, progress
streaming, and result delivery. `@aja/cli` is the operator-facing client that
submits and tracks those jobs over HTTP through the `@aja/motion-pipeline-sdk`
client. `aja-svc-motion-ai` is a separate HTTP service that exposes the
pose/retargeting/analysis and "embodied-instruction" capabilities (the Metis
coaching handoff surface) directly. All three depend on the shared
`@aja/motion-formats` library so a clip parsed in one process serializes
identically in another.

## How it fits the wider system

`@aja/cli` is the human entry point: it talks to the pipeline service's REST API
(default `http://localhost:3000`, overridable via `--api-url` / `AJA_API_URL`)
and never touches motion data directly except to scale/inspect downloaded
results. `@lilith/svc-motion-pipeline` is the service the CLI (and any other
client) calls; it depends on the `@oshun/*` platform libraries (`@oshun/cache`,
`@oshun/database`, `@oshun/event-bus`, `@oshun/queue`) and on
`@lilith/fastify-core` for the server scaffold. `aja-svc-motion-ai` exposes the
motion-AI capabilities and is consumed by the Metis embodied-instruction flow —
its `@aja/domain-motion-pipelines` types (`AjaMetisEmbodiedInstructionDomain`)
and its `/api/v1/embodied-instruction/*` endpoints are the cross-domain boundary
those consumers integrate against.

The honest boundary to keep in mind: the deterministic, format/geometry side of
Aja is real and implemented here, but the **neural inference** side (running an
actual pose model on pixels) is deferred. The pipeline processor explicitly
skips the ML stages (`pose-estimation`, `skeleton-fitting`, `domain-analysis`,
`retargeting`) as V2 work, and the motion-AI estimators wrap real conversion
math around model adapters whose inference call is still a placeholder. Each
entity block below is explicit about where that line falls.

## Entity reference

### @aja/cli

The operator command-line client for the Aja motion pipeline (`apps/aja/cli`,
package `@aja/cli`). `src/index.ts` builds a `commander` program named `aja`
(version `0.1.0`) with global `--api-url` / `--api-key` / `--json` / `--quiet`
options and registers seven real subcommands from `src/commands/`: `process`,
`convert`, `inspect`, `debug`, `config`, `jobs`, and `health`. The `process`
command (`src/commands/process.ts`) is fully fleshed out — it resolves
glob/URL/directory inputs, runs an `inquirer` interactive wizard, submits jobs
through the `@aja/motion-pipeline-sdk` client, polls for completion with a
progress bar, and downloads result artifacts. It depends on the real motion
libraries `@aja/motion-pipeline-sdk`, `@aja/motion-formats`,
`@aja/motion-quality`, and `@aja/motion-validation`, and ships a unit test
(`src/commands/convert.test.ts`). This is a genuine, working CLI, not a
scaffold; it does assume the pipeline service is reachable at the configured API
URL.

### aja-svc-motion-ai

The motion-AI HTTP service (`apps/aja/svc-motion-ai`, package
`@aja/svc-motion-ai`; the Nx project name is `aja-svc-motion-ai`).
`src/server.ts` serves a Hono app (default port `3040`) defined in `src/app.ts`,
with an origin-allowlisted CORS policy (reads `AJA_CORS_ORIGINS`) and
Zod-validated routes for video-to-motion, retargeting, skeletal animation, video
analysis, and the `/api/v1/embodied-instruction/*`
capability/demonstration/coaching/handoff surface backed by
`EmbodiedInstructionService` (`src/embodied-instruction/service.ts`). A large
amount of the supporting math is real and domain-specific — temporal processing,
keypoint normalization to COCO-17/BODY-25/unified formats, quality metrics
(bone-length variance, joint-angle smoothness, jitter), ensemble
confidence-weighted fusion, physics refinement, and motion retargeting all live
under `src/video-to-motion/`, `src/motion-retargeting/`,
`src/motion-enhancement/`, and `src/skeletal-animation-motion/`. **The honest
gap:** the actual neural pose-inference is a placeholder — e.g.
`MediaPipePoseEstimator.runInference` in
`src/video-to-motion/pose-estimation-engine.ts` returns empty landmark arrays
with a `// Placeholder for actual MediaPipe inference` comment (the
`@mediapipe`/ONNX/TF.js call is not wired), and
`src/live-motion-capture/realtime-pose-estimation.ts` explicitly generates
simulated keypoints. So the orchestration, HTTP, and geometry layers are real;
the raw pixel-to-keypoint model step is not yet implemented in-process.

### @lilith/svc-motion-pipeline

The pipeline-orchestration service (`apps/aja/svc-motion-pipeline`; the Nx
project name is `@lilith/svc-motion-pipeline`, while its `package.json` name is
`@aja/svc-motion-pipeline`). `src/server.ts` boots a Fastify server (default
port `8090`) built by `buildServer` in `src/app.ts` on top of
`@lilith/fastify-core`, exposing a full `/v1/pipeline/*` REST surface: pipeline
configuration CRUD (including YAML import/export and quality presets), job
submit/ get/list/cancel/retry/priority/batch, output listing and signed-URL
download, a webhook-delivery subsystem, and Server-Sent-Events progress
streaming. Jobs are driven by a real processor: `setupJobProcessor` wires
`createPipelineProcessor` from `src/jobs/pipeline-processor.ts`, which uses
`@aja/motion-formats` (`importAnimation`/`exportAnimation`) to run ingestion,
validation, preprocessing, format-conversion, postprocessing,
quality-assessment, and delivery stages, writing byte-accurate files to disk
with SHA-256 checksums and computing genuine deterministic motion statistics
(root velocity mean/max/min/std, per-joint range of motion, jerk-based
smoothness/jitter quality scores). It is explicit and fail-loud about its
limits: the ML-only stages (`pose-estimation`, `skeleton-fitting`,
`domain-analysis`, `retargeting`) are marked `skipped` with a structured
`V1-P2-0331` deferral log rather than faked, the default input resolver rejects
`url`/`s3` sources instead of returning empty bytes, and the default storage
provider (`createMockStorageProvider`) is a clearly-labelled mock that callers
override with a real `StorageProvider`. It has a processor unit test
(`src/jobs/pipeline-processor.test.ts`).