Eve library

GaussCtrl on CPU: depth render and depth ControlNet (audit E.2)

Other

Revision 1

On this page

GaussCtrl edits a 3DGS scene by rendering depth from each view, conditioning a depth ControlNet on it, and fitting the edited views back into the scene. This directory is the measured CPU attempt at its first two steps. There is no GPU on the dev server.

  1. make_probe_scene.py writes a small 3DGS scene in the reference trainer's PLY format: 3,552 Gaussians forming a sphere on a floor.
  2. render-depth.ts reads that PLY with the library's parseGaussianPly and renders depth with renderGaussianDepth (libs/isis/3dgs-diffusion-editing/src/gaussian-depth-renderer.ts, EWA splatting and front-to-back compositing). It writes a PFM.
  3. controlnet_depth_cpu.py turns that depth into the inverse-depth image the depth ControlNet was trained on, and runs Stable Diffusion 1.5 with lllyasviel/control_v11f1p_sd15_depth on CPU. It writes the image, the conditioning image and a JSON record of the run.
bash
python3 -m venv --system-site-packages .venv-isis-controlnet   # system torch 2.11.0
.venv-isis-controlnet/bin/pip install "diffusers==0.35.2" "transformers==4.57.1" "accelerate==1.10.1"
cd tools/isis-gaussctrl-cpu
../../.venv-isis-controlnet/bin/python make_probe_scene.py /tmp/gc/scene.ply
npx tsx --tsconfig ../../tsconfig.json render-depth.ts /tmp/gc/scene.ply /tmp/gc/depth.pfm
../../.venv-isis-controlnet/bin/python controlnet_depth_cpu.py /tmp/gc/depth.pfm /tmp/gc/out --steps 20 --seed 7

Measured on the dev server (4 cores, AMD EPYC-Milan 2.0 GHz), 2026-09-21#

Depth. 3,552 Gaussians, of which 3,136 were splatted, rendered at 512×512 in 159 ms. Depth ran from 1.56 m to 8.71 m over 146,351 covered pixels, and every uncovered pixel is NaN. evidence/control-depth.png is the conditioning image: near is bright, far is dark, no surface is black.

Depth ControlNet. Stable Diffusion 1.5 with lllyasviel/control_v11f1p_sd15_depth, fp32, 20 steps at 512×512, 4 threads, torch 2.11.0, diffusers 0.35.2 and transformers 4.57.1. diffusers 0.35.2's safety checker does not load under transformers 5.x.

seed generate per step result
7 311.2 s 15.56 s the safety checker flagged it and returned a black image
11 314.0 s 15.70 s evidence/controlnet-seed11.png: the sphere and floor follow the rendered depth

The records are evidence/run-seed7.json and evidence/run-seed11.json. Model load from cache took 1 s; the first download was about 5 GB.

So the depth-conditioned ControlNet GaussCtrl needs runs on this CPU at about 5 minutes per view, before any multi-view attention or the refit of the splats. An edit over 40 views would spend over three hours in ControlNet alone. It works; it is not interactive.