Production-ready, single-node Hetzner Cloud deployment of the Bellona Remote Gateway data plane: a TLS edge (Caddy + ACME), the gateway service, a coturn TURN relay, PostgreSQL, MinIO object storage, Prometheus/Grafana monitoring, Loki/Promtail log aggregation, nightly Postgres backups, and a smoke test.
Honesty note. The artifacts in this directory ARE the deliverable. The live
docker compose upagainst a real Hetzner VM is account/credential-gated and is not run in CI or on the build box. Everything here is validated for syntax (docker compose config,bash -n) and internal consistency. The smoke test is the post-deploy validation hook; it fails loud (exit non-zero) when no deployed URL is provided and never fabricates a pass.
Topology#
internet
│ 80/443 (only public app ports)
▼
┌───────────┐ ACME / Let's Encrypt
│ caddy │── automatic HTTPS for $BELLONA_GATEWAY_DOMAIN
└─────┬──────┘
/health /ready /v1/* │ /v1/hosts/connect (WSS) /grafana /metrics-internal
▼ ▼
┌───────────┐ private "app" network (never published)
│ gateway │──┬─ postgres (persistent store)
│ :4070 │ ├─ minio (artifact object store)
└───────────┘ ├─ prometheus + grafana
└─ loki ◄─ promtail (all container logs)
coturn :3478/:5349 + relay 49160-49200 (TURN media relay, 180.C.25.04)
pg-backup cron → nightly pg_dump + MinIO mirror + checksum + retention
The gateway, Postgres, MinIO, Prometheus, Grafana, and Loki are bound to a private Docker network. Caddy is the only service publishing 80/443; coturn publishes its STUN/TURN/relay ports (host networking) for WebRTC media. Nothing else is reachable from the internet.
Files#
| File | Purpose |
|---|---|
docker-compose.prod.yml |
The full topology. |
.env.example |
Every required env var (copy → .env, fill secrets). |
caddy/Caddyfile |
TLS edge + reverse-proxy routes. |
coturn/turnserver.conf |
TURN relay (time-limited creds, TLS, hardened). |
monitoring/prometheus.yml |
Scrape config (gateway, coturn, loki, promtail). |
monitoring/loki-config.yml |
Log store (filesystem, 14-day retention). |
monitoring/promtail-config.yml |
Ships com.bellona.*-labelled container logs. |
monitoring/grafana/provisioning/ |
Prometheus + Loki datasources, dashboard provider. |
backup/pg-backup.sh |
pg_dump + SHA-256 + MinIO mirror + retention. |
backup/entrypoint.sh |
Installs the backup cron, runs a baseline dump. |
Env wiring#
The gateway env in docker-compose.prod.yml mirrors
apps/bellona/remote-gateway/src/config.ts exactly:
BELLONA_REMOTE_GATEWAY_HOST/PORT/LOG_LEVEL/STORE_FILE/HOST_TOKEN_SIGNING_SECRET/ HOST_TOKEN_TTL_MS/PAIRING_CODE_TTL_MS/APPROVAL_REQUEST_TTL_MS/ HOST_HEARTBEAT_INTERVAL_MS/HOST_HEARTBEAT_TIMEOUT_MS/SHUTDOWN_GRACE_MS.
The data-plane vars (BELLONA_DATABASE_URL, BELLONA_OBJECT_STORE_*,
BELLONA_TURN_*) point at the in-stack Postgres/MinIO/coturn.
HOST=0.0.0.0 is safe here because the gateway is only attached to the private
Docker networks; only Caddy is published.
Provisioning#
1. Hetzner VM + Cloud Firewall#
-
Create a
cx22(or larger) Ubuntu LTS VM in your closest region. -
Point your domain's A (and AAAA) records at the VM and set
BELLONA_GATEWAY_DOMAINto it. -
Hetzner Cloud Firewalls deny inbound by default. Open:
Direction Protocol Port Source Purpose Inbound TCP 22 operator CIDR SSH (remove after commissioning) Inbound TCP 80 0.0.0.0/0, ::/0 ACME HTTP-01 + redirect Inbound TCP 443 0.0.0.0/0, ::/0 HTTPS/WSS gateway edge Inbound UDP 3478 0.0.0.0/0 STUN/TURN Inbound TCP 3478 0.0.0.0/0 TURN over TCP Inbound TCP 5349 0.0.0.0/0 TURN over TLS Inbound UDP 49160-49200 0.0.0.0/0 TURN relay range Do not open 4070 (gateway), 5432 (Postgres), 9000/9001 (MinIO), 9090 (Prometheus), 3000 (Grafana), or 3100 (Loki).
2. Install Docker + Compose#
curl -fsSL https://get.docker.com | sh
docker compose version # confirm the compose plugin is present
3. Configure#
git clone <repo> /opt/oshun && cd /opt/oshun/infra/bellona-remote/hetzner
cp .env.example .env
# Fill EVERY secret. Generate strong values, e.g.:
# openssl rand -base64 48 # signing secret, db/minio/turn secrets
# caddy hash-password --plaintext '<pw>' # metrics basic-auth hash
$EDITOR .env
The compose file marks security-critical vars required (:?), so a missing
secret fails docker compose config instead of silently deploying a default.
4. Build + start#
docker compose -f docker-compose.prod.yml --env-file .env build gateway
docker compose -f docker-compose.prod.yml --env-file .env up -d
docker compose -f docker-compose.prod.yml --env-file .env ps
Caddy obtains the certificate on first request to :80. coturn's TLS listener
reuses that certificate from the shared caddy_data volume — after the first
issuance, confirm the cert/key paths in coturn/turnserver.conf resolve (they
follow Caddy's storage layout for acme-v02.api.letsencrypt.org-directory); if
your ACME CA differs, adjust the cert=/pkey= paths or symlink the leaf into
place, then docker compose restart coturn.
5. Smoke test#
BELLONA_SMOKE_BASE_URL=https://$BELLONA_GATEWAY_DOMAIN ../smoke/smoke-test.sh
Asserts /health → healthy, /ready → ready (with checks), the
/v1/pairing/exchange route rejects an invalid body (route + validator live),
and /v1/devices returns a device array. Without BELLONA_SMOKE_BASE_URL it
exits non-zero — it never reports a fake pass.
Monitoring & logs#
- Grafana:
https://$BELLONA_GATEWAY_DOMAIN/grafana(admin /GRAFANA_ADMIN_PASSWORD). Prometheus + Loki datasources are pre-provisioned. - Prometheus:
https://$BELLONA_GATEWAY_DOMAIN/metrics-internal/behind Caddy basic auth. - Logs: every Bellona container is labelled
com.bellona.service; Promtail ships them to Loki. Query in Grafana → Explore → Loki, e.g.{service="remote-gateway"} | json | level="error".
Operational logs are not the audit record — audit/timeline evidence stays owned by the gateway services and persistent store.
Backups#
pg-backup runs BACKUP_CRON (default 02:30 daily): a custom-format
pg_dump --compress=9 written to the backup_data volume with a SHA-256
checksum and JSON metadata, mirrored to the MinIO BELLONA_BACKUP_BUCKET, with
local dumps older than BACKUP_RETENTION_DAYS pruned. A baseline dump runs on
container start.
Restore:
# Verify integrity first.
sha256sum -c /path/to/bellona_bellona_<ts>.dump.sha256
PGPASSWORD=$POSTGRES_PASSWORD pg_restore --clean --if-exists \
-h <host> -U $POSTGRES_USER -d $POSTGRES_DB /path/to/bellona_bellona_<ts>.dump
The gateway's first-stage JSON store lives on the gateway_store volume;
snapshot it
(docker run --rm -v bellona-gateway-store:/v -v "$PWD":/out alpine tar czf /out/gateway-store.tgz -C /v .)
before destructive maintenance.
Limitations (honest)#
- The live
docker compose upagainst real Hetzner is account-gated and not run here; these files are validated for syntax + consistency only. - The gateway ships a first-stage JSON store today; Postgres/MinIO are provisioned and wired so the topology is complete and the connection strings are correct, ahead of the gateway's persistence/artifact code paths consuming them.
- coturn TLS reuses Caddy's ACME cert via a shared volume; the exact cert path depends on your ACME CA and may need a one-line adjustment after first issuance.
- A live WebRTC media-relay smoke (proving an actual
relaysession through coturn) requires real peers and is outside this script's scope; the metric classification + summary it feeds are unit-tested inlibs/bellona/remote-protocol/src/stream.test.ts.