Roost

A self-hosted app platform on one small box. One ARM SBC (or any Debian/Ubuntu machine), Dokku, a Cloudflare tunnel, and a toolbelt of scripts — serving any number of apps at <name>.yourdomain, deployed by git push, behind CGNAT with no public IP and no port forwarding.

Roost also runs the box after the apps are up: status boards that publish what the fleet is doing, fleet observability through pulse, backups and restore for the whole stack, and disk reclaim. A roost is where things come home to rest, so keeping them safe is part of the job.

Internet ─▶ Cloudflare ─▶ tunnel (dials OUT) ─▶ nginx :80 ─▶ Dokku app containers

New app, one command, ~40 seconds to a live URL:

roost new myapp --static          # or --node, --swift (Hummingbird 2), --board (statusgen); hatchery makes it

The roost command

Everything is driven by one dispatcher, bin/roost (add bin/ to your PATH). roost help prints this list; roost doctor diagnoses the setup when anything misbehaves.

Command	What
roost new <name> [--static|--node|--swift|--board]	nothing → live, declared app: runs hatchery new <stack>/<name> --kind <kind> --yes and prints the line
roost route <subdomain>	routes nothing: prints hatchery route <stack>/<service> --yes, which publishes the tunnel route of each name a service declares (house#155)
roost lan-cert <name> <user@host> <dir> [label]	issue or renew a LAN name's certificate through DNS-01 in the certbot container, and place it on the host that serves it; bin/install-lan-cert.sh runs it daily
roost status ["message"]	collect + validate + deploy the status site (no message: narrative auto-composed from merged PRs)
roost stats	run the configured board-stat collectors
roost tapo [--json|--watch|--discover]	read the Tapo smart plugs over their local API, write the cache node-report reads, and post the readings to pulse /api/tapo
roost ha-scoop [--days N|--json|--entities]	copy the hourly power statistics of Home Assistant into pulse, to fill the gaps in its sampled history
roost forge-log <owner>/<repo> <run> [job]	read the log of a forge job from the SQLite database of the forge over ssh, read-only. --task <id> reads one task
roost kick	fire the status runner's hourly deploy now
roost rollout [--kick]	ff-only pull roost + statusgen on every writer machine after a merge
roost apps / ps [app] / logs <app> [-n N] / restart <app> / config <app>	day-2 Dokku reads over ssh. Config writes moved to hatchery config set, which keeps the declaration true; --force keeps the old direct write for emergencies
roost prune [project] [--yes] [--deep] [--caches]	reclaim build artifacts (dry-run by default)
`roost backup [--json	--run
roost secrets	say where this host reads each secret from: vault, a legacy file, or nowhere
roost doctor	diagnose ssh, token, zone, and tooling
roost ui	full-screen terminal: console, monitor, config, docs, and backups tabs

Configuration lives in ~/.roostrc (roostrc.example). The pulse NODE_KEY and the ci-live CI_KEY come from vault, described in the next section. Every other secret is still a chmod-600 dotfile (~/.cf_api_token, ~/.ha_token, ~/.tapo_pass), never in the rc file and never in the repo.

The Swift roost

The repository also holds a Swift package with one product, roost (house#133). It carries four commands. version prints the commit it was built from, and roost --version prints the same line. watch is the box watch from bin/box-watch.py (house#134), with the same environment keys, files, view and alerts. The shell bin/roost keeps every verb in the table above. bin/box-watch.py and tests/test_box_watch.py stay until the opi and the mini have run roost watch for a week, and then they are removed. The declared job box-watch on each host changes its program to ~/.local/bin/roost watch when hatchery places the binary on that host. The watch says why a check fails (house#143). When a check reads down, or a service on the box fails, it asks the checks of that service's declared needs, and a need that does not answer is named once: "rookery degraded: it waits for vault (sign-in, the skill registry and the forge credentials)". One alert goes out for that need, titled Need down: <need>, and none for each check above it. The view's failing check or service carries because with the need's name, and the key is absent otherwise. A need is asked by its own check on the box, its own service on the box, or a GET at its first domain and healthPath, where expectedStatus also reads as an answer. A declared job whose needs name the rookery office is a runner of the office, and it is read by its last check-in on the office's GET /api/runners, with ROOST_ROOKERY_URL and ROOST_ROOKERY_TOKEN from ~/.roostrc. No check-in for BOX_WATCH_CHECKIN_LIMIT seconds, 600 by default, reads failing with "no check-in since" and the time. A box with no token reads the runner by its process, as before. The watch only reads and says: every command it runs is a supervisor or docker read or the mesh line, and every request is a GET or a post of an alert or the view. beat runs one scheduled job and posts its beat to pulse (house#149): roost beat <stack> <job> --pulse <url> -- <program> [args...]. It runs the program as a child with the same environment and standard streams, passes SIGTERM and SIGINT on to it, and exits with its code, or 128 plus the signal that killed it. After the run it posts {stack, job, host, code, durationMs} to <url>/api/beats with the key from ~/.roost_node_key, and gives up after two seconds with no bytes and five in all. No key file or an empty key posts nothing, and no answer from pulse changes nothing about the run. hatchery writes it in front of each scheduled job on a host stack that declares fetched: roost, and keeps its shell line on a stack that does not. report is the node report from bin/node-report.sh (house#150). It reads the same ~/.roostrc keys, the same node key and the same caches, and posts the same body to <pulse>/api/nodes, so pulse and the watts page need no change. roost report --print writes the body to standard output and posts nothing. bin/node-report.sh and bin/install-node-report.sh stay for a week beside the command, and then they go with their lint lines. The declared job roost-node-report on each host changes its program to the resident's report: /Users/jimmyhoughjr/.local/bin/roost report on the air and the mini, and /home/jimmy/.local/bin/roost report on the opi. swift test runs the vote's tests ported from the Python one for one, two watchers on loopback that see each other until one stops, the beat against a loopback pulse, and the report's parsers and readers against output captured on the air, the mini and the opi in tests/roostTests/Fixtures.

The forge builds it in .forgejo/workflows/build.yml. The Linux job runs on the opi runner in the roost-swift-ci:6.3.2-noble container, links the Swift standard library in statically, and strips the binary. On main it publishes roost-linux-arm64 and roost-linux-arm64.sha256 to https://forgejo.jimmyhoughjr.net/api/packages/jimmy/generic/roost/<sha>/, where <sha> is the full commit. The macOS job builds roost-macos-arm64 on the mini and signs it again after the strip, but it is off while the mini is held. bin/stamp-build writes the commit into Sources/roost/BuildStamp.swift before the build, and bin/publish-build does the upload with FORGE_PACKAGE_TOKEN. Do not commit a stamped BuildStamp.swift. Run GITHUB_SHA=none bin/stamp-build to put unknown back.

hatchery install places the published file at ~/.local/bin/roost on a host that declares fetched: roost (house#132). That path is not the shell tool: a host that puts bin/ on its PATH ahead of ~/.local/bin still runs the shell roost. No host declares it yet.

Secrets from vault

A host used to hold one file per secret, placed there by a person. Rotating the pulse NODE_KEY meant visiting every node and getting every one of them right. A host now holds one secret of its own, its vault app key, and fetches the rest from vault's sealed per-app document.

lib/roost-secret.sh and lib/roost_secret.py are the one reader, and both resolve a name in the same three arms:

1. the vault document for ROOST_VAULT_APP, fetched with ROOST_VAULT_APP_KEY
2. the legacy file for that name: ~/.roost_node_key for NODE_KEY, and ~/.roost_ci_key for CI_KEY
3. nothing, and the caller says so rather than sending an empty key

Three keys in ~/.roostrc turn the first arm on:

Key	What
ROOST_VAULT_URL	vault base URL (default https://vault.jimmyhoughjr.net)
ROOST_VAULT_APP	the app whose document holds this host's secrets
ROOST_VAULT_APP_KEY	the app key that opens it, the one secret the host still holds

The app is one per host, named roost-<ROOST_NODE_NAME>, so roost-jimmys-mac-mini and roost-opi. A node key belongs to one host, and revoking one host's key must not touch the others. The names inside the document are NODE_KEY and CI_KEY.

The app key travels in a curl header file and never on a command line. The document is cached for 300 seconds at mode 600 under TMPDIR, shared by both languages, because node-report runs every 30 seconds and a read per run would be ten vault calls a minute from every host.

The cutover runs one host at a time, because the legacy file keeps answering while it exists. Put the three keys in that host's ~/.roostrc, then run:

roost secrets

It prints one line per name and never a value. Delete the legacy file on that host once both lines say vault.

What's here

Path	What
docs/getting-started.md	Prerequisites → first deploy: hardware, accounts, installs, tunnel
docs/tutorial.md	Guided tour: deploy an app, board it, operate it — and which of the three repos to touch
docs/playbook.md	The operating manual: storage, crons, secrets, accounts, status boards, fleet observability, backups and restore, disk reclaim, and every gotcha learned the hard way
homeauto	Smart plugs, bulbs and Home Assistant: wiring, credential locations, rebuild-from-nothing plan (separate repo)
docs/status-events.md	Design sketch: push-based CI to a central ingest, then boards and history. pulse has the ingest route /api/events, and the CI side is not built
docs/opi-backup-restore.md	The restore order for the nightly opi backup
bin/opi-restore-proof.sh	Once a week, restores one file from the newest snapshot of each repository into a scratch folder, checks its bytes, and tells pulse, so the backups page says when a restore was last proven (house#88). It also restores the forge's database copy from the critical repository and opens it with no repair (house#127)
docs/mesh-alert-setup.md	The LoRa alert path: what mesh-alert.sh needs on the opi before it can send
docs/roost-and-hatchery.md	How roost and hatchery divide the estate
ci/README.md	The forge and its CI: mirrored actions, the runner config, and the job images in ci/images with how to build one again
bin/roost	The dispatcher — every command above
lib/roost-secret.sh / lib/roost_secret.py	The one secret reader, shell and Python: vault first, the legacy file after it, and one cached document per host
bin/new-app.sh	Retired on 2026-10-07: it runs roost new, which hands the whole job to hatchery new (house#83)
bin/publish-route.sh	Retired on 2026-10-23: roost route prints the hatchery route line, and hatchery makes the Cloudflare calls (house#155)
bin/status.sh	The roost status orchestrator: self-update, collect, validate, deploy
bin/node-report.sh	Per-node telemetry (load/mem/disk/watts/battery/runner/temps, and the Swift and Xcode of a host that runs a forge runner) → pulse /api/nodes, macOS + Linux; launchd/systemd installer alongside
bin/ci-live-report.sh	Live CI-run poller (runs on the CI Mac) → the ci-live app; launchd installer alongside
bin/gen-narrative.py	Composes the board narrative from merged PRs when roost status gets no message
bin/roost-prune.py	Disk reclaim (dry-run default)
bin/opi-backup.sh	Nightly disaster-recovery backup of the opi stack; runs on the opi, writes restic repositories to the drive on the mini. It dumps each PostgreSQL cluster and copies the forge's SQLite database whole before the tars (house#127), and it refuses to run while ~/.config/opi-backup/hold exists
bin/install-opi-backup.sh	The first installer a blank opi runs (house#120): places the backup script's copy at ~/opt/opi-backup, the pinned restic at ~/bin/restic, the repository password from the mini's copy, and a hold on a box that has never finished a backup. hatchery install runs it for the copy's row
bin/install-venv.py	Makes each Python venv a box needs from its pinned list in venvs/ (house#122): the Meshtastic CLI at ~/opt/meshtastic, python-kasa at ~/.roost-tapo-venv, and the router reader's at ~/.router-venv. The box's own python3 makes the venv, and one pinned pip wheel installs wheels only, with every hash checked and no cache. A venv that holds its pins is left alone, --facts prints what each holds against its list, and --remake makes again one that differs. hatchery install runs it for the venvs a host stack names.
bin/emu-pull.sh	Nightly pull of the ROM library from the mini to the opi's NVMe, the third copy; a declared job runs its copy at ~/opt/emu-pull at 04:30
bin/box-sweep.sh	Runs on the opi weekly (box-sweep in the box stack). Removes a repository's Swift cache and a runner's repository checkout under _work when no file inside was touched in fourteen days; the runners' own _actions, _tool and _temp stay. Without --yes it lists and removes nothing (house#124)
bin/backup-ui.html	The page roost backup --serve presents: status, snapshot browser, extract and restore
bin/backup-status.py	Reads that service from any machine: the schedule and last run on the service host, the repositories on the storage host
bin/backup-report.sh	Pushes that reading to pulse /api/backups for the dashboard card; hourly launchd/systemd installer alongside
bin/rollout.sh	roost rollout. Runs on the laptop, and pulls roost and statusgen ff-only there and on every ROOST_WRITERS machine
bin/forge-log	roost forge-log. Runs on any machine with ssh to the opi, and reads job logs from the forge database or its log store, read-only
bin/tapo-poll.py	roost tapo. Reads the Tapo plugs directly or through Home Assistant, writes ~/.roost-tapo.json for node-report, and posts to pulse /api/tapo. install-tapo-poll.sh makes it a long-running service (launchd or systemd user unit) and makes its python-kasa venv
bin/ha-scoop.py	roost ha-scoop. Copies the hourly power statistics of Home Assistant into pulse. install-ha-scoop.sh runs it at 10 minutes past each hour (launchd or systemd user timer)
bin/volts.py	Prints mains voltage and load from pulse /api/history as terminal sparklines, to match a voltage dip to its cause. Runs on any machine that reaches pulse
bin/dokku-reconcile.sh	Runs on the opi every 10 minutes (install-dokku-reconcile.sh, systemd user timer). Starts deployed apps that are down, names apps with no image, rebuilds the vhosts, asks each declared app its health path, names a forge job image the box lost, and posts its readings to pulse /api/answers and its events to pulse /api/events Repairs three proven faults by a restart, once each, and says so as an event: sites answering 502 while their containers are up, a forge that takes no runs, and a house DNS that answers nobody on the LAN address. A fault that holds after its one try waits for a person, and a run from a terminal gives every fault a fresh try (house#78).
bin/job-images.py	The forge job images a build box holds: state, the Swift in each, created, size, and what wants it. check names each wanted image the box does not hold and exits 1. It reads only, and the reconcile posts its rows to pulse
bin/job-images-registry.py	push <image> sends a forge job image to the forge's registry, and restore <image> pulls a lost one back under its local name. A person's step, never a timer. --dry-run prints each docker command and the bytes it writes
bin/mesh-alert.sh	Sends one line over LoRa through Meshtastic to MESH_DEST, when the network cannot carry an alert. It runs on the opi, and the reconcile alert calls it. docs/mesh-alert-setup.md has the setup
bin/backup-watch.sh	Runs on the opi at 08:00 (install-backup-watch.sh, systemd user timer). Sends an ntfy alert when the nightly backup did not write its finish stamp
bin/runner-watchdog.sh	Runs on the opi every 15 minutes (install-runner-watchdog.sh, systemd user timer). Watches the self-hosted GitHub runners through the GitHub API, and sends an ntfy alert for a runner offline too long or a run queued too long
bin/colima-ensure.sh	Runs on a Mac that uses colima, every 5 minutes (install-colima-ensure.sh, launchd). Starts the colima VM when it is not running, so the Mac keeps its amd64 capability through Rosetta
bin/lan-cert.sh	roost lan-cert. Runs on a Linux box with docker (the opi), and issues or renews a LAN name's certificate through DNS-01. install-lan-cert.sh runs it daily
bin/install-macmon.sh	Installs macmon on this Mac or on another Mac over ssh, so node-report sends measured watts and temperatures. A copied binary is signed again
bin/move-containerd-to-nvme.sh	A one-time step on the opi, run with sudo. Copies the containerd store to /mnt/nvme/containerd and bind-mounts it on /var/lib/containerd. Every service stops for the copy
ci/	The forge runner config, the swift-ci composite action, the job image recipes in ci/images, and an S3 conformance probe. ci/README.md describes them
bin/roost-ui.py	roost ui — full-screen terminal in five tabs: console (prompt + streaming commands), monitor (live fleet via pulse), config, docs pager, backups (status, snapshot browser, extract and restore) (stdlib only)

Each script carries its own usage/config header — the headers are the authoritative per-tool reference. Tests: python3 -m unittest discover -s tests.

Rendered docs: docs.jimmyhoughjr.net

The reference roost

What this pattern runs in production, on one 8-core / 16 GB Orange Pi:

• watts — electric cost calculator (EIA rates cron, seasonal modeling)
• vault — Apple/Google sign-in + per-app user storage (Swift/Hummingbird 2)
• head2head — measured implementation shootouts (Node vs Swift, bout 1)
• status — statusgen boards with git-generated history
• pulse — live fleet dashboard and system map, and the Backups card
• docs — this repo's docs plus living usage reports
• a blog, and a hello created by new-app.sh as its living test

Companion projects

• statusgen — data-driven status boards (and the bare-metal SETUP.md for the locally-managed-tunnel variant)

Built by Jimmy Hough Jr & Claude. Donations appreciated: $jimmyhoughjr