Add beta update channel and harden in-place updates #2

Merged

aluminum-ice merged 1 commit from feat/beta-updates into main

2026-10-26 08:13:20 +00:00

aluminum-ice commented

2026-10-26 08:09:17 +00:00

Owner

What this PR does

Adds a beta update channel to the updates plugin, and hardens the in-place update path, following an adversarial review of it.

Beta channel

A release version must be X.Y.Z (stable) or X.Y.Z-beta.N (beta); the Release workflow refuses anything else. It publishes betas as Forgejo pre-releases and skips PyPI for them.
The plugin has a new beta option (a checkbox, off by default). Versions compare by SemVer rules, so 0.7.0-beta.1 < 0.7.0-beta.2 < 0.7.0.
Turning beta off never downgrades a device, and it cancels any beta offer already waiting.
A stable release's manifest and notes are now built against the previous stable release, picked by version number. Before this, a beta could hide a base-OS change from stable devices.

Hardening (numbers refer to the review findings)

#1: root no longer writes into the pi-owned /var/lib/kestrel/update/, where a planted symlink could redirect a root write.
- The "check now" trigger is created only fresh (O_EXCL|O_NOFOLLOW) and never chowned. Previously, chown pi:pi through a symlink handed any file on the device to pi.
- The engine's run status, lock, in-progress marker, install list and backups move to a new root-owned /var/lib/kestrel/update-run/.
#2: update apply takes host, repo and token only from config.toml, then re-verifies the release as root with a new update-apply --verify step. That step checks: published, not a draft, newer than the installed version, on the device's channel, an app-layer manifest that pins a commit, and no reflash release skipped in between. The pi-written state file is only a hint for which release to check.
#4 and #5:
- Every file is written atomically (temp file, then rename).
- The package is snapshotted and restored as pi.
- Files an update added are removed on rollback.
- After the restart, the daemon must stay up, or the whole update is rolled back.
- A new kestrel-update-recover.service rolls back an interrupted update at boot.
#6: an update lock, plus a fixed --unit=kestrel-update name, stops two updates running at once.
#9: the web page passes the confirmed tag through plugins-ctl as --expect TAG, so the update installs exactly what the operator confirmed.
#10: --ref goes through the same verification. --force overrides only the safety checks, and lists each one it overrode.
#11: the clone gets the token through git's GIT_CONFIG_* environment variables, so it's no longer in the process's arguments.
Plugin (#7, #8, #12–#15):
- A release list that can't be read, or that doesn't reach back to the installed version, no longer gets an in-place offer.
- The worker thread survives bad replies.
- Reply sizes are capped.
- A failed check is retried after 15 minutes instead of 6 hours.
- Both channels pick the newest release by version number.
- Links are https-only.
#16: shipped plugins and scripts that a release drops are pruned on update.

Release signing (#3) is deferred on purpose. The new docs/developer/deferred.md tracks it, along with the other items deferred here, and CLAUDE.md points to it.

Checklist

Lint passes — make lint (ansible --syntax-check, yamllint, shellcheck --severity=error). The Lint workflow runs this on the PR.
Tests pass — make test (the kestrel_pi pytest suite). The Test workflow runs this on the PR.
No secrets, no personal information. This is a public repository. The diff contains no real account passwords or hashes, Wi-Fi passphrases, private keys, API tokens, .env files, real names, personal emails, MAC addresses, device serials, or GPS coordinates — placeholders only.
Docs updated in the same PR, if this changes something an operator sees or configures: the README package table, the relevant page under docs/, and the on-device operator guide (GUIDE_MD in kestrel_pi/userguide.py) where it applies.
Commit messages are short and imperative.

Release impact

App-layer-safe — changes only files the in-place update engine carries: the kestrel_pi package, plugins/*.py, systemd units, the journald drop-in, a lib script already in the engine's LIB_SCRIPTS, or the CLI.
Reflash-class — touches a build input the engine does not carry onto a running device: apt or pip packages, boot options / config.txt / cmdline, kernel modules, the base image or partition size, builder/config.json, builder/config.yml, the Makefile, .forgejo/workflows/, a shipped config default (config.toml / default.conf), or a brand-new lib script.

This touches .forgejo/workflows/main.yml and builder/config.yml (the new root-owned run folder and enabling the recovery unit), so leave update_kind on auto. The engine, CLI, plugin, web page and new unit would themselves reach devices in place. A device updated in place also creates the run folder and enables the recovery unit itself.

Notes for review

Not tested on hardware, and no image was built. Coverage is the unit suite (795 passed, 7 skipped), lint, a scratch harness for the CLI's update section, and one read-only --verify run against the real release server. That run refused 1.0.0 with its own manifest's reflash reason, and refused an unpublished tag. Worth trying on a device before cutting a release: a real update, the health-check rollback, and a power cut during an update.
The first update onto this version runs the old CLI, which skips root verification and passes the old --state path. The new engine ignores that path, so even that run never writes root state into the pi folder. Every update after that uses the new path.
The engine and the plugin each implement version ordering and the skipped-reflash check. A test checks the two agree on version ordering. Merging them into one module is in deferred.md.
The daemon health check waits up to 90 seconds for the daemon to come up, then 20 seconds of steady running (HEALTH_TIMEOUT / HEALTH_SETTLE in update-apply.py).

🤖 Generated with Claude Code

## What this PR does Adds a **beta update channel** to the `updates` plugin, and **hardens the in-place update path**, following an adversarial review of it. **Beta channel** - A release version must be `X.Y.Z` (stable) or `X.Y.Z-beta.N` (beta); the Release workflow refuses anything else. It publishes betas as Forgejo pre-releases and skips PyPI for them. - The plugin has a new `beta` option (a checkbox, off by default). Versions compare by SemVer rules, so `0.7.0-beta.1` < `0.7.0-beta.2` < `0.7.0`. - Turning beta off never downgrades a device, and it cancels any beta offer already waiting. - A stable release's manifest and notes are now built against the previous *stable* release, picked by version number. Before this, a beta could hide a base-OS change from stable devices. **Hardening** (numbers refer to the review findings) - **#1:** root no longer writes into the `pi`-owned `/var/lib/kestrel/update/`, where a planted symlink could redirect a root write. - The "check now" trigger is created only fresh (`O_EXCL|O_NOFOLLOW`) and never chowned. Previously, `chown pi:pi` through a symlink handed any file on the device to `pi`. - The engine's run status, lock, in-progress marker, install list and backups move to a new root-owned `/var/lib/kestrel/update-run/`. - **#2:** `update apply` takes host, repo and token only from `config.toml`, then re-verifies the release as root with a new `update-apply --verify` step. That step checks: published, not a draft, newer than the installed version, on the device's channel, an app-layer manifest that pins a commit, and no reflash release skipped in between. The `pi`-written state file is only a hint for which release to check. - **#4 and #5:** - Every file is written atomically (temp file, then rename). - The package is snapshotted and restored as `pi`. - Files an update added are removed on rollback. - After the restart, the daemon must stay up, or the whole update is rolled back. - A new `kestrel-update-recover.service` rolls back an interrupted update at boot. - **#6:** an update lock, plus a fixed `--unit=kestrel-update` name, stops two updates running at once. - **#9:** the web page passes the confirmed tag through `plugins-ctl` as `--expect TAG`, so the update installs exactly what the operator confirmed. - **#10:** `--ref` goes through the same verification. `--force` overrides only the safety checks, and lists each one it overrode. - **#11:** the clone gets the token through git's `GIT_CONFIG_*` environment variables, so it's no longer in the process's arguments. - **Plugin (#7, #8, #12–#15):** - A release list that can't be read, or that doesn't reach back to the installed version, no longer gets an in-place offer. - The worker thread survives bad replies. - Reply sizes are capped. - A failed check is retried after 15 minutes instead of 6 hours. - Both channels pick the newest release by version number. - Links are https-only. - **#16:** shipped plugins and scripts that a release drops are pruned on update. Release signing (#3) is deferred on purpose. The new `docs/developer/deferred.md` tracks it, along with the other items deferred here, and CLAUDE.md points to it. ## Checklist - [x] **Lint passes** — `make lint` (ansible `--syntax-check`, `yamllint`, `shellcheck --severity=error`). The **Lint** workflow runs this on the PR. - [x] **Tests pass** — `make test` (the `kestrel_pi` pytest suite). The **Test** workflow runs this on the PR. - [x] **No secrets, no personal information.** This is a public repository. The diff contains no real account passwords or hashes, Wi-Fi passphrases, private keys, API tokens, `.env` files, real names, personal emails, MAC addresses, device serials, or GPS coordinates — placeholders only. - [x] **Docs updated in the same PR**, if this changes something an operator sees or configures: the README package table, the relevant page under `docs/`, and the on-device operator guide (`GUIDE_MD` in `kestrel_pi/userguide.py`) where it applies. - [x] Commit messages are short and imperative. ## Release impact - [ ] **App-layer-safe** — changes only files the in-place update engine carries: the `kestrel_pi` package, `plugins/*.py`, systemd units, the journald drop-in, a lib script already in the engine's `LIB_SCRIPTS`, or the CLI. - [x] **Reflash-class** — touches a build input the engine does not carry onto a running device: apt or pip packages, boot options / `config.txt` / `cmdline`, kernel modules, the base image or partition size, `builder/config.json`, `builder/config.yml`, the `Makefile`, `.forgejo/workflows/`, a shipped config default (`config.toml` / `default.conf`), or a brand-new lib script. This touches `.forgejo/workflows/main.yml` and `builder/config.yml` (the new root-owned run folder and enabling the recovery unit), so leave `update_kind` on `auto`. The engine, CLI, plugin, web page and new unit would themselves reach devices in place. A device updated in place also creates the run folder and enables the recovery unit itself. ## Notes for review - **Not tested on hardware, and no image was built.** Coverage is the unit suite (795 passed, 7 skipped), lint, a scratch harness for the CLI's update section, and one read-only `--verify` run against the real release server. That run refused 1.0.0 with its own manifest's reflash reason, and refused an unpublished tag. Worth trying on a device before cutting a release: a real update, the health-check rollback, and a power cut during an update. - **The first update onto this version runs the old CLI,** which skips root verification and passes the old `--state` path. The new engine ignores that path, so even that run never writes root state into the `pi` folder. Every update after that uses the new path. - **The engine and the plugin each implement version ordering and the skipped-reflash check.** A test checks the two agree on version ordering. Merging them into one module is in `deferred.md`. - **The daemon health check** waits up to 90 seconds for the daemon to come up, then 20 seconds of steady running (`HEALTH_TIMEOUT` / `HEALTH_SETTLE` in `update-apply.py`). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

aluminum-ice added 1 commit

2026-10-26 08:09:17 +00:00

Add beta update channel and harden in-place updates

Lint / lint (push) Successful in 30s

Details

Test / test (push) Successful in 30s

Details

Lint / lint (pull_request) Successful in 29s

Details

Test / test (pull_request) Successful in 30s

Details

8803e1f1ce

Releases tagged X.Y.Z-beta.N are published as pre-releases and offered
only to devices with the updates plugin's new beta option on; versions
compare by SemVer precedence. Stable releases' manifests and notes are
now built against the previous stable release.

Harden the update path: root re-verifies the release from root-owned
inputs only and never writes into the pi-owned state dir; the engine
installs atomically, snapshots the package, rolls back when the daemon
does not stay up, and recovers an interrupted update at boot. Also lock
updates, pass the confirmed tag through, fail closed on an unreadable
release list, keep the plugin worker alive on bad replies, cap reply
sizes, retry failed checks sooner, pick releases by version, prune
dropped files, and keep the token out of process arguments.

Track deferred improvements in docs/developer/deferred.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

aluminum-ice merged commit f1fd5e7c73 into main

2026-10-26 08:13:20 +00:00

aluminum-ice deleted branch feat/beta-updates

2026-10-26 08:13:20 +00:00

aluminum-ice referenced this pull request from a commit

2026-10-26 08:13:21 +00:00

Merge pull request 'Add beta update channel and harden in-place updates' (#2) from feat/beta-updates into main