Upgrade runners

Move a runner fleet to a new release. Start with one canary, verify the result, and keep the previous version ready for rollback.

To upgrade the workstation bridge instead, see Upgrade the MCP bridge.

Who upgrades what#

Four parts upgrade separately. Upgrading one never upgrades another.

Find the versions you run#

The Runners list in the console shows the version each runner last reported. On the host, run:

# one machine-readable line: "emisar version X.Y.Z"
$ emisar --version

# the same version plus Go, platform, and build metadata
$ emisar version

Compatibility status#

emisar compares each version a runner reports against two published numbers: the oldest release we still support, and the one we recommend you run.

• — outdated is above the minimum but behind the one we recommend. It keeps working. The console shows a version chip, and nothing else changes.
• — unsupported is below the minimum. The runner is flagged on the Runners page. It still connects today, but that is not a promise. Upgrade it.

The version is self-reported, so treat it as housekeeping and never as a security control. Your policies and pack trust decide what a runner may do, whatever version it claims.

Upgrade your runners and bridges in the same pass. For which versions work together, see compatibility and deprecation.

Plan the rollout#

1. Read the changelog for the releases between your current version and the target.
2. Write down the version you run today. That tag is your way back. Nothing in the product remembers it for you.
3. Check Runs for work in flight on the hosts you are about to touch. Filter to running and sent, and let anything long-lived finish first.
4. Pick one non-critical runner as the canary, ideally in a group whose actions you can run safely.

Upgrade one runner#

The runner updates to the latest stable immutable release by default. Pin the release on the canary so every later host gets the exact same release.

# on the canary host
$ sudo emisar update --version <target-version>

This command works only for a runner installed by install.sh, which records the paths it owns in a root-owned receipt. A copied binary, container image, or infrastructure-managed runner is refused. Update that deployment at its source instead of letting the host drift from it.

      # signed checksum metadata — produced for this tag by the trusted workflow
$ gh attestation verify SHA256SUMS --bundle SHA256SUMS.sigstore.jsonl \
  --repo andrewdryga/emisar \
  --signer-workflow AndrewDryga/emisar/.github/workflows/runner-release-trusted.yml \
  --source-ref refs/tags/runner-v<version> \
  --deny-self-hosted-runners
✓ Verification succeeded!
…
# archive bytes — match the authenticated checksum
$ awk -v file='emisar-<version>-linux-amd64.tar.gz' \
  '$2 == file { print; found=1 } END { exit !found }' SHA256SUMS \
  | sha256sum -c -
emisar-<version>-linux-amd64.tar.gz: OK
…
    

The update touches only the binary: config.yaml, runner.env, the cached token, installed packs, the dispatch log, and the local audit journal all stay as they are. Before the service stops, the installer authenticates the signed checksum metadata, verifies the archive against it, checks the staged binary's version, and runs that staged binary against your existing config.yaml. If the staged binary refuses the config, the update is refused here with the runner's own error. The installer exits nonzero with nothing touched, and the service keeps serving on the version it already had.

If any later step fails, the installer restores the previous binary, runner.env, and cached token, restarts the service if it was running before, and exits nonzero. Packs installed during the attempt stay installed.

Verify the canary#

Six checks, in order. Stop at the first one that fails.

# 1. the binary is the version you pinned
$ emisar --version

# 2. the service came back and stayed up
$ systemctl status emisar

# 3. config, credential, packs, action binaries, and reachability
$ sudo emisar doctor

# 4. the packs and hashes this host will advertise
$ sudo emisar pack list

Then verify the console. In Runners, confirm that the canary is online with the new version and expected actions. Run one low-risk read-only action. Confirm that it succeeds and appears in Audit. The audit row completes the check: doctor covers the host, and only a real run shows the whole path works.

Roll through the fleet#

Repeat the same two steps (upgrade, then the full verification) in bounded batches, one runner group at a time. Stop when any of these happens:

• — The update exits non-zero. If the swap had started, that host is already back on its previous binary. Read the output before you touch the next host.
• — emisar doctor reports a failed check, or the runner does not come back online within its usual reconnect window.
• — The advertised action set changed in a way the release notes did not describe, or a pack lands as pending trust in Packs.

Go to production covers fleet shape and batch sizing. Manage the runner fleet covers groups, labels, and host-side removal.

Return to a known-good release#

Going back is the same update command with the version you wrote down, followed by the same verification.

# runner — the release you were on before
$ sudo emisar update --version <known-good-version>

Rollback has two limits. Only a finished, published release installs; a draft or a release still being written is refused. The older binary must also be able to read the current dispatch state. If it cannot, the installer refuses and the current installation keeps running. Rollback works only within the supported release line, so go back no further than the minimum above.

An upgrade or rollback never touches packs. If the problem is an action's behavior rather than the runner, the version to change is the pack's. See Roll out and roll back packs.