Skip to content

punix fleet

Deploy and roll back a whole fleet of stacks across many hosts in one command (ADR-012).

Subcommands

punix fleet apply FILE [flags...]
punix fleet rollback FILE [flags...]

FILE is a Fleet PCL module: a module Fleet { hosts = [...] }. Each subcommand runs one per-host deploy (or rollback) per hosts entry, isolating failures. This is orchestration over the existing per-host pipeline (punix service); every host stays atomic and independently rollback-able. See Deploying a fleet.

The Fleet module

# fleet.pcl
module Fleet {
  hosts = [
    { file = "hosts/web.pcl"  stack = "WebStack"  target = "ssh://deploy@web.example.com"  machine = "web" }
    { file = "hosts/db.pcl"   stack = "DbStack"   target = "ssh://deploy@db.example.com"   machine = "db" }
  ]
}

Each host record carries:

  • file: a self-contained stack PCL file, resolved relative to the fleet file. An absolute path is rejected per-host (the host fails; the rest continue).
  • stack: the stack module name inside file.
  • target: the deploy target ssh://user@host[:port]. (Under --transport-root the target is mapped to a local subdir; see below.)
  • machine: optional; the host's identity in the cross-host artifact graph and the ADR-018 fleet data model. Only required for hosts that produce or consume a cross-host artifact, or that need the per-host isSelf flag (e.g. a wireguard mesh).

A fleet may also declare a typed cross-host data model (machines, globals, uniqueFields): the single source of truth for facts one host references about another (ADR-018). See examples/nlnet/fleet.pcl (a wireguard mesh) and examples/tangled/fleet.pcl (nine hosts).

punix fleet apply

Deploy every host in the Fleet module. Each host runs the full per-host pipeline (compose stack → realise closure → push closure over SSH → write config → atomic flip → optional activate). A failing host does not abort the others unless --fail-fast.

punix fleet apply fleet.pcl --key ~/.ssh/punix-deploy

Flags

  • --store-root PATH: store location (default ~/.punix/store).
  • --deployments-root PATH: where each host's gen-NNN.json lives (default ~/.punix/deployments).
  • --transport-root PATH: hermetic: deploy each host to <root>/<sanitized-target>/ via LocalTransport (for tests / local runs). Two host targets that sanitize to the same subdir are rejected up front (exit 2).
  • --dry-run: RealiseDryRun for every host; no real builds.
  • --key /path/to/private_key: SSH private key for ssh:// targets (default: ssh-agent).
  • --known-hosts /path/to/known_hosts: host-key pin file for ssh:// targets.
  • --fail-fast: stop launching new deploys at the first failing host (default: continue).
  • --tls-certs DIR: pre-supplied cert dir to provision TLS per host (ADR-013); same shape as service deploy. See TLS.
  • --vault-secrets FILE: JSON {ref: value} export resolving each host's {from_vault="REF"} secrets.
  • --activate: after each host's flip, daemon-reload + restart that host's units (ADR-017). Off ⇒ config only. See Activation.
  • --enable: also systemctl enable each unit for boot persistence (backlog J1). Independent of --activate.
  • --scenario NAME: evaluate every host under the named scenario.

There is no --target flag; each host names its own target in the Fleet module. --transport-root and the ssh:// targets are the two transport modes; they are not combined.

Deploy ordering

Hosts deploy in a stable topological order over the cross-host artifact graph (ADR-022): a host that produces an artifact deploys before any host that consumes it, with ties broken by fleet-file order (so identical input ⇒ identical order on every run). A fleet with no artifact edges deploys in fleet-file order, unchanged.

The graph is built before any host is touched: each host is composed (evaluation only; no effects) to read its producesArtifacts and the artifact refs it consumes. A consumed (producer, name) that the named producer does not declare is a located error (exit 1). A cycle among producers/consumers is refused pre-deploy (ARTIFACT-CYCLE); zero hosts deploy. A host that consumes an artifact must declare a machine.

Cross-host wiring is otherwise config-only: PCL has no imports, so each host is a separate evaluation and reaches a peer by address (or by the ADR-018 fleet model injected per host), never by a cross-module reference. The artifact graph carries only deploy-time digests (e.g. a cert's TLSA digest) by reference, ordered producer→consumer; the cert/key bytes never cross, and a consumer records the reference alone in its generation (§20.6). See examples/tangled-fleet/ and ADR-022.

Output

A per-host line, then a summary:

  ✓ WebStack → ssh://deploy@web.example.com
  ✗ DbStack → ssh://deploy@db.example.com: [E11] secret(s) not set: from_vault:db_pw
fleet: 1/2 host(s) deployed

Each host gets one of four markers, matching its status:

  • ✓: ok: the host deployed; its current flipped.
  • ✗: failed: the host's own deploy raised (its error follows, on stderr).
  • ⊘: blocked: a producer this host consumes an artifact from failed or was itself blocked, so this host was skipped without attempting (blocked: upstream <producer>:<artifact> unavailable, on stderr). Applied transitively.
  • ⊘: not-attempted: a --fail-fast run stopped launching before reaching this host (not attempted (fail-fast), on stderr).

Under --activate, a failed (re)start surfaces as a per-host ⚠ warning on stderr; the flip already succeeded, so the host is not marked failed. The summary fleet: N/M host(s) deployed counts only ok hosts.

Exit codes

  • 0: every host deployed (N == M).
  • 1: at least one host was not ok (failed / blocked / not-attempted), OR a fleet-level error before the per-host loop: a located [E#] in the fleet file, a malformed Fleet module / host record, a consume-without-produce or consumer-without-machine artifact error, an artifact cycle (ARTIFACT-CYCLE), or a bad --vault-secrets file.
  • 2: usage error: two --transport-root targets collide on the same subdir.

punix fleet rollback

Roll back every host in the Fleet to its previous generation. Each host's rollback is the same byte-exact config restore + atomic flip as service rollback (backlog H1): O(1), no rebuild.

punix fleet rollback fleet.pcl --key ~/.ssh/punix-deploy

Hosts roll back in reverse-topological order over the cross-host artifact edges: a consumer before its producer, which avoids the transient mid-rollback dangling window (notes/specs/022 §6 / ADR-028). The graph is rebuilt at rollback time (evaluation only, no effects). If the graph cannot be rebuilt (a host will not compose, a fleet-model fault, a cycle, an edge gap), the run falls back to reverse-declaration order and reports the reason on stderr:

note: using reverse-declaration order — artifact graph unavailable (<reason>);
consumer-before-producer ordering is not guaranteed

Each host's rollback is independent; the rollback order only has effect once the deferred re-resolve lands. A failing host does not abort the others unless --fail-fast.

Flags

rollback re-uses each host's already-pinned previous-gen manifest, so it takes no PCL evaluation flags. The flags are a subset of apply:

  • --store-root PATH: store location (default ~/.punix/store).
  • --deployments-root PATH: deployments root (default ~/.punix/deployments).
  • --transport-root PATH: hermetic: roll back each host under <root>/<sanitized-target>/.
  • --key /path/to/private_key: SSH private key for ssh:// targets.
  • --known-hosts /path/to/known_hosts: host-key pin file for ssh:// targets.
  • --fail-fast: stop at the first failing host (default: continue).

There is no --dry-run, --activate, --enable, --tls-certs, --vault-secrets, or --scenario on rollback. rollback still reads the fleet file, because that is where the host list and the artifact graph come from; it never evaluates a host's stack to decide what to write.

Output

  ↩ DbStack → gen-001 (was gen-002); 3 restored, 1 removed
  ↩ WebStack → gen-003 (was gen-004); 5 restored, 0 removed, 1 secret-skipped
fleet: 2/2 host(s) rolled back

Each ↩ line reports the target generation (and the prior one), the count of config files restored and removed, and, when non-zero, a secret-skipped count. A failed host is reported with ✗ STACK → TARGET: <error> on stderr. The summary fleet: N/M host(s) rolled back counts only successful hosts.

Exit codes

  • 0: every host rolled back (N == M).
  • 1: at least one host failed (e.g. [E12]: no previous generation, or a pinned store path missing), OR a malformed Fleet module.

Failure isolation

By default, a failing host is recorded and the run continues, with a per-host report; the command exits 1 if any host was not successful. --fail-fast stops at the first failure. Under apply, the four statuses (ok / failed / blocked / not-attempted) make it explicit which hosts ran, which broke, which were skipped for an upstream failure, and which were never reached.

Examples

# Local hermetic dry-run of the whole fleet (safe to run anywhere)
punix fleet apply fleet.pcl --dry-run --transport-root /tmp/fleet-out

# Real fleet deploy over SSH, activating each host's units
punix fleet apply fleet.pcl \
  --key ~/.ssh/punix-deploy \
  --vault-secrets secrets/vault.json \
  --activate --enable

# Roll the whole fleet back (reverse order)
punix fleet rollback fleet.pcl --key ~/.ssh/punix-deploy
  • Deploying a fleet: the Fleet module and orchestration in prose.
  • punix service: the per-host deploy / rollback this command drives.
  • Deploy over SSH · Activation · Secrets at deploy · Rollback.
  • examples/tangled-fleet/: a live two-host SSH deploy showing both cross-host links (a config address and a deploy-time artifact, ADR-022 D1).
  • notes/adrs/ADR-022-cross-host-effect-ordering.md: the cross-host artifact ordering rules.