2.2.2 adds staged firmware rollouts.
When you assign a package to every device of a profile or to a list of devices, you can turn on Send in waves. The package goes to a small first wave, Hermetry checks how it went, waits, and then goes on to bigger waves. Each wave has a success gate and a failure gate, and the rollout pauses by itself when a wave fails too many devices, closes with too few updated, or closes with none finished. Devices that already started finish their update; nobody else gets it until someone resumes.
The usual plan is 1 % (at least one device, at most 10) with a 30-minute wait, then 10 % with 30 minutes, 50 % with 15, then the rest. You can set your own sizes, gates and waits. While it runs, the package page shows the rollout live: each wave with its state, its progress and where it stands against its gate. See Rolling out firmware in waves and, for packages and assignments, Firmware updates (OTA).
Two smaller device changes in the same release: the device profile editor now sets the profile of new child devices, and devices whose profile says they don’t log in no longer get an access credential they would never use.
Does a device in wave 3 see anything before its wave starts? Our firmware checks the fw_* attributes on every boot.
Nothing. A device in a later wave gets no fw_* attributes until its wave starts, and its page says “Waits for wave n”. The order is stable for the rollout, so a device doesn’t move between waves.
Some of our pump controllers are on a cellular link that’s off for hours at a time. If they’re offline when their wave closes, do they count as failed and pause the whole thing?
No. Devices still offline or waiting when a wave closes are carried on and don’t count as failures. Online devices are put first, so the canary wave reports back quickly. Devices on metered links may show Waiting instead of updating, and they’re carried on the same way. The one case that does pause is a wave that closes with no device finished at all.
After an automatic pause, if I resume, do the failures that caused the pause still count against the gate?
Resuming accepts what happened so far: only new failures count against the gate, and a wave stopped at its end counts as passed. So look at the failed devices before you press it.
Wired ota.rollout.paused into our on-call webhook today. Wave pauses at 2 in the morning are exactly what I want to hear about.
- You must be logged in to reply to this topic.
Join the conversation
Sign in to ask and answer, like helpful posts and mark the reply that solved your problem. Every contribution earns points and levels.