Flows run unattended, so Hermetry guards them: a flow caught in a loop or failing over and over pauses itself, side effects are rate limited, and messages that could not be run are kept as failed messages for you to replay. This page explains each protection and what you do when one acts.
Flow protections
| Protection | Default | What happens |
|---|---|---|
| Runs a minute | 6,000 runs of one flow in a sliding minute | The flow pauses itself: a loop is suspected |
| Failing runs | 100 failed runs in a row | The flow pauses itself |
| Runs at once | 16 runs of one flow at the same time | Further runs wait (at most 1,000 per flow; the oldest are dropped beyond that) |
| Fan-out | 100 messages sent by one block for one input | The rest of that block’s messages for that input are dropped |
| Message size | 256 KiB per message | The message is refused (msg_too_big) |
| Loop guard | 4 hops of events and Run flow between flows; 512 steps in one run | The chain is cut |
| Retries used up | At most 5 attempts of an external call | The input becomes a failed message unless the flow catches the error |
| Repeated messages | A device message seen again within 60 seconds | Acknowledged without a run |
| Too old | A message waiting longer than 5 minutes | Not run (device messages still run in their root, marked late) |
| Device commands a minute | 30 commands and settings from flows to one device | The command is refused (limit) |
| Repeated notifications | The same notification to the same person within 10 minutes | That person is left out |
Your plan may set other values within fixed bounds. Every time a protection acts it is counted: on the flow’s row (Limited · protection), in the editor (“A flow protection acted on this flow: …”, with the blocks marked), and in the usage card (Flow protections this month). See Flow limits.
A flow that paused itself
A paused flow starts no run until it is resumed. Its row says why:
- “It paused itself: it ran more often in a minute than this workspace allows, so a loop is suspected.”
- “It paused itself: too many of its runs failed in a row.”
The managers of automation get a Flow paused itself alert. What to do:
- Open the flow and look at Activity > Errors and the Runs of the last minutes.
- Fix the cause: break the loop (a flow that triggers itself, a Status trigger watching a block that changes its status on every message), or fix the failing call.
- Deploy, then Resume from the flow’s row menu or the editor’s banner. A pause keeps the flow’s memory and waiting messages.
Roots are never paused automatically
A root flow stores the data of every device it serves, so it is exempt from the automatic pause and from Runs at once. If a root fails too often, the organisation’s owners and admins get the critical alert Root flow not paused (“This root flow was not paused, because it stores the data of every device it serves…”), at most once every 6 hours per flow. Fix it quickly: failing runs of a root may mean readings are not stored.
The emergency stop
The emergency stop pauses every running flow of the organisation, on every host, at once, roots included.
- In Automation, open the header menu and select Stop every flow.
- Read: “Every running flow of this workspace, on every host, is paused at once: device messages, schedules, webhooks and events start no run until you resume. Runs in progress finish. A root flow is paused too, so device data is not stored meanwhile.”
- Tick “I understand that nothing runs until the flows are resumed.”, type
STOPand confirm.
A banner shows “Emergency stop: n flows are paused…”. To end it, select Resume the flows: every flow the stop paused runs again; a flow that had paused itself stays paused. Both actions are audited.
Use it only when flows are doing harm (sending wrong commands, flooding a partner system). Device data is not stored while it is on.
Failed messages
A message the platform could not run is kept as a failed message (a dead letter). Open them from the flow list’s header menu (Failed messages, all flows of a host) or in the editor (Failed messages, the messages addressed to this flow).
| Reason | Meaning |
|---|---|
| Kept failing | Delivered several times without finishing |
| A call failed every retry | An external call used up its retries and the flow did not catch the error |
| Waited too long | A webhook call or a named timer waited longer than its time-to-live (after an outage). “Replay it to run it now.” |
| Left unfinished by a redeploy | Its run did not end before the version it ran on was replaced |
| Backlog too long | Too many messages of the organisation were waiting |
| Could not be read | The stored input is damaged |
| Refused by the host | The host did not accept it |
Each row shows First seen, Reason, Error and Tries. View shows the stored input (to people who may debug).
- Replay puts the message back on the queue with its original ids. “Its run may already have done part of its work: calls, messages or writes it made can happen again.” Blocks that call out send the same idempotency key, so a receiver that deduplicates sees one call.
- Discard deletes it. It cannot be replayed afterwards.
Each host keeps its own failed messages. At most 10,000 are kept by default; beyond that the oldest are trimmed. At 80 % the managers get the alert Failed messages nearly full, at most once every 6 hours.
A late device message in a root is never a failed message: it is stored. A late schedule tick is simply skipped.
Alerts
Errors and alerts (flow list header menu) shows the automation alerts of the last 6 hours, a row per deployed flow with its errors in 24 hours, failed messages today and recording level, the most frequent errors of 7 days and totals by day.
The Alert rule decides who is told and when:
| Field | Meaning |
|---|---|
| Send alerts | On or off |
| Errors per minute | 1 to 10,000 |
| Minutes in a row | 1 to 19 |
| Flows | None ticked = all flows |
| Events | None ticked = all alert kinds |
| Recipients | One member user id per line, at most 100. Empty: the people who may manage automation. E-mail addresses are not accepted. |
Alert kinds: Many errors, Calls paused (breaker open), Side effect throttled, Block does not start, Failed messages nearly full, Flow paused itself and Root flow not paused. Root flow not paused is always sent, whatever the rule says, to the owners and admins and the rule’s recipients.
Calls paused (breaker)
A block whose external calls keep failing (an HTTP endpoint that is down) pauses its calls for a while; its messages fail with circuit_open meanwhile. The flow’s events show Calls paused (breaker open) and later Calls resumed (breaker closed). Fix the receiver; the block recovers by itself.