Every monitoring project reaches the same moment. The devices are connected, the dashboards look good, and the first alarm rules go live. A week later the alarm list is hundreds of lines long, phones buzz all night, and operators have started to clear alarms without reading them. The system is working exactly as configured, and nobody trusts it.
Alarm fatigue is a design problem, not a people problem. This guide covers the habits that keep alarms meaningful.
Start from the action, not the measurement
The most useful test for any alarm is one question: what should the person who receives it do? If there is no clear answer, it is not an alarm. It may be a useful event to log, a trend to show on a dashboard, or a line in a weekly report, but it should not interrupt someone.
Write the expected action next to each alarm rule when you create it. For example: "Tank 3 level high: check the inlet valve and stop the transfer pump if level keeps rising." If you cannot write that sentence, reconsider the rule.
Use severity on purpose
Severity tells people how fast to respond. It only works if each level means something specific and the levels are used consistently across sites.
- Critical: act now; safety, environment or a major loss is at risk.
- Major: act within this shift; production or service is affected.
- Minor: plan a fix; something is degrading.
- Warning: watch; a value is drifting toward a limit.
- Indeterminate: the system cannot tell, for example because a sensor reports a fault.
Agree the definitions in writing before you build rules. If most of your alarms end up critical, the levels are not doing their job.
Stop alarms from chattering
A value that hovers around a limit raises and clears an alarm again and again. Three simple tools stop this.
Delay
Raise the alarm only when the condition has held for a set time. A pressure that spikes for two seconds during a pump start is not a fault; one that stays high for two minutes may be.
Deadband
Use different limits for raising and clearing. Raise at 85 °C, clear only below 80 °C. The gap stops the alarm flipping on small changes.
Suppression by state
Do not alarm on low flow when the pump is stopped on purpose, or on low temperature when a line is in cleaning. Make the rule check the state of the equipment as part of its condition.
Make acknowledgement mean something
Acknowledging an alarm should say: I have seen this and I am dealing with it. It should not make the alarm go away. An alarm stays active until the condition clears; acknowledgement only records who took ownership and when.
Ask for a short note on acknowledgement for critical and major alarms. Over time those notes become the most useful record of how problems were actually solved.
Escalate when nobody responds
An unacknowledged critical alarm at 3 a.m. should not wait for the morning. Escalation sends the alarm further when it is not acknowledged within a set time.
- First, the on-duty operator by push notification.
- After a set time, the shift lead by SMS.
- After a further set time, the site manager.
Keep the chain short and the times realistic. Test it with a real alarm before you rely on it.
Pick the channel by severity
Not every alarm deserves a phone call. Match the channel to the urgency: critical alarms by SMS or push, minor alarms by email or in the alarm list, warnings on the dashboard only. People keep notifications switched on when they trust that a notification matters.
Review alarms like you review production
Once a month, look at the alarm history and ask:
- Which ten rules raised the most alarms? Are their limits, delays or suppressions right?
- Which alarms were acknowledged without any action? Should they be events instead?
- Which problems happened without an alarm? Is a rule missing?
- How long did critical alarms wait for acknowledgement?
A small, trusted set of alarms protects a site better than a long list nobody reads.
How Hermetry supports this
In Hermetry, alarms carry a severity, move from open to acknowledged to resolved, and can escalate when nobody responds. Alerts go out by email, SMS or push. Alarm conditions are built with the same rule engine as the rest of the platform: a trigger, conditions and actions. Rules can run manually or automatically, and every run is kept in the history, so the monthly review starts from real data. Read more about alarms and rules.