Bit of a weird one. We’re rolling out vibration sensors at our UK plant. Our sister plant in the US already runs the same sensors on Hermetry, so their controls guy sent me his firmware project and I just changed the tokens to the ones from our devices.
Every single device fails:
[mqtt] connecting to mqtt.<their host>:8883 ...
[mqtt] TLS ok, sending CONNECT
[mqtt] CONNACK rc=4 (bad user name or password)
[mqtt] backing off 300 sWhat I’ve checked:
- tokens copied straight from the Create device drawer, triple-checked
- username = token, password empty, as per the docs
- the device Log tab in our org shows nothing at all, no refused logins
- port 8883 open outbound, TLS handshake clearly succeeds
Our Hermetry organisation was created by our IT in January. Theirs is older. Is there some per-organisation thing I’m missing? I’d rather not call their controls guy again, he’s already spent an afternoon on this.
CONNACK 4 with nothing in your own Log tab is a big hint. Your org never saw those logins.
Maybe the tokens got rotated? We had that once after someone hit Rotate by mistake.
Tokens are brand new, nobody rotated anything. Tyler, what do you mean never saw them?
Look at the host in your log line. That’s the broker host from the US firmware.
Hermetry runs as separate deployments per region (EU, US, TR) and every organisation lives in one home region. Devices have to connect to the device host of their own organisation’s region, the one on each device’s Connection details card. I’d bet the US plant’s organisation is in the US region and yours, created by your IT from the UK, is in the EU. Your UK devices are knocking on the US broker with tokens that only exist in your organisation’s region. To that broker they’re unknown tokens, so you get a 4. Nothing reaches your Log tab because your org was never asked.
I hit exactly this when we cloned a project between two of our utilities, one in each region. Fix:
- Open any of your devices, Credentials tab, Connection details. Note the MQTT broker host.
- Replace the host in the firmware (and the HTTPS host if they use it).
- Keep connecting by host name. SNI is required, so don’t hardcode an IP from a DNS lookup.
Owners and admins can also see the home region under the organisation settings, Residency. The page is Regions and data residency.
And since you’re getting 4s: the firmware backs off 5 minutes, good. Keep that. Repeated failed logins from one address block that address for a while, and the whole plant is behind one NAT.
Good catch. Worth putting the host in a config file instead of the build, so the next plant doesn’t copy a hardcoded host.
Also check what CA bundle the US firmware pins. If they pinned their region’s server cert instead of using a normal CA store, it may fail on the EU host for a different reason.
Ha, we had the same with two hotels, two orgs in two regions, same firmware.
That was it. Residency page says EU, the US plant’s org is US. Changed the host to the one on our Connection details card and all 23 sensors were online within ten minutes. Lewis, you were right as well, the firmware had their server certificate pinned. I switched it to the normal CA bundle, otherwise the handshake would have been the next failure. Host now comes from a config partition. Thanks all.
- You must be logged in to reply to this topic.
Join the conversation
Sign in to ask and answer, like helpful posts and mark the reply that solved your problem. Every contribution earns points and levels.