HotLoop core 4.18.11 release notes

HotLoop’s database is PostgreSQL 17 from this release, and every install that has PostgreSQL 16 data moves on its next upgrade without you running anything. That is the one thing in 4.18.11 you need to read before you touch a running plant, because it is a change of database engine, it needs disk, and there is one way back that is not a plain rollback.

The full notes are in the README: What changed in 4.18.11.

PostgreSQL 17, and the upgrade that comes with it

What changed. The database the charts and the Quadlet units run is PostgreSQL 17: TimescaleDB 2.30.1 on it for Edge and the Gateway, plain PostgreSQL 17.11 for IoT and for database.type: postgresql. SQLite installs have no PostgreSQL and are not touched. PostgreSQL will not open another major version’s data directory, so a bare image change would have left every existing box with a database that cannot start. I wasn’t willing to ship that.
What happens on the first start. One script, hotloop-pg-upgrade.sh, runs before the database. On the chart it is two init containers on the database pod: the old image hands over its PostgreSQL 16 install and the new image runs pg_upgrade. On Quadlet it is hotloop-db-upgrade.container, a oneshot unit that hotloop-db.container requires. It works in copy mode into a new directory beside the old one, starts the new server once to prove it opens, and only then swaps the names. A fresh install, or a volume already on 17, does nothing.
Expect a wait. The database pod sits in Init for as long as a copy of your data takes, and the app waits for it. The log is kubectl logs <release>-postgresql-0 -c pg-upgrade on the chart and journalctl -u hotloop-db-upgrade on Quadlet.
Free space. It needs about as much free space beside the data as the data takes, and it checks before it starts. If there isn’t room it stops, with the 16 data exactly as it was and the database not started.
pgdata.pg16 is kept, and it is yours to delete. The old directory is renamed pgdata.pg16 and left on the volume. It is the whole PostgreSQL 16 database as it stood at the upgrade, which is the backup and the way back, and it is as big as the database. It stays until you remove it. When the box has run on 17 for as long as you want:
Terminal window
```
# chart
kubectl -n hotloop exec hotloop-postgresql-0 -c postgresql -- rm -rf /var/lib/postgresql/data/pgdata.pg16
# Quadlet
sudo rm -rf "$(sudo podman volume inspect hotloop-db --format '{{.Mountpoint}}')/pgdata.pg16"
```
A failure leaves you where you were. If anything fails before the swap, the 16 data is untouched and the database does not start on a half-built directory. On the chart the pod stays in Init:Error, on Quadlet the upgrade unit fails and the database unit behind it never starts. Fix what the log says and start it again.
helm rollback to 4.18.10 will not start its database. This is the caveat. A rollback puts the chart’s database image back to PostgreSQL 16, and 16 will not open a 17 directory. Nothing is lost and nothing is half changed, the database just does not start. Getting it back means the swap in docs/DEPLOYMENT.md: stop the database, move pgdata to pgdata.pg17 and pgdata.pg16 to pgdata. That drops every write made on 17 since the upgrade, because pgdata.pg16 is the database as it was at that moment. Roll forward when you can. The swap-back only works once, and the next upgrade refuses to run while a pgdata.pg16 is in the way.
Quadlet boxes aren’t hit by that. Podman’s auto-update rolls back only the app, never the database, and the 4.18.10 app reads a PostgreSQL 17 database fine, because migrations only add. A Quadlet box that rolls an update back keeps running.
Take a backup first anyway. HotLoop’s own backup is made by the running app through the running database, so it can’t be taken in the step that runs before the database starts. Take one on 4.18.10 before you upgrade and copy it off the box. Restoring it onto 17 is the ordinary restore.
Turning it off. database.postgresql.upgrade.enabled: false drops the init containers, and a volume still on 16 then stops the database from starting. Leave it on. With it on, a fresh install still pulls the PostgreSQL 16 image once per node, because an init container’s image is pulled before it runs.

API changes

New routes: GET and PUT /api/notify/escalation, and GET /manifest.webmanifest and GET /sw.js, which need no session because a phone asks before anybody has signed in. All in docs/API.md.
Integration routes: GET /api/integrations (devices.read), and POST /api/integrations/{id}/probe and /setup (devices.write). No new permission.
Two new protocols, snmp and bacnet, in GET /api/protocols and in the no driver for protocol error’s list. HotLoop now speaks ten.
Kiosk and lockout routes: POST /api/auth/kiosk (no session), POST /api/auth/activity, PUT and DELETE /api/users/{username}/pin and DELETE /api/users/{username}/lockout, both under users.manage. GET /api/users says hasPin and locked, never the code. The login route answers 429 with a Retry-After for a locked account. A kiosk session reports idleSec. New settings ui.kioskIdleSeconds, ui.lockoutAttempts, ui.lockoutSeconds and ui.panelDelaySeconds.
New permission: notify.escalation, admin only. The viewer and operator roles did not get it.
The alarm trail has two new event types, escalated and escalation-ended, with actor set to escalation. A script that reads an alarm’s history and assumes the old list will meet them.
GET /api/stream closes when its session ends, and every time history returns is in UTC. Both are under “What else is in it”.

Upgrading from 4.18.10

Chart: helm upgrade <release> hotloop/hotloop --version 4.18.11 --reset-then-reuse-values, and the same for hotloop-edge and hotloop-iot. The database pod runs the upgrade first, as above.
Quadlet: copy the new hotloop-db.container, hotloop-db-upgrade.container and hotloop-db-upgrade.sh from your product’s directory in deploy/quadlet/ into the units directory, run systemctl daemon-reload, and restart hotloop-db.service. The database unit requires the upgrade unit, so a box that only got the new database unit does not start on 16 data and says why. The PostgreSQL 16 image is pulled only when the volume needs it.
One new migration, 0025_kiosk_pin, for PostgreSQL and SQLite. It only adds columns (a PIN hash, a session’s idle limit, and the lockout counters), every one with a default, so 4.18.10 runs on the schema it leaves. pg_dump first, like always.
Edge Relay: no database and nothing to do. It ships at 4.18.11 because every product ships in lockstep.

What else is in it

Integrations: point HotLoop at a device you know the address of. One flow for every kind of device: ask what the integration needs, look at the real device with the answers, keep the points you want, and nothing it makes is armed. Every tag a setup makes is read-only, and a setup that fails part way leaves no device behind. OPC UA goes through it first. Why it matters: a new kind of device used to need its own screen and its own idea of what a setup does, and now adding one is the same three steps. docs/INTEGRATIONS.md.
SNMP, read-only. Versions 1, 2c and 3 for switches, UPSs, PDUs, power meters and any Linux box running snmpd. HotLoop never sends a SET, because a SET can change a switch. There is no default community, since public is how a plant’s switches get read by whoever asks. The community and the v3 passphrases are stored encrypted like every other device secret. Proven against net-snmp’s agent in the lab at every v3 security level. Not proven on a physical switch, UPS or PDU: vendors differ on MIBs, interface numbering and bulk requests, so treat the first read of a real device as commissioning and compare the probe’s sampled values with the device’s own display.
BACnet/IP, read-only. Building controllers and meters on the IP network: objects, units and status flags, with a device behind a router said to be unreachable instead of silently missing. HotLoop never writes a BACnet property, because a write lands on a priority array and a priority left set holds the equipment where it was put. The client is written for HotLoop from the standard and held to frames recorded off a device and to bacpypes3 in the lab. Not proven on a physical controller, and nothing on an MS/TP trunk behind a router can be reached. docs/PROTOCOLS.md.
Kiosk sign-in, and a lockout that is on by default. A panel opened at /?screen=<name> asks for one field, a PIN or whatever a keyboard badge reader types, and every write through the gate is that person’s, so the audit trail says who opened the valve instead of “panel-3”. Left alone for 15 minutes it signs them out on the page and on the server, so a hung page can’t leave a name signed in. An admin sets a code per account in Settings, Users, or over the API. Codes are bcrypt like passwords, six characters at least, one code opens one account. On your own computer nothing changes: you are signed in and a write goes straight through. The behavior change to know about: wrong passwords and PINs now lock the one account after 10 in a row for 5 minutes, and an admin can unlock it. That is on for every install, including ones that never use a kiosk. A script that retries a wrong password more than ten times will start getting 429. Set HOTLOOP_AUTH_LOCKOUT_ATTEMPTS=0 for no lockout. A wrong code typed alone locks no account, so it can’t be used to lock the whole panel; the panel waits half a second longer per miss, up to 3 seconds, and is never locked. Not built: reading a badge from a reader that isn’t a keyboard. docs/AUTH.md.
Alarm escalation. An alarm nobody acknowledges now goes to the next person. Under Settings an admin edits a policy: people who have an ntfy topic or an address of their own, steps that say who is told after how many minutes, and a repeat. An acknowledgement from anywhere stops it, because the escalator asks the alarm engine before every send, and every step and every stop is a row in the alarm’s own trail, so the logbook has it. A restart in the middle of a chain carries on from what the audit rows say went out. It is off until you turn it on, it needs HOTLOOP_NOTIFY_ENABLED, and it can only use the channels your deployment already has. Why it matters: a page to a phone on a nightstand used to be the end of it. It is proven against the lab’s real ntfy, Gotify and Mailpit with a fake clock, not against a plant’s phones. Details in docs/NOTIFY.md.
The UI installs on a phone and opens with no connection. HotLoop serves a manifest, icons and a service worker. The phone keeps the page and the last answer to the reads a person looks at, a bar across the bottom says what time it is as of, and every write is refused before it leaves the page: the worker never answers, keeps or queues a write. Signing out empties what the phone kept. It only runs over https or localhost, so a plant on plain http gets the same UI as before. Why it matters: an old page that says it is old beats a blank one when the Wi-Fi drops in the one spot you needed it. docs/DASHBOARDS.md.
The live feed stops when its session does. GET /api/stream looked at the session once, when it opened, and a stream stays open for hours, so it kept sending the plant’s live data after a sign out, an expiry, a revoked session or a disabled account. It checks every five seconds now and closes when the session is gone, without counting the check as the session being used, so a screen left on a wall still goes idle. No route, field or permission changed.
Every time history hands back is written in UTC. History gave some of its times back in the box’s own timezone (on SQLite, the readings themselves), so the same trend spelled -04:00 on a plant set to New York and Z on one set to UTC. That covered a bucket’s edge, a raw reading, a tag’s newest value, what it held before a window, the oldest and newest times on the diagnostics page, and the from and to of the window. Each is an instant and the same instant either way, so a script that compares times sees no change. A script that compared the strings will see the offset become Z.
A node that reconnects announces itself once. When an edge node got its connection back, the reconnect and the node’s sender could each find the session not yet born and each publish an NBIRTH, a millisecond apart, with the same bdSeq and nothing new in the second. If the node was being stopped just then, the second landed after its NDEATH and a host saw a stopped node announce itself again. The sender now publishes a birth only when the session has none or a tag needs announcing. It is in the Edge Relay too. No topic, field or setting changed.
Docs that were wrong in 4.18.10. Four links and counts: the Edge Relay page linked a discovery anchor that was never the heading, the 4.18.10 notes linked a 4.18.4 anchor, and the MCP page said twenty-five tools over a table of twenty-seven, which is what the code registers. The README and the roadmap said IoT and Edge were coming next release, and they shipped in 4.18.10.

What isn’t proven

The upgrade is proven on 4.18.10 data, for IoT, Edge and the Gateway, on the chart and on Quadlet, and test/pgupgrade/run.sh keeps it honest with the images that ship. How long it takes scales with your data, so watch the log the first time on a big one.
The swap back after a helm rollback is documented, not run by a test. test/pgupgrade/run.sh checks the kept directory is a PostgreSQL 16 directory, and doesn’t run the swap.
IoT’s plain PostgreSQL directory has no TimescaleDB extension in it, so the move up a product on a PostgreSQL 17 volume has not been run again on 17. It was run on 16 in 4.18.10.
Everything the 4.18.10 notes list as unproven still is: Edge and IoT as live Quadlet units under systemd (except IoT’s SQLite unit), a real Pi, and the 32-bit arm images on real hardware.