HotLoop roadmap

What HotLoop is, what’s left to build, and what’s broken. If it isn’t on this list it isn’t getting done, so put it on the list.

What HotLoop is

HotLoop is an automation platform for the plant floor, written from scratch in Go. One static binary. No Python, no runtime to babysit, no privileged container, no capability it doesn’t need. Entities, automations, helpers, scripts, a logbook, dashboards, notifications, MCP, and MQTT with discovery, on top of the industrial layer: real PLC protocols, a real historian, ISA-18.2 alarms, OPC UA, Sparkplug B.

Every line of it is ours. Nothing is embedded, bridged or forked, so when something breaks at 3 AM there’s one codebase to read and nobody else’s bug to wait on.

It ships as four products out of one codebase. See Products.

Products

Four products, one codebase, one tag. Every release ships all four in lockstep, same version, each with its own image and its own Helm chart. You pick the one that fits the box, not the one that happens to have the driver you need.

Product	What it is	Databases	Where it stands
HotLoop IoT	Entities, automations, helpers, scripts, the logbook, dashboards, notifications, MCP, MQTT with discovery, the ISA-18.2 alarm engine, backups and every driver. The base the other three are built on	SQLite or Postgres	Split out: its own image (hotloop-iot, amd64, arm64, arm/v7), chart and Quadlet units, built and published by every tag. First released in 4.18.10
HotLoop Edge	IoT, plus the OPC UA server, store and forward upstream, and local screens at the machine. Our Ignition Edge	SQLite, Postgres or TimescaleDB	Split out: its own image (hotloop-edge, amd64, arm64, arm/v7), chart and Quadlet units, built and published by every tag. First released in 4.18.10
HotLoop Gateway	Edge, plus fleet, multi-site and scheduled reports. The site brain	SQLite, Postgres or TimescaleDB	Shipping (cmd/hotloop)
HotLoop Edge Relay	Dumb and tough: poll, forward, survive. No database, no UI. Its own product, not Edge with the screen unplugged	none	Shipping (cmd/edge-relay). Every release publishes its image and chart from 4.16.0 on

Features split the products. Protocols don’t. Every product speaks MQTT on top of every driver we have. A hobbyist with a used S7-1200 on the bench is an IoT user, and telling him to buy the Gateway to talk to it would be a scam.

• The split itself, in the code. go build ./cmd/hotloop is still the Gateway, so nothing changed for anybody on 4.17. -tags hotloop_edge is Edge, -tags hotloop_iot is IoT, and both at once won’t compile. A feature a product doesn’t have isn’t switched off, it isn’t in the binary, and a test asks the toolchain what each one links and fails on anything internal/product/packages.go didn’t give it. Its API routes answer 404 and say which product has it. hotloop --product and GET /api/status say which one you’re talking to
• Four images and four charts from one tag, built in parallel, published to the same Helm repo. The release refuses a tag that any of the four charts doesn’t match. The Gateway keeps ghcr.io/hotloop-io/hotloop and chart hotloop, so nobody on 4.16 wakes up broken
• The database is a setting, not an image. SQLite, Postgres or TimescaleDB, picked at runtime, all in the same binary. SQLite is the pure Go one, no cgo, so the arm builds stay static and a Pi doesn’t need a compiler toolchain to exist. The full suite runs against all three, because a database we don’t test is a database we don’t support
• Moving a site IoT to Edge to Gateway keeps every device, tag, alarm, user and hour of history: swap the image, done. Going back down deletes nothing, the missing features just sit there dormant until you come back
• A Gateway-only setting on IoT or Edge warns and runs, and it still shows and saves in Settings. You can see exactly what you set and why it’s not doing anything, instead of guessing
• IoT keeps 30 days of history by default. Pi disks are small, and the Gateway is where long history lives
• No license keys and no unlocking at runtime. No tier is paid. A tier is an image, and upgrading is an image swap on the same data
• Every box follows :stable and updates itself. The Gateway, Edge and IoT Quadlet units track :stable like the relay’s, with Podman’s auto-update, a healthcheck that needs no shell, and a rollback when the new image can’t come up. release_guard.sh refuses a release whose units are pinned, unhealthy-able, or auto-updating their database. The database stays pinned to the tag its chart runs: a PostgreSQL major is a dump and reload, and the app’s TimescaleDB extension update is one way, so unlike the app it can’t be undone. Run for real on the Gateway: an update that adds a migration, taken with the data intact, and one that migrates and then can’t start, rolled back. That run is why the pod is gone: with the app in a pod Podman restarted the pod’s unit, which came up fine, so a broken release was reported as a success and never rolled back. See the Quadlet README
• The license’s scope line names all four products, in the canonical license and every copy of it. It used to say Gateway, which arguably left the other three with no grant at all
• The split waits for MQTT discovery (Phase 4). IoT’s first release finds Shelly and ESPHome devices out of the box, or it doesn’t ship. An automation base that can’t find a smart plug on its own network is a joke we’re not telling

The rules

These are not suggestions. Every one of them exists because something went wrong without it.

• No stubs. No placeholder, no TODO in the code, no “wire this up later”. If it isn’t built, tested and working, it doesn’t merge.
• Zero regressions. Every change runs the full suite, integration tests included, against real services. The baseline is what passes on main today, and nothing on that list is allowed to stop passing.
• Verified means verified. Against the real thing: real servers, real PLCs, real Ignition. “Tested against a simulator” gets written down as exactly that.
• Every write goes through the one write gate. New features that command equipment use it. No side doors, ever.
• Written as Patrick, shipped as Patrick. No AI trailers, no AI mentions, anywhere.

---

The automation base

In order. Each phase ships as its own tested PRs, and each one stands on the one before it. All of it lands in HotLoop IoT, which means all of it is in Edge and the Gateway too.

Phase 1: entities

The foundation. Every automation, helper, script and screen in HotLoop talks about entities, so they came first. Merged in PRs 1 through 6, after 4.15.3, and first released in 4.16.0. The whole thing is in ENTITIES.md and EQUIPMENT.md.

• Entity model: typed domains (sensor, binary sensor, switch, number, select, button, text, plus OT ones: valve, motor, PID loop, counter, alarm), a state, attributes, stable entity IDs. Pumps, fans, conveyors and compressors are kinds of motor, not domains of their own
• Composite equipment: one motor entity built from its start, stop, reset, running, fault and speed tags, because that’s what a motor actually is
• Map existing tags onto entities without breaking a single tag, API call, MCP tool, OPC UA node, automation or edge relay device file
• Services (switch.turn_on, number.set_value and friends) that go through the existing write gate and nowhere else
• One ISA-95 equipment tree (enterprise, site, area, line, cell, unit, equipment), plus labels. The site / area / line fields we already carry keep working
• Units and device classes
• Entities in MCP, automations and the live feed, and the Entities and Plant screens
• A for: duration on entity state triggers (“the pump has been off for 5 minutes”). Shipped in Phase 2 as forSec on a state trigger; a hold in progress does not survive a restart (AUTOMATIONS.md)

Phase 2: helpers, scripts, scenes, logbook

• Helpers: input boolean, number, select, text; timers, counters, schedules. On a plant floor these are setpoints and shift schedules. Kept across restarts and in backups, timers on the wall clock; see HELPERS.md
• Scripts: named sequences of actions, callable from automations, the UI and MCP. Typed fields, run modes, a dry run through the real gate, every write as script:<id>, in the logbook and in backups; see SCRIPTS.md
• Scenes as recipes: capture a set of setpoints, apply them together, audited. Checked whole and refused whole, never clamped, written in order under one call id and never rolled back, read back from the device, immutable versions, applied from the UI, a rule, a script or MCP; see RECIPES.md
• Logbook: one timeline of every state change, write, alarm and automation run, per entity and per area (LOGBOOK.md)
• Blueprints: reusable automation templates. Typed inputs refused out of range, never clamped; a rule keeps the version it was made from until somebody reviews the upgrade and takes it; four built in; see BLUEPRINTS.md
• The automation language: condition, wait and stop steps in a sequence, and forSec on a state trigger (“on for five minutes”)

Phase 3: dashboards

This is where “better and cooler” has to be obvious from across the room.

• Dashboard builder: cards, drag and drop, as many dashboards as you want. Saves name the version they started from, so two people building one page can’t overwrite each other unseen; see DASHBOARDS.md
• Card set: gauges, numbers, trends, entity lists, write buttons with the confirmation, script buttons, alarm lists, markdown, camera, a floor plan with live tags on it, recipe pickers and the plant status strip. A card commands nothing itself; a button writes through the write gate
• Per-role and per-screen dashboards; the wall board is one of them, in the same switcher and taking the same assignments
• Looks the same on a panel PC, a laptop and a phone: one layout that folds from twelve columns to one, tested at five widths
• Built so the Edge HMI can be made from it: a dashboard bound to a screen name opens at /?screen=<name> with no header and no tabs

Phase 4: integrations and discovery

• Integration framework: an integration adds devices and entities, has a config flow, and is tested against the real thing
• MQTT discovery: Zigbee2MQTT and ESPHome in the format they publish, Tasmota in its own, Shelly Gen2 and later from the statuses it publishes. Devices show up reading, on the embedded broker or any other, and nothing is armed until a person accepts the device and arms each point. An Edge Relay discovers too, read and forward only. Frigate is not here: its MQTT page documents no discovery messages. Everything is verified against recordings of real devices, and Tasmota and Shelly also against the makers’ documentation, not against a device on the bench. See DISCOVERY.md and EDGE-RELAY.md. The four-product split waited on it
• Network discovery (mDNS, SSDP, DHCP): devices heard on the plant network show up on the Discovered screen with what they are and how HotLoop could talk to them, and a person accepts or ignores each. An OPC UA server becomes an OPC UA device, a Shelly an HTTP device, an ESPHome node points you at MQTT; nothing is added by itself and nothing is armed. Listening only by default; probing is switched on per network by the deployment. DHCP needs host networking and port 67, so it’s off until a deployment gives it both. Tested against real avahi, MiniDLNA and udhcpc; not yet against real devices on the bench. See NETWORK-DISCOVERY.md
• BACnet, SNMP, ONVIF cameras, Shelly native
• Firmata, for Arduinos on USB. Verified on real boards only: Arduinos and a few other Firmata devices are on the way to the bench, and a Firmata emulator would just be us agreeing with ourselves
• UniFi, see UniFi
• Sun, for anything outdoors: binary_sensor.sun_up, the sun’s elevation and azimuth, and sun triggers at dawn, sunrise, noon, sunset and dusk with an offset. Worked out on the box from the site’s latitude and longitude with NOAA’s equations, no service, and held to every sunrise, sunset and noon NOAA published for 2026 at six places from Utqiagvik to Quito. See SUN.md
• Weather. Not from a cloud weather API: HotLoop doesn’t phone anybody to find out it’s raining

Phase 5: energy, phones, notifications

• Energy: kWh, gas, water, compressed air, and cost per unit made. Home Assistant’s energy dashboard, pointed at a factory
• Web app that installs on a phone and works offline
• Notifications, see below

---

Notifications

Webhook, email, ntfy, Gotify, a Discord webhook and a Discord bot exist, all on the same plumbing. See NOTIFY.md.

• Plumbing first: a rate limiter per channel, coalescing (“7 more alarms in the last minute” as one message), and bounded retry that honors Retry-After
• Signed acknowledge links: an ack from a phone lands as a real ISA-18.2 acknowledgement, and a stale button can’t ack the next time the same alarm trips. ntfy doesn’t say who pressed, so the name on it is the channel’s (notify:ntfy); the Discord bot puts the person’s there

Four channels. That’s the list.

• ntfy, first. Free, self-hosted, works on a plant network with no internet (Android; iPhones need the server to relay through ntfy.sh), and its action button acks the alarm straight back into HotLoop over the plant network. Tested end to end in CI against the real server in a container
• Gotify. Free, self-hosted, works with no internet. No ack button and no iPhone app, so it’s a pager, not a two-way. Tested in CI against the real server in a container
• Discord webhook. Paste a URL, done. allowed_mentions locked down so an alarm name can’t @everyone, and critical alarms can ping a role. A webhook Discord refuses is switched off, not retried into an IP ban. The real send needs Discord, so CI checks the exact payload and the rate-limit handling against a stand-in, and a real send runs when a test webhook is configured. Not yet run against the real Discord
• Discord bot. Ack and shelve buttons right in the channel, gated on Discord roles (nobody until you name them), recorded under the person who pressed. Talks to Discord over an outbound connection (the Gateway), so the plant doesn’t need a public address. CI runs the Gateway protocol and a press end to end against the real alarm engine, with Discord stood in; a live test waits for a real press. Not yet run against the real Discord
• Escalation: nobody acked in N minutes, go wake up the next person or the next channel
• Not doing: SMS and Google Messages (RCS needs carrier approval per company, per country, and a self-hosted box can’t get it), Slack, Teams, Google Chat, Pushover, PagerDuty, Telegram, Opsgenie, Signal, Matrix

---

UniFi

A plant’s network is equipment. When a PLC goes quiet, the first question is “is it the PLC or the switch”, and today you answer it by walking to the UniFi app. So UniFi comes in as a protocol driver, not a side system: every switch, port, PoE budget and WAN is a tag. That gets it the write gate, audit, dry run, alarms, trends, entities, automations, OPC UA and MCP for free, instead of building half of each again for one vendor.

It lands in the IoT base, so all four products get it, and the Edge Relay can poll a console like any other device.

Network

In order. Nothing in here writes to a controller until U5.

• U1: read-only driver. One session per console, reused for everything, with a login budget. Logging in on every check got our own Cloud Gateway to answer 429 and lock us out, twice, so HotLoop logs in once and waits out a lockout instead of extending it. The console’s certificate is pinned on first use, not waved through. Devices, switch ports, PoE draw, WAN and failover come in as tags that sync themselves: a switch adopted tomorrow shows up by itself, one that’s gone goes bad quality instead of vanishing out from under its alarms. Console passwords are entered in HotLoop and stored encrypted with a key that lives outside the database, so a dump or a backup doesn’t hand anyone the keys to the network. Proven read-only on the house Cloud Gateway and Express 7 on 2026-09-28. Its first real read called the 5G backup up when it had been dead for two days, because it trusted the WAN block’s stale up the same way the console’s summary did. The health checks knew, so the driver reads those now and a test pins it. Done, see UNIFI.md
• U2: clients and OT watch. Clients keyed by MAC, never IP, because the controller’s client table shows addresses a machine left hours ago. Mark a network as OT, accept what’s on it, and anything new on it raises an alarm. Link a HotLoop device to its switch port once, and a silent PLC tells you which port it’s on and whether that link is up. Built, proven against the lab controller, and read-only on both house consoles on 2026-09-29 (right switch and port for every wired client, SSIDs, the 90 s timeout holds). The box gets ticked after the two steps that need somebody at the rack: a real cable pull and a phone joining SCADA. See UNIFI.md
• U3: screens and the topology map. Overview with the WAN strip (goes amber when the site is running on the 5G backup), a topology map laid out the same way every time so nothing jiggles on a wall board, switch faceplates, clients, networks and WiFi. Two consoles stitched into one picture. Click a switch and see which lines on the Plant tree it feeds. The cables are the console’s own downlink table, so a switch nothing lists as plugged into anything stands on its own instead of being hung off the gateway on a guess. Built and proven against the lab controller in a real browser on 2026-10-18; the amber strip is held to the house Cloud Gateway’s real answers on a stand-in console, because the lab’s emulated gateway has no WANs. Not yet looked at on real gear. The map pinches, drags and zooms to eight times, there is a Plant overlay that groups the linked HotLoop devices under their lines, latency sits beside each WAN and a port’s in and out are traced, and an admin can create the usual alarms (device offline, uplink down, gateway on its backup, PoE over a limit they give) in one press. Those were proven on 2026-10-19, with touch emulated rather than on a screen, and a WAN throughput trace is not built: no console field for it has been read from a real gateway. See UNIFI.md
• U4: events. The controller’s events websocket, so a switch dropping off shows in seconds instead of at the next poll, and lands in the logbook. Polling stays the truth; a dead websocket costs speed, never correctness. Proven on the lab controller: a client joining reached the logbook six seconds after the cable went in, and a switch that stopped informing was in the logbook the moment the controller gave up on it, a minute and a half later, with its tags reading it on the next poll. A dropped websocket and a silent one reconnect without a login. Events are filed in the logbook under the switch or AP they are about, and the equipment filter finds them from a linked device through the switches that feed it. The agent tools get_network_topology and get_network_events read the map and the events. See UNIFI.md
• U5: writes through the gate. Locate LED first: zero risk, visible from across the room, and it proves the whole write path on real gear. Then PoE mode, port power-cycle, WLAN on and off, client block, and port disable for an unknown device once that shape is proven on the lab controller and one empty port. Power-cycle has a 15 minute cooldown per port, because an automation cycling a PLC every two minutes is how you cook a power supply. Protected ports come from the topology: HotLoop will not cycle an uplink or the port powering the 5G backup, no matter who asks. Device restart is a plain button: press it, the device restarts, no confirmation, still audited
• U6: self-healing. A built-in blueprint: device silent for N minutes, cycle its port once, then alarm. Once, not forever

Protect, Access and power

Built from Ubiquiti’s docs, and each one merges when it’s run against real gear on the bench, not before.

• Protect. Cameras as entities, motion and smart detections as binary sensors and automation triggers, snapshots on demand and attached to alarm notifications. A HotLoop alarm can fire a Protect alarm trigger so the cameras record. Live video comes with the Phase 3 camera card
• Access. Door position, door events into the logbook, and unlock as a write through the gate. Unlock from MCP is allowed behind the MCP write switch like any other write, and audited like any other write
• SmartPower PDU. Outlet power, on, off and cycle as tags, feeding the Phase 5 energy work

How it gets tested

A real UniFi Network Application runs in the Podman lab and in CI, with emulated switches adopted over the real inform protocol. The controller is real, the hardware on the far end is stood in, and it’s written down as exactly that, same bargain as the Modbus harness. Real PoE, real link drops and UniFi OS logins get proven on the house consoles (read-only until writes go live) and on a dedicated UniFi OS Server box with an isolated test segment and its own PoE switch.

• Not doing: config writes (VLANs, firewall, DHCP, static IPs, SSIDs). We’ve already watched a switch lose its controller after one static IP change; that’s what the UniFi app is for. Firmware upgrades, for the same reason plus a failed gateway upgrade is an outage. Connect EV chargers, until there’s an official API or a charger to test on. Alarm webhooks from the controller into HotLoop, because the websockets carry the same events on connections HotLoop opens itself

---

Historian targets

The built-in historian stays. On top of it, HotLoop writes history out to the databases a plant already runs, because the IT department has a SQL Server they back up and patch, and they are not adopting ours to get a trend.

• Postgres and TimescaleDB (somebody else’s, not the one HotLoop runs on). See HISTORIAN-TARGETS.md
• SQL Server
• InfluxDB (2 and 3)
• MySQL and MariaDB
• A target that’s down never slows polling or the built-in historian. It catches up when it’s back, or it says out loud what it dropped. It catches up from HotLoop’s own history, so a restart in the middle of an outage loses nothing either
• Each one verified in CI against the real database in a container

---

Ignition: verify every product against it

We compete with Ignition and we have to play nice with it. Every product gets verified against the current stable release (8.3.9 as of September 2026; Ignition 2027 was previewed at ICC 2026, no ship date).

What	Status
Gateway OPC UA client → Ignition OPC UA server, secured, signed in	Verified (4.15.2), and it found a real bug
Ignition OPC UA client → Gateway OPC UA server	Verified (September 27, 2026, Ignition 8.3.9 against 4.15.3), every policy Ignition offers. Found two bugs: a new server certificate on every redeploy, and Discover tags declaring text tags float
Gateway Sparkplug edge node → Ignition MQTT Engine	Not done. Needs the Cirrus Link MQTT Engine 5.x module for 8.3
Edge Relay → Ignition MQTT Engine	Not done. Same module. Closes “never run against a certified Sparkplug host”
Gateway as Sparkplug host ← Ignition MQTT Transmission	Not done
Edge Relay and Edge vs Ignition Edge, feature for feature	Not done
Re-run the lot when 8.3.10 goes stable	Waiting on Inductive Automation

• Commit the Ignition lab (container setup, config edits, cert trust) as a repeatable script so anyone can rerun this. Done for Ignition’s client against the Gateway’s server (test/ignition/lab.py); the Gateway’s client against Ignition’s server is still by hand
• Ignition 2027 is adding a TimescaleDB historian. We already are one. Figure out interop before they ship

---

HotLoop Edge

• Its own image and chart, see Products
• Local screens and logic that keep running with the uplink down. A test boots the real Edge binary with its Sparkplug uplink pointed at a link that swallows everything and answers nothing, then works the screen, the write gate, a script, a recipe, an automation, an alarm and its ntfy notification on the LAN, and gives the link back to watch the queue replay. The one thing that blocked on the uplink was the edge publisher’s start, and it took the whole Edge down with it at boot; fixed with store and forward. The HMI screens themselves come with the Phase 3 dashboards
• HMI writes that name a human. On your own computer you’re already signed in, so a write goes straight through and logs you. A shared kiosk panel signs operators in with a PIN or badge and signs them out when idle, because “panel-3 opened the valve” isn’t an audit trail
• Store and forward it can prove: replay order, bounded disk, nothing lost. test/storeforward cuts the uplink for minutes under 2000 readings a second, SIGKILLs the Edge mid-outage, overruns the bound and then counts every reading. The first run on the old code lost 86% of a five-minute outage without a word. Now the order is exact, the file stays under its bound, and every reading the bound costs is counted. Numbers in docs/EDGE-PUBLISH.md

HotLoop Edge Relay

• Actually release the image. release.yml only built the Gateway. Now a release also pushes ghcr.io/hotloop-io/hotloop-edge-relay (amd64 and arm64) and publishes hotloop-edge-relay to the same Helm repo, and refuses a tag the relay chart doesn’t match. The first real push happens on the next tag
• Remote updates through fleet management. What “remote” can honestly mean is different per install, so it is. A Quadlet relay follows :stable and Podman’s auto-update restarts it onto each release, with a healthcheck that makes Podman put the old image back if the new one can’t come up. A site can opt into holding updates until an admin approves one on the Gateway. A Helm relay is updated by helm upgrade, and the docs say so plainly. Every relay reports its version and image digest, the fleet list shows them, and each change lands in the system log. Proven with real Podman auto-update runs against a real registry: an update, a release that can’t start rolled back, and a held update approved. Not built, on purpose: signed or digest-pinned images (the Gateway can’t enforce either on a box it can’t reach; Podman’s pull policy is where signatures belong) and a staged rollout (the rollback limits a bad release, and holding sites gives you the order). See EDGE-RELAY.md
• Live config reload. A pushed device file used to sit on disk until something restarted the relay, which meant dropping its broker session and every device session to change one tag. Now it applies the moment it lands: new devices connect, removed ones let go, changed ones reconnect, and the rest keep their sessions. The Sparkplug session and the forward queue stay up, the node rebirths for its new metric set, and a removed metric goes out as null first so no host sits on its last good value. A bad file is refused and the old one keeps running, and /status says which file is running and which one was refused. Proven against Modbus slaves, opc-plc and the Gateway’s own Sparkplug host, including mid-outage; see EDGE-RELAY.md

---

Broken or missing now

Found in code review and in the PRs so far. Each one is real, checked against main.

• Restore couldn’t load a backup from an older schema, and didn’t check the backup’s hotloopVersion at all. The manifest now records every table’s columns and the schema migrations applied. Restore loads each file into those columns by name, lets newer columns take their defaults, leaves newer tables for the app to fill, and refuses a backup from a newer schema, or one with a column this version dropped, renamed or retyped, before loading a row. Backups from before this still restore by position. Tested from every schema level there has been. Fixed, see BACKUP.md
• The notification webhook leaked its URL into errors, the log, the audit event and the /api/notify/test response, and a mistyped one into the startup log. For Discord and ntfy the URL is the password. Fixed
• docs/NOTIFY.md claimed the generic webhook works for Discord, Pushover and PagerDuty. It doesn’t, and now it says so. Fixed
• The log scrubber ate the word after token: in plain log messages. Fixed, and every real credential after those names is still hidden
• A tag range that started at 0 showed a dash for the 0 in the Write tooltip: the API left out a zero minimum. It always sends minValue, maxValue and hasRange now, and MCP’s list_tags sends the range exactly when there is one. Fixed
• Local test runs that share one MQTT broker or one Mailpit step on each other: Sparkplug topics collide, and one run’s report email lands in the other’s mailbox. Give every test its own topic namespace and mailbox so parallel runs can’t collide (CI is fine, each run gets its own services)
• Trend charts and reports disagreed on bad data: store.HistoryBuckets averaged bad and uncertain readings in, reports left them out. Both follow the quality rule now (README, “The quality rule”), and a bucket with nothing good is a gap on the chart. Fixed
• HOTLOOP_DB_STATEMENT_CACHE=false broke creating a device (invalid input syntax for type json), and every other jsonb write with it: on the simple protocol a []byte goes out as bytea. Every json parameter now goes as text. Fixed
• writeError turned every unknown error into a 400, including a shutdown and a database outage. Now it’s a 500 that points at the log line and leaks nothing, and every mistake that really is the caller’s is marked where it’s made. Fixed
• A timezone that wouldn’t load fell back to UTC without a word. Now HotLoop refuses to start and names the variable and the value, and the Settings screen already refused one. Fixed
• The backup round-trip test only covered TimescaleDB mode. It runs on every database now, and from each one into the others: SQLite to Postgres and back, plain partitions to a hypertable and back. Fixed
• No lockout or rate limit on failed logins
• There was no user management screen, only the API. Settings has a Users card now for anyone with users.manage: add a user, change a role, switch one off, set a new password, delete. The server’s own refusals (the last enabled admin, your own account) show up in the dialog as they are. A password over 72 bytes used to come back from the API as a 500; it is a 400 that says the limit. Fixed, see AUTH.md
• Edge publishing has no edgePublish: block in the Helm chart. It has one now: credentials through a Secret, the queue on its own PVC, Recreate rollouts, and a default render that didn’t change by a byte
• Email only did STARTTLS, so a relay on port 465 got a plaintext greeting and the send died in the handshake. Port 465 is implicit TLS now, and HOTLOOP_NOTIFY_SMTP_IMPLICIT_TLS overrides the port either way. Alarm emails and scheduled reports both. Proved against a real SMTP server with a TLS-only listener and against Mailpit with --smtp-require-tls; a commercial provider’s 465 isn’t tried. Fixed, see NOTIFY.md
• Device credentials sat in the database and in backups in the clear. Every keyed credential (every password, API key headers) is now encrypted with a key kept outside the database, the chart generates and keeps one, and a restore refuses a backup it can’t open unless told the key is gone. Fixed, see BACKUP.md
• A credential typed into a device’s address or a URL option (mqtt://user:pass@host, ?token=) or into free text is still stored in the clear. Every screen hides it; the database and backups don’t
• OPC UA browse labelled every node float instead of reading its data type, so a text tag served back out through our own OPC UA server came back Bad_TypeMismatch. Ignition 8.3.9 caught it. Fixed (#42)
• OPC UA browse was noisy: it listed the server’s own Server object, which on opc-plc is hundreds of status, capability and configuration variables ahead of the plant’s own. Discover tags leaves it out now. Set browseServerNodes on the device to get it back, or point browseRoot at i=2253 to look inside it on purpose. Fixed
• OPC UA X.509 user tokens, client and server
• Our OPC UA server always offers the deprecated Basic128Rsa15 and Basic256 policies (library limit, needs an upstream fix or a fork)
• OPC UA passwords over 64 bytes can’t log in to our server (library limit). Checked, and it can’t be fixed from our side: awcullen/opcua v1.4.0 refuses a token whose length is over 96 (password plus a 32-byte nonce) before it calls us, in three places, and exposes no option for it. It needs a change in the library or a fork, and no upstream issue exists yet. Documented with the lines in OPCUA-SERVER.md, and a test pins the 64 and 65 byte edge so a library upgrade can’t move it unnoticed
• The chart pins TimescaleDB 2.29.1; 2.30.1 is out. Bumped, tests run on the same version, and an existing install’s extension is now actually updated on start instead of staying on the old version forever
• The edge publisher could publish NBIRTH before it subscribed to NCMD, which Sparkplug B 3.0 section 5 forbids, so a host’s instant Rebirth or Reboot got dropped. Fleet had the same gap listening for births. Fixed
• Sparkplug wants NBIRTH and NDATA at QoS 0 and we published them at QoS 1, because a PUBACK was the only proof the broker took a reading before the forward queue let go of it. Now they go out at QoS 0, the NDEATH will stays at QoS 1, and a chunk comes off the disk only after the broker answers a request sent behind it, which on one ordered connection proves it read everything ahead of it. A QoS 0 write into a dead link is a test now: without that confirmation all 30 readings left the disk. Paho no longer resends anything onto the next session either, because the edge publisher does its own reconnecting on a clean one
• The OPC UA client library (gopcua v0.9.1) can leave its publish loop paused forever when two goroutines race at subscribe, and the device then reads bad for good while swearing it’s connected. The driver now falls back to polled reads when a new subscription stays silent. Fixed
• Same library: if its publish loop stops after values have arrived, the driver keeps serving the last values as good. Catching that needs keep-alive visibility the library doesn’t expose. Needs a watchdog on publish traffic, or an upstream fix (and report the race upstream)
• A recorder’s “no data” codes read as good measurements. A Yokogawa DX channel returning 0x8005 (undefined) on input:1:int16 at scale 0.1 came back good, -3276.3, and was historized as a temperature. A tag now lists the raw values that mean “no measurement”, each with a reason; one is checked before scale and offset and reads bad with the reason, and never trips a value alarm. Modbus checks the register’s bit pattern (every integer width, byte and word order, float patterns in hex), and every other driver that decodes a number checks the same list. Proven against a Modbus server that isn’t ours, with all eight DX codes. Fixed, see PROTOCOLS.md
• A Modbus device that went silent with its socket open (pulled cable, hung PLC) was marked degraded, not down, and its entities sat on their last value until TCP gave up. Now it’s down within one poll plus twice its timeout. Fixed
• S7comm and EtherNet/IP have the same early return when a poll runs out of time. Fixing them needs an in-process S7 and CIP server to prove it against, the way the Modbus fix was proven

Not verified yet

• S7comm and EtherNet/IP against a physical PLC. Nobody should trust either on a live machine until this is done
• OPC UA against KEPServerEX, WinCC, Siemens and Beckhoff
• Reports: real mail providers, real spreadsheet apps, a browser, and the biggest report on a big historian

Tests and CI

On September 26, 2026 the full suite was 555 of 557 passing on main. Both of those are fixed, and CI runs every test on every push and pull request now, so a pass count written here would be stale by lunch. The Actions tab has today’s. The two that were failing:

• TestSchedulerFiresOnItsOwnAndEmails hardcoded a date, so it passed on exactly one day of the year. Fixed
• TestAddressSpaceFollowsConfiguration flaked under load. It wasn’t the test: our OPC UA server had holes in its address space while the config changed, and clients could hit them. Fixed in 4.15.3
• CI skipped whole groups of integration tests because it never started the mail servers, the Emberburn stand-in or the OPC UA write target. That’s how a test that only passes one day a year got in. CI now starts every service and fails on any skipped test it doesn’t allowlist (#21)
• Commit the test services to the repo so anyone can run the full suite with one command: test/fullsuite.sh, with the services under test/services/ (#21)
• Headless browser tests for the UI, so “zero regressions” covers the screens too. They drive the real page in headless Chrome, and the full suite runs them (#20)

Docs

• API reference, docs/API.md: every endpoint, its method, the permission it needs, and what goes in and comes out. TestAPIReferenceMatchesRoutes holds it to the real route table (every route documented, nothing documented that isn’t registered, the permission the one the code checks) and TestAPIReferencePermissionsTable holds its roles table to internal/auth, so it can’t drift

Housekeeping

• The repository moved to HotLoop-io/hotloop (private), and the charts are served from https://hotloop.io/hotloop. The old embernet-ai.github.io/industrial-iot chart repository is a 404
• Repoint the EmberNET App Store at https://hotloop.io/hotloop (owner action). Until then the store is reading a URL that no longer exists
• Cut the first release out of the HotLoop-io org. 4.16.0, tagged 2026-09-28: the first public release since 4.3.1, and the first one under its own license terms
• Delete the stale .kilo worktree
• Decide whether docs/legacy/ still earns its space

---

Last: a modern UI

This goes after everything else on this page, on purpose. A new front end is the most fun thing to build and the easiest way to spend a quarter while the plant still can’t do what it needs to. So it waits, and nothing above gets delayed for it.

• Rewrite the web UI of HotLoop IoT, Edge and the Gateway in Astro, the same framework the HotLoop website is built with. It builds to static assets that get embedded in the one Go binary, exactly like the UI does now, so every product still ships as one image with nothing to install beside it. No Node on the box, no second process, no CDN. The goal is a modern UI across every HotLoop project, one look and one set of parts instead of a screen per era
• A design pass comes first. Before a single component gets written: the screens, the navigation, the way an alarm looks at 3 AM on a phone, and one design system the site and every product share. Rewriting screens without it just moves the mess into a newer framework
• Nothing regresses. Every screen the browser tests drive today keeps working and keeps being driven, and the API doesn’t change for the UI’s sake. A rewrite that loses a screen somebody uses on the floor isn’t done

Nothing here has been started.