The embedded MQTT broker

Added in 4.11.0. internal/broker lets this Gateway run its own MQTT broker in-process, so a small, single-box HotLoop install has nothing extra to deploy. Off by default (HOTLOOP_BROKER_ENABLED): a deployment that already runs Mosquitto or EMQX has nothing to point this at, and starting a second broker beside one that already exists is not a safe default to guess into existence.

Why this exists

internal/edgepublish (this Gateway acting as a Sparkplug edge node) and internal/fleet (tracking registered edge relays) both connect outward to an MQTT broker over BrokerAddress, they have never run one themselves. That is fine when a deployment already has a broker, whether that’s an existing Mosquitto instance or one run alongside HotLoop for other reasons. It is not fine as the only option: this project exists to cut the number of moving parts a plant floor depends on, and requiring a second piece of infrastructure just to get a Sparkplug hub running for one Gateway would be adding a moving part back in through the side door.

With HOTLOOP_BROKER_ENABLED=true, this same process listens for MQTT connections on its own address. internal/edgepublish, internal/fleet, or an external edge relay’s HOTLOOP_EDGE_PUBLISH_BROKER can all be pointed at localhost:1883 (in-cluster, the Gateway’s own Service address) instead.

When to use this, and when not to

Use it for a single Gateway, a handful of edge relays, and traffic volumes an embedded broker sharing a process with the historian, the alarm engine and the HTTP API can comfortably absorb, the common case this whole project targets: one plant, one box, minimal infrastructure.

Do not use it as a substitute for a production-grade broker at real scale: multiple independent Sparkplug host applications, very high fan-out, bridging between sites, or workloads that need clustering, horizontal scaling, or a broker’s own dedicated operational tooling (detailed metrics, dynamic ACLs, plugin ecosystems) are all real reasons to run Mosquitto, EMQX, or another dedicated broker instead, and to point this Gateway’s own EdgePublishConfig/FleetConfig at it like any other deployment already can. This package does not compete with that use case; it exists for the deployments that would otherwise have no broker at all.

Logins, and what each one may touch

Three kinds of login get in, and they are not equal.

Logins made in HotLoop (Settings, MQTT, or /api/broker/logins in API.md) carry rules: the topics the login may publish to and the topics it may subscribe to. %u in a rule is the login’s own name and %p the discovery prefix. They start from a profile and are yours to edit after:

Profile	Publish	Subscribe	For
shelly	%u/#, shellies/announce	%u/#, shellies/command, %p/status	a Shelly Gen2 or later, login named after its MQTT topic prefix
esphome	%u/#, %p/+/%u/+/config, esphome/discover/%u	%u/#, %p/status, esphome/discover	an ESPHome node, login named after the node
tasmota	tele/%u/#, stat/%u/#, tasmota/discovery/+/config, tasmota/discovery/+/sensors	cmnd/%u/#, cmnd/tasmotas/#	a Tasmota device, login named after its topic
zigbee2mqtt	%u/#, %p/#	%u/#, %p/status	a Zigbee2MQTT bridge, login named after its base topic
integration	nothing	#	a dashboard, a historian, you with MQTT Explorer
full	#	#	anything, the whole plant

Name a Shelly’s login after its topic prefix and it can announce itself and report as itself, and that’s it. It can’t publish to cmnd/boiler/POWER, it can’t announce a device in somebody else’s name, and it can’t subscribe to # and read the plant. The broker checks every publish, every subscription, and every message on its way out to a subscriber, so a rule can’t be stretched with a clever wildcard. A last will is checked too, when the client connects: the broker library publishes wills without asking anyone, so a login whose will points somewhere it may not publish doesn’t get in. Otherwise a cheap plug could leave a retained ON on the boiler and pull its own power cord.

A change to a login holds from that client’s next packet. Narrow a login and its connected device loses the topics right then; delete one and the client still hanging on can do nothing at all, and can’t connect again. Passwords are bcrypt hashes in broker_logins, which is in every backup; the API never answers with either.

Zigbee2MQTT is the odd one: one bridge announces every device on its radio, so its profile may publish anywhere under the prefix. That’s the bridge’s job, and it’s why the Discovered screen shows who announced what.

HOTLOOP_BROKER_USERS is the old list of user:password pairs (only the first colon splits, so a password may contain one). These can do anything, on every topic, exactly like 4.16 and 4.17. That’s on purpose: nobody upgrades into a plant whose Sparkplug traffic suddenly bounces off an ACL they never wrote. It also means each of those logins is a key to the whole plant. Move devices to logins made in HotLoop and keep the env ones for the things that really need everything: the fleet, an edge relay, the edge publisher. Checked in constant time; a username with the wrong password is refused.

Anonymous (HOTLOOP_BROKER_ALLOW_ANONYMOUS, default false) is anything, for anyone who can reach the port. It’s a per-deployment opt-in for a trusted LAN or a test rig, never something this process decides is fine on its own, and the Discovered screen says so in red while it’s on. It doesn’t loosen anything for a client that does send a username: a wrong password is still a wrong password.

A broker with no HOTLOOP_BROKER_USERS and anonymous off starts fine. Until somebody makes a login in HotLoop, the only thing that can connect is HotLoop.

Other brokers are their own business. On Mosquitto or EMQX, the ACLs are that broker’s to enforce, the same boundary FLEET.md draws, and DISCOVERY.md has an ACL for HotLoop’s own login there.

Discovery, and HotLoop’s own login

With the broker on, HotLoop listens on it for devices that announce themselves (see DISCOVERY.md). The broker notes which login and client published each discovery config. The Discovered screen shows it, and a device accepted from one login that gets announced by another is held for a person, with a warning in the log (“Kitchen re-announced by login garage-node; it was accepted from kitchen-node”). Either somebody moved the device to a new login, or a login is pretending to be a device it isn’t. A person decides which.

HotLoop’s own client on this broker (the discovery listener, and the drivers of discovered devices) logs in with a login the broker makes at every start and keeps only in memory. It can do anything. There is nothing to configure for it, nothing on disk, and nothing in a backup; it is a different one after every restart.

TLS

A plain TCP listener (HOTLOOP_BROKER_LISTEN_ADDR, default :1883) is always available once the broker is enabled. Setting HOTLOOP_BROKER_TLS_ENABLED=true with HOTLOOP_BROKER_TLS_CERT_FILE and HOTLOOP_BROKER_TLS_KEY_FILE adds a second, independent listener (HOTLOOP_BROKER_TLS_LISTEN_ADDR, default :8883) terminated with that certificate. Both listeners run at once when TLS is enabled, nothing closes the plaintext one, so an operator who wants TLS-only reachability enforces that at the network layer (a NetworkPolicy, a firewall rule), the same way this chart already leaves broader network exposure decisions to networkPolicy.ingress rather than the application deciding on its own which of its own ports may be reached.

This mirrors the reasoning internal/notify.SMTPChannel’s STARTTLS fix documents for the alarm-notification SMTP channel: a broker that only ever accepts credentials in plaintext is not a smaller version of that problem, it is the same one, and arguably more direct, since here this process is the side of the connection actually asking for a password. Unlike the SMTP channel’s in-band STARTTLS upgrade of a single connection, the broker’s TLS listener is a second, separate socket, the same shape Mosquitto, EMQX and most MQTT brokers use for a plaintext port alongside a TLS one, and the more natural fit for MQTT, which (unlike SMTP) has no widely-implemented in-band upgrade of its own.

This package does not generate a certificate of its own, self-signed or otherwise, nothing downstream would end up trusting one it invented, so a real certificate (from an internal CA, cert-manager, or any other source) is always the operator’s to provide.

Configuration

Variable	Default	Details
HOTLOOP_BROKER_ENABLED	false	The feature is off until this is set
HOTLOOP_BROKER_LISTEN_ADDR	:1883	The plaintext TCP listener
HOTLOOP_BROKER_TLS_ENABLED	false	Adds a second, independent TLS listener
HOTLOOP_BROKER_TLS_LISTEN_ADDR	:8883	Only used when TLS is enabled
HOTLOOP_BROKER_TLS_CERT_FILE / _KEY_FILE	n/a	Required when TLS is enabled
HOTLOOP_BROKER_ALLOW_ANONYMOUS	false	Per-deployment opt-in; never set this by default
HOTLOOP_BROKER_USERS	none	user:password,user2:password2. Each can do anything; see above. Logins with rules are made in HotLoop
HOTLOOP_BROKER_DISCOVERY	true	Listen for devices announcing themselves; see DISCOVERY.md
HOTLOOP_BROKER_DISCOVERY_PREFIX	homeassistant	The topic prefix announcements arrive under

Startup ordering

cmd/hotloop/main.go starts the broker immediately after the polling runtime, before the Sparkplug edge publisher and the fleet manager. Both of those already retry their own broker connection on failure, so this ordering is not required for correctness, but an operator pointing either of them at this same process’s own address has no reason to lean on that retry when starting the broker first costs nothing and removes the race entirely.

What has and has not been verified

Tested, not verified, see this repository’s own distinction between the two. The broker is tested with a real eclipse/paho.mqtt.golang client, the same client internal/edgepublish and internal/fleet already depend on, driving an embedded broker instance started for each test on an OS-assigned loopback port:

• A correctly-credentialed client connects, publishes, and a second correctly-credentialed, subscribed client receives the message.
• A registered username with the wrong password is refused.
• An unregistered username is refused.
• An anonymous connection is refused when AllowAnonymous is false, and accepted when it is true, and a registered username with a wrong password is still refused even when AllowAnonymous is true, proving anonymous access does not loosen credential checking for a client that does present credentials.
• Every profile held to its topics: what it may publish arrives, and a command topic, another device’s topics, a config in another device’s name, Sparkplug and # don’t. A login hears only what its rules cover. A last will outside its rules keeps it out. A narrowed or deleted login loses the topics on the next packet. HOTLOOP_BROKER_USERS logins and HotLoop’s own still publish and subscribe to Sparkplug, the fleet’s topics and #.
• Who published each config: recorded for configs that got through, not for refused ones or state topics, and never more than 4096 topics.
• A real TLS client, configured to trust a generated test certificate, connects successfully over the TLS listener.
• A plaintext connection to the TLS listener’s port fails cleanly (the TLS handshake never completes) rather than being silently accepted or left hanging.

Manually run end to end against the compiled hotloop binary with HOTLOOP_BROKER_ENABLED=true, connecting a real client over the network to publish and subscribe across the plaintext listener, not just through go test.

Not verified against any MQTT client other than the paho Go client, not, for example, mosquitto_pub/mosquitto_sub, a certified Sparkplug host application, a different language’s MQTT client library, or MQTT 5-only client behavior. mochi-mqtt/server/v2 itself claims to pass the Paho interoperability test suite for both MQTT v3 and v5; this repository has not independently re-run that suite. Also not verified: behavior under the kind of sustained high-throughput or high-fan-out load a dedicated broker is built for, see “When to use this, and when not to” above.