Tests next to the flows

“Did my change break the line?” should get answered before the deploy, by the person making the change, not at 3 AM by the person on call. So a flow can have tests, and they’re YAML that lives next to the flow file and goes through the same review:

line3.test.yaml

flows: line3.json            # relative to this file; flows.json if you leave it out
tests:
  - name: a high reading raises the alarm and nothing else
    inject:
      - node: reading                  # by name or by id
        msg: {topic: line3/temp, payload: 92.5}
    expect:
      - node: over limit?              # the Switch
        port: 1
        msg: {payload: 92.5}
        within: 500ms
      - node: over limit?
        port: 2
        nothing: true
      - node: alarm out                # a Debug: checked on what it received
        msg: {payload: HIGH}

  - name: a negative reading is caught, not alarmed on
    inject:
      - node: reading
        msg: {payload: -4}
    expect:
      - node: errors                   # a Catch node
        error: "out of range"

hotloop-flow test -junit report.xml line3.test.yaml

--- PASS: a high reading raises the alarm and nothing else (0.00s)
--- PASS: a negative reading is caught, not alarmed on (0.00s)
ok  line3.test.yaml  2 passed, 0 failed, 0 errors (0.00s)

That’s the real runtime and the real nodes, in the process, with no server and no editor. Every test gets a runtime of its own, so nothing one test leaves in a Delay or a context can leak into the next.

• inject puts messages in, in order, each one handled before the next goes in. A node with an input gets it there. A source like an Inject or an MQTT In sends it out of its first output as if it had produced it, and an Inject node given no message gets its button pressed.
• expect watches a node’s output, counting ports from 1 the way the editor does. A node with no outputs, like a Debug, is checked on what it received. A subflow instance is watched on its own outputs, wherever inside it they come from.
• msg is what the message has to contain. Every property you give has to be there with that value, all the way down, and anything else on the message is ignored, so a test about the payload doesn’t break when somebody starts setting msg.topic. assert is a JSONata expression that has to come out true, for the checks a literal can’t make. error is text the error a Catch node caught has to contain.
• within is a deadline. A message that turns up late fails, and the failure says how late. nothing: true and count: N are checked once the flow has gone quiet, which is the only time “nothing came out” means anything. Two expectations on one port are two messages, in that order.

A test ends as soon as everything it waits for has happened and the flow has nothing left to do. The timeout is 5 seconds unless you say, on the test’s own clock, which is the next section.

A failure says what came out instead and where it differs:

--- FAIL: the alarm says LOW (0.00s)
    expect 1: debug "alarm out" (alarm-out): 1 message(s) received and none matched; the first one differs at msg.payload: wanted "LOW", got "HIGH"
      at 0s: {"payload":"HIGH","topic":"line3/temp"}

A test that can’t run is an error, not a pass: a node it names that isn’t in the flow, a name two nodes share, a port the node hasn’t got, or a node in the flow that won’t start. So is a misspelt key. expcet: is refused when the file is read, because a typo that was quietly skipped would leave a test with nothing to check, and a test with nothing to check passes. Exit status 1 for anything short of all passing, and -junit writes the report every CI system reads.

On the test’s clock

Nobody runs a test suite that waits out a five-minute Delay, so a test doesn’t wait. It keeps its own clock and moves it straight to the next thing anything in the flow is waiting for: a Delay letting go, a Trigger’s second message, an Inject on an interval or a crontab, a Join or a Batch timing out, a Link Call giving up. The timer fires, the flow handles whatever it sent, and only then does the clock look for what’s due next, the same order the real minutes would have taken. Five minutes of Delay is one step and about a millisecond.

  - name: a reading at 4m pushes the watchdog's off out to 9m
    clock: 2026-10-05T06:00:00Z     # when the test starts; the real time if you leave it out
    timeout: 20m
    inject:
      - {node: watchdog, msg: {payload: reading}}
      - {node: watchdog, msg: {payload: reading}, at: 4m}   # at: on the test's clock
    expect:
      - {node: watchdog, msg: {payload: "on"}}
      - {node: watchdog, msg: {payload: "off"}, within: 9m}

  - name: and not a moment before
    timeout: 8m59s
    inject:
      - {node: watchdog, msg: {payload: reading}}
      - {node: watchdog, msg: {payload: reading}, at: 4m}
    expect:
      - {node: watchdog, count: 1}  # the "on", and nothing else yet

at puts an injection at a moment on the test’s clock, and everything due before it happens first. within and timeout are on that clock too, so timeout: 1h on a flow with an Inject every ten minutes is six ticks and a few milliseconds. clock sets when the test starts, for a flow that cares what time it is: a crontab that fires at half past six fires when the test’s clock says half past six. A Function node’s Date, an Inject’s timestamp, a Change node setting a date and a cookie’s Expires all read the same clock, so a flow that stamps and compares times agrees with itself under a test.

The suite proves every timing node in the palette against it, to the millisecond: Delay in its fixed, random, rate and timed modes, Trigger with extend, Inject on an interval, once at startup and on a crontab, Join and Batch timeouts and a Link Call that never gets its answer. All of it, an hour of plant time included, runs in well under a second. What a test can’t move is the CPU: a flow still busy after a minute of real time fails the test rather than hanging the suite. JSONata $now() and $millis() still read the wall clock.

Tests from the editor

Nobody writes a test for a flow that works. They write one the day it stops working, which is a day late. So the editor makes one out of what the flow just did, in two clicks.

Every message in the debug sidebar has a Test button. Click it and the editor drafts a test from that very message: what came in at the source it entered by, put in there again, and what this Debug node showed, expected here. Change anything you like, because a reading that carries a timestamp is better checked with an assert than pinned to the millisecond. Click Add to tests and it’s in the suite beside the flow file, at the end, with whatever was already written there left exactly as it was, comments included. A name the suite already has gets a number on the end rather than replacing anything.

How the editor knows what came in: the runtime keeps the last 64 messages that entered the flow at a source and the last 64 that reached a Debug node, under the id a message carries from one node to the next. That’s a ring with a size cap, and a message over 64 KB isn’t kept at all, so it can never be the thing that grows a pod. A message that’s been pushed out, or that a node replaced with a new one on the way, can’t be drafted, and the editor says so instead of guessing what went in.

The Tests panel runs the suite against the working copy, deployed or not, which is the point: you find out before the deploy. Every node a passing test went through gets a tick, every node a failure is about gets a cross with the reason on hover, and clicking a failure takes you to its node. Edit anything and the marks go, because they describe a flow that no longer exists. Edit opens the whole suite as YAML. A deploy the gate refuses lands in the same panel, failures on the nodes.

All of it runs in a real browser against the real binary on every pull request.

The deploy gate

Tests on somebody’s laptop prove the flow they had on their laptop. The gate proves the one being deployed. Keep the suite on the instance, next to the flow file it tests (data/flows.test.yaml beside data/flows.json, through PUT /tests or the editor), turn on tests.gate, and every deploy runs it against the flows it’s about to put in place. One that doesn’t pass is refused, a 422 with the whole report:

$ hotloop-flow deploy -file flows.json -note "raise the alarm at 95"
1 changed

Tab "Line 3" (t1)
  ~ switch "over limit?" (chk)
      rules[0].v: "80" -> "95"
hotloop-flow: the deploy was refused: 1 of 2 flow tests didn't pass, and the flows already running are untouched; "a high reading raises the alarm and nothing else": expect 1: switch "over limit?" (chk): nothing sent from port 1

--- FAIL: a high reading raises the alarm and nothing else (0.00s)
    expect 1: switch "over limit?" (chk): nothing sent from port 1
    expect 2: switch "over limit?" (chk): expected nothing, and 1 message(s) sent from port 2
      at 0s: {"payload":92.5,"topic":"line3/temp"}
    expect 3: debug "alarm out" (alarm-out): nothing received
--- PASS: a negative reading is caught, not alarmed on (0.00s)
FAIL  flows.test.yaml  1 passed, 1 failed, 0 errors (0.00s)

That’s a real run: the binary with the gate on, the suite from the top of this section kept beside its flow file, and a deploy token.

The flows already running don’t notice. The gate runs before a byte is written: no flow file, no credentials, no deployment record, no restart. The refusal goes in the audit trail as a deploy.refused that says the tests failed. And the tests don’t run in the server process at all. They run in a child, this same binary as hotloop-flow test -stdin, handed the candidate flows and the suite on stdin, because a test starts every node in the flow and some of them register where the whole process shares them: a Link In by its id, an HTTP route by its path. Run in here, the candidate’s Link In would answer for the running one’s the moment it started. In a child it can’t touch anything the line uses, a node that panics on a goroutine of its own takes down the test and not the plant, and the memory goes back when it exits. The test proves it the hard way: the refused candidate has the same Link In, and after the refusal a message through the running flow’s link still arrives.

Three decisions worth knowing. It’s off by default, because a gate an operator didn’t ask for is a deploy that fails at 3 AM for a reason nobody on shift has heard of. Tests that can’t run don’t pass: a child that won’t start or a run longer than tests.timeout refuses the deploy and says which. And rollbacks aren’t gated. A rollback is how somebody gets out of a bad state, to one that already ran, and holding it up on today’s tests is how an outage gets longer. With the gate on and no suite there’s nothing to fail, so the deploy goes and the log says why it wasn’t checked.

Context and credentials

Every test starts with empty context, whatever the test before it left behind. A counter one test pushed to 42 is not where the next one starts, because a suite where the second test only passes when the first ran first is a suite that lies the day somebody runs one test on its own. So a test that needs a counter at 41 says so, and says what context has to hold when it’s done:

  - name: picks up where the line left off
    context:                        # what the test starts with
      global: {limit: 90}
      flow: {Line 3: {count: 41}}   # flow context, by tab label or id
    credentials:                    # by node name or id; never the real ones
      historian: {token: test-token}
      plc api: {password: s3cret}
    inject: [{node: count, msg: {payload: 95}}]
    expect:
      - {node: out, msg: {count: 42}}
      - context:                    # what it holds at the end
          global: {alarm: true}
          flow: {Line 3: {count: 42}}
          node: {count: {last: 95}} # node context, by node name or id

context in an expectation is matched like msg, and null means the key must not be set. A failure says what the scope does hold, since the usual answer is a typo: flow context of "Line 3" (t1): cuont: not set; it holds count.

A test never sees the real credentials. It runs with the ones it names, which is what makes a test safe to commit, and they reach a node exactly as the real ones would: an HTTP request’s password shows up in the Authorization header it would have sent, and an InfluxDB configuration that won’t start without a token starts with the test’s. A node that reads a password from a mounted file through ew_credentialFiles gets the test’s credential for that field instead, because a test reads nothing off the disk, and an empty one when the test names none.

Context lives in memory for a test, never in the state database, so a test can’t read what the running flows left behind or leave anything for them. A fixture is YAML in the test, not a file pretending to be a database.

Nothing leaves the test

A test that ran your MQTT Out for real would publish to the plant’s broker, and at that point it isn’t a test, it’s a deploy. So under a test, every node that talks to the world outside the process gets a stand-in instead: MQTT in and out, HTTP in, response and request, PostgreSQL, InfluxDB, TCP and UDP in, out and request, WebSocket in and out and the listener and client behind them, the file nodes, exec, scan and netinfo.

The node still does its whole job right up to the wire. The MQTT Out still works out the topic, the QoS and the bytes; the HTTP Request still renders the URL, builds the body and follows redirects; the PostgreSQL node still builds the SQL and its parameters. Then it hands that to the stand-in, which records it, and answers with whatever the test scripted. Unscripted, the answer is nothing and success: no rows, an empty 200, a command that exited 0. The sources (MQTT In, HTTP In, TCP and UDP In, WebSocket In, Watch) don’t subscribe, listen or poll, so what arrives is what the test injects.

  - name: the limit comes from the database and the alarm goes to the broker
    inject:
      - node: reading
        msg: {topic: "3", payload: 92.5}
    replies:
      - node: lookup limit                   # a PostgreSQL query
        reply: {rows: [{limit_c: 90}]}
      - node: maintenance api                # an HTTP Request
        reply: {statusCode: 503, payload: {error: down for maintenance}}
      - node: historian                      # InfluxDB, and it's down
        error: connection refused
    expect:
      - node: lookup limit
        sent: {query: "select limit_c from limits where line = $1", params: ["3"]}
      - node: to the plant                   # an MQTT Out
        sent: {topic: line3/alarm, payload: HIGH, qos: 1, retain: true}
      - node: errors
        error: connection refused

sent is what the node would have sent, matched like msg. replies script what comes back: the first reply listed for a node answers its first call, the next its second, and the last keeps answering, so “it answered, then it stopped” is two lines. error makes the call fail the way the real thing does, so the path your flow takes when the historian is down gets tested before the historian is.

Node	sent	reply
MQTT Out	topic, payload, qos, retain
HTTP Request	method, url, headers, payload	statusCode (200), headers, payload
HTTP Response	statusCode, headers, payload
PostgreSQL	query, params	rows for a query, rowCount for an insert
InfluxDB	line, the line protocol
TCP Out, UDP Out	host, port, payload (session when replying)
TCP Request	host, port, payload	payload, read the way the node reads a real reply
WebSocket Out	payload (session when replying)
File	filename, action, payload
File In	filename	payload, the contents
exec	command, args	stdout, stderr, code (0)
scan	range, ports	devices
netinfo		interfaces

A few things a test doesn’t check, on purpose, because they belong to the box and not the flow: the exec allowlist, the path and discovery scopes, and the certificates in a tls-config. Nothing runs, opens, probes or dials under a test, so there’s nothing for them to fence in or encrypt, and the deploy still answers to all of them on the box it lands on.

The proof is a test that points every one of those nodes at a TCP port, a UDP port and a directory the test holds, runs a reading through all of them, checks what each would have sent byte for byte, and then counts: zero connections, zero datagrams, no files written, nothing executed. A node that reaches outside and has no stand-in is refused by name before anything runs, and the palette test fails if anybody adds one.