Back-pressure, visibly
This is the whole project. The signature failure of a flow engine on an edge box is a pod that quietly inflates until the kubelet kills it, and the log it leaves explains nothing to whoever is holding the pager. So every place a message or a byte can pile up has a ceiling, and every ceiling is a written, documented decision in Node-RED compatibility, not a cap we picked and never mentioned.
| What can pile up | What stops it |
|---|---|
| A node’s inbox | A ceiling (1024 by default) and one of four overflow policies |
| Delay and rate-limit queues | A ceiling, refused to a Catch node past it |
| Trigger timers | A ceiling |
exec output |
Capped per stream; truncation raises an error |
exec processes |
A concurrency limit, refused past it |
| File reads | Capped, with per-line and chunked modes |
| HTTP request bodies | Capped |
| Waiting on an HTTP response | A 504 after a timeout |
| TCP connections and frames | Bounded |
| WebSocket send queues | Bounded; a slow client is disconnected |
The overflow policies, set globally and overridable per node:
| Policy | What happens |
|---|---|
block |
The sender waits for space, up to blockTimeout. Back-pressure goes upstream, which is the right answer almost every time. |
drop-newest |
The arriving message is discarded. Counted and announced. |
drop-oldest |
The head of the queue goes to make room. Counted and announced. For “only the latest reading matters.” |
error |
The send is refused and raised to a Catch node, so the flow decides. |
Nothing gets thrown away quietly. Every drop is counted, pushed to the editor as an event, and exported as a metric you can alert on. When a limit bites, you find out from a graph, not from an operator.
One goroutine per node
Section titled “One goroutine per node”Every node instance gets its own goroutine and its own inbox. Ordered within a node, parallel across nodes. A Function node doing real work occupies one core while the editor, the API, the websocket and every other flow keep moving.
Every recipient gets its own copy
Section titled “Every recipient gets its own copy”When a wire fans out, every node on the other end gets its own copy of the message, so two branches that each think they own it can’t corrupt each other, and a branch never behaves differently from its identical-looking siblings because of the order you wired them in. That’s a bug that reproduces on Tuesdays, and it doesn’t exist here.
It isn’t free. Copying a small message (a reading and a topic) costs 411 to
453 ns and 680 bytes, and a five-node chain moves a message every 1.14 to
1.40 µs, copies included. Where copying hurts is a big binary payload: a 1 MiB
buffer costs 128 to 156 µs and another megabyte on the heap for every recipient.
So large binary payloads travel as ImmutableBytes, which shares the buffer
instead of copying it: 250 to 279 ns and 360 bytes for the same 1 MiB
message. The file and HTTP nodes hand binary payloads out that way already, so
you get it without asking.
Context that can count
Section titled “Context that can count”Flow and global context have get and set, and they also have
CompareAndSwap, Increment and Update. Two branches doing get-modify-set on
one counter race each other and lose updates, and there’s nothing you can do
about it with get and set alone. With Increment there’s nothing to race.
And it survives a restart. Flow, global and node context live in a SQLite file
on the volume, loaded when a scope is first used and written behind, so a counter
or a latch in a flow is where you left it when the pod comes back. The atomic
operations are the same code as before. Changes are batched into one write every
thirty seconds and a value set to what it already was writes nothing, because we
care what this does to an SD card more than we care about being clever. A clean
stop writes everything; a kill -9 loses up to those thirty seconds, and
store.context.flushInterval is the knob. Details, and what it costs a card, are
in docs/durable.