Governing a Cascade instance

How a plant keeps a growing pile of workflows under control: shared settings, folders, annotated runs, redaction, numbers over time, outside secrets, log streaming and where large data lives. Everything here is free and in every build.

Every endpoint named below is in the OpenAPI document at /api/v1/openapi.json, and every one has a test that calls it, on SQLite and on Postgres.

Variables

In the editor: Variables, in the top bar. Instance wide ones are at the top, and each project’s under its name. Only what your role allows is offered.

A variable is a named value a workflow reads as vars.NAME: a setpoint ceiling, a plant’s name, a base URL that differs between the pilot line and the real one.

- name: Check limit
  type: core.if
  version: 1
  params:
    condition: "{{ item.json.temp > toNumber(vars.TEMP_LIMIT) }}"

• A variable is instance wide or belongs to one project. Where both have a name, the project’s wins, and only for that project’s workflows.
• A value is text, up to 8 KiB. Use toNumber(vars.X) when you need a number.
• A name is letters, digits and underscores, starting with a letter or an underscore, up to 64 long, because it has to read as vars.NAME.
• Variables are not secrets. Anyone who can read the project reads its variables, and the value is not scrubbed from run data. A password, a token or a key goes in the credential vault, or in an external secret store (below).
• A run reads its variables when it starts, and again if it resumes after a wait or a restart. A change reaches the next run, not one already in the middle of a step. A name that isn’t there reads as nothing, not as an error.

Call	What it does
GET /api/v1/variables	Instance variables and those in projects you can read; ?project= narrows to one project.
POST /api/v1/variables	Make one. With projectId it’s the project’s, with none it’s instance wide.
PUT /api/v1/variables/{variable}	Change its name and value.
DELETE /api/v1/variables/{variable}	Delete it.

Who may: in a project, the variable.read permission lists them and variable.create, variable.update and variable.delete change them. Viewers and operators read, editors and admins change. Instance wide variables are readable by anyone signed in, because any project’s workflow can read them, and only an instance owner or admin changes them. A custom role can carry any of those four permissions. Changes are in the audit log as variable.created, variable.updated and variable.deleted, with the name and never the value.

Folders

In the editor: the workflow list is the folders. Make one with New folder, nest it with Add subfolder, and move a workflow from the list on its own row. Choosing a folder when you make a workflow puts it there.

Folders group a project’s workflows and nest, the way a plant is laid out: area, line, cell.

• A folder is in one project. Names are unique among the folders next to each other, and a name is up to 100 characters.
• Folders nest up to 10 deep.
• A workflow is in one folder or at the top of its project. Saving a new version doesn’t move it.
• A folder with workflows or folders in it can’t be deleted (409), so deleting a folder is never how a workflow goes missing. A folder can’t be moved inside itself or one of its own subfolders, and can’t be moved to another project.

Call	What it does
GET /api/v1/folders	Folders in projects you can read, flat, each with its parentId; ?project= narrows.
POST /api/v1/folders	Make one in projectId, inside parentId or at the top.
PATCH /api/v1/folders/{folder}	Rename it, or move it with parentId (empty string for the top).
DELETE /api/v1/folders/{folder}	Delete an empty folder.
PUT /api/v1/workflows/{workflow}/folder	Put a workflow in a folder, or at the top with an empty folderId.
GET /api/v1/workflows?folder=	One folder’s workflows, or folder=none for the top of each project.

Every workflow the API returns says its folderId. The folder.read, folder.create, folder.update and folder.delete permissions work like the variable ones, and moving a workflow takes workflow.update. Folder changes are in the audit log, and moves as workflow.moved.

Execution annotations

In the editor: every run’s page has an Annotation box with a thumbs up or down, a note and tags, and the executions list filters by tag and by vote.

A person can say something about a run after it happened: a thumbs up or down, a note, and tags to find it by. “First good batch after the die change” and scrap are the kind of thing that goes here.

curl -X PUT $CASCADE/api/v1/executions/$RUN/annotation -H "Authorization: Bearer $KEY" \
  -d '{"vote":"up","note":"first good batch after the die change","tags":["die-change","first-shift"]}'

• A vote is up, down or empty. A note is up to 2000 characters. A run has up to 20 tags, each 1 to 40 characters with no control characters, listed once.
• PUT replaces the whole annotation, so send everything you want kept. An annotation that says nothing is the same as none. DELETE /api/v1/executions/{execution}/annotation clears it.
• Every run the API returns carries its annotation, and the listing carries it on each row. GET /api/v1/executions?tag=scrap and ?vote=down filter, and they combine with the other filters.
• The annotation records who left it and when, from the same identity the audit log uses.
• Annotating takes the execution.annotate permission, which operators, editors and admins have and viewers don’t. Viewers read annotations.
• An annotation goes when its run is deleted. The audit log keeps execution.annotated and execution.annotation_cleared with the vote and tags, and records only that a note was left, not what it said.

Execution redaction policies

In the editor: Settings, Redaction makes the policies. A workflow picks one in the editor’s side panel with nothing selected, under Workflow settings.

The credential vault already keeps secrets out of everything a run stores. A redaction policy is for the data that isn’t a credential but doesn’t belong in the execution history either: a badge number, an operator’s name, a card number that came in on a webhook.

A policy is a name and rules:

• fields: item key names. A key a policy names has its whole value replaced with [redacted], at any depth, whatever type the value is. Matching ignores case, and * and ? work as wildcards, so *secret* takes plant_secret and SecretKey.
• patterns: regular expressions, in Go’s RE2 syntax. Every match inside any text is replaced with [redacted]. A pattern that matches the empty text is refused.
• steps: optional. The policy covers only the steps listed. Empty means every step.
• appliesToAll: optional. The policy covers every workflow, whether or not it names the policy.

A workflow chooses one by name in its settings:

settings:
  redaction: Badges

What it does and when:

• It runs when the execution finishes, in the same transaction that marks it finished: succeeded, failed or stopped. Until then the run has its real data, because the next step needs it and a restart resumes from it. A run that is waiting still holds its real data.
• It scrubs each step’s stored output, each step’s stored error message, the execution’s own error, and a stored webhook answer. It doesn’t touch file contents: a policy reads item data, and a file is a reference to bytes.
• What was stored is what a replay or a retry from the middle of that run will start from, so a replay sees the scrubbed values. The workflow’s trigger items are in the first step’s output, and they are scrubbed too.
• It reads the workflow as that run had it, so editing a workflow, or the policy, never changes a run that is already over.
• A workflow that names a policy that doesn’t exist fails closed: its runs keep no item data at all, the way the metadata save setting works, and the workflow’s problems say so. A missing policy must never mean data kept that somebody meant to scrub.
• A policy that workflows name can’t be deleted (409), and its name can’t change. Rules can.

Call	What it does
GET /api/v1/redaction-policies	Every policy. Anyone signed in can read them, because the people who write workflows need the names.
POST /api/v1/redaction-policies	Make one.
PUT /api/v1/redaction-policies/{policy}	Replace its description and rules.
DELETE /api/v1/redaction-policies/{policy}	Delete one no workflow names.

Making, changing and deleting a policy takes the instance redaction.manage permission, which owners and admins have. Each is in the audit log as redaction.created, redaction.updated and redaction.deleted.

Insights

In the editor: Insights, in the top bar. Pick a window, a project and a workflow; it shows the totals, a chart and a table per day, and a table per workflow.

GET /api/v1/insights adds the runs in a window up: how they ended, how long they took, per day and per workflow.

curl "$CASCADE/api/v1/insights?from=2026-10-10&to=2026-10-14&project=$PROJECT" -H "Authorization: Bearer $KEY"

• from and to are an RFC 3339 time or a date (midnight UTC). With none, it’s the last seven days up to now. The most it answers for is 366 days.
• project and workflow narrow it. Without them it counts every project the caller can read executions in, so each person’s numbers are their own.
• A run counts on the UTC day it was created.
• totals and every row in days and workflows carry total, succeeded, failed, canceled and unfinished (queued, running or waiting), the average and longest run time in milliseconds over runs that have both a start and a finish, and failureRate.
• failureRate is failed over succeeded plus failed. A canceled run isn’t a failure and an unfinished one hasn’t decided, so neither is in it. It is 0 when nothing has finished either way.
• days lists every UTC day in the window, with zeros for a day nothing ran, so a chart has no gaps to guess at. workflows is the 50 busiest, busiest first.

It reads the same execution rows the execution list does, so it needs the execution.read permission and a key scoped for it.

External secrets

In the editor: Settings, External secrets. Add a provider, try it against a path from the same card, and see what the reference for it looks like.

A credential’s secret field can point at a secret kept in HashiCorp Vault or OpenBao instead of holding the secret itself. Rotate the password in the store and every workflow uses the new one on its next run, with nobody editing a credential, and a copy of your database never holds the password.

A reference looks like this, in any secret field of a credential:

secret://bao/plant/line-3/db#password

That is the provider named bao, the secret at plant/line-3/db, and its password key. The credential stores that text, sealed like any secret. Nothing it points at is ever stored here: the value is read when a run opens the credential, held in memory for the provider’s cache time, and scrubbed from everything the run saves, the same as a password in the vault. Only secret fields take references. A reference typed into a field that isn’t secret is just text.

Setting up a provider

A provider is the server, how Cascade signs in to it, and where it may read. Making, changing and removing one is an instance owner or admin’s, the secrets.manage permission, and has to be done by a person signed in, not by an API key.

curl -X POST $CASCADE/api/v1/secret-providers -H "X-CSRF-Token: ..." -d '{
  "name": "bao",
  "address": "https://bao.plant.example:8200",
  "authMethod": "approle", "roleId": "...", "secretId": "...",
  "allowedPaths": ["plant"]
}'

Field	What it does
name	1 to 40 lowercase letters, digits and dashes. It is in every reference, so it can’t change later.
address	The server’s URL.
mount, kvVersion	Where the KV engine is mounted (secret by default) and whether it is version 1 or 2 (2 by default).
namespace	Sent as X-Vault-Namespace, for a namespace of the server.
authMethod	token with token, or approle with roleId and secretId. authMount is where AppRole is mounted (approle by default). AppRole signs in itself and again when its token runs out.
caCert	A PEM certificate authority to trust for the server, when its certificate isn’t from one the system trusts.
allowedPaths	Required. The only places a reference may read from. plant allows plant and everything under it, and not plantation.
projects	Optional. The only projects that may use the provider. Left empty, every project may.
cacheSeconds	How long a read secret is kept in memory, 0 to 3600. 30 by default. 0 reads every time.

How Cascade signs in (the token, or the AppRole id and secret) is sealed under the instance key, rewrapped by a key rotation with everything else, and never shown by any call. Leave it out of a change and the stored one stays.

POST /api/v1/secret-providers/{provider}/test signs in and, with a path, reads it, and says whether it worked. It never returns a value. GET /api/v1/secret-providers lists providers for anyone signed in, because the people who build credentials need the names. The calls are in the audit log as secrets.provider_created, _updated and _deleted.

What it won’t do

• A provider needs allowed paths. A token for Vault can usually read far more than anyone who edits credentials should be able to name, and without a list, anyone who can make a credential could point one at any secret the token can reach and have a workflow send it somewhere. Give the provider the paths its credentials need and its policy the same.
• A reference is checked when the credential is saved, not only when a run uses it: the provider has to exist, be open to the credential’s project, and allow the path. A path with an empty part, a dot, or a .. is refused.
• A run that can’t read the secret fails and says why, naming the provider, the path and the key and never a value: no such provider, a path the provider doesn’t allow, no secret there, no such key, or the server’s own refusal (a policy that doesn’t allow the read is “permission denied”).
• Deleting a provider doesn’t edit credentials. Ones that point at it fail when a run opens them, saying the provider is gone.
• A secret value that really begins with secret:// can’t be stored as a secret, because it reads as a reference.
• Values that are numbers or true or false are read as text. A value that is a whole object or a list is refused.

Log streaming

In the editor: Settings, Log streaming (owners and admins). Each destination shows how much it has sent, when it last delivered and its last error, with a button that sends one test event.

Send the audit log, and a line for every run that finishes, to the places a plant already keeps its logs: a syslog server, or anything that takes OTLP logs over HTTP (an OpenTelemetry collector, Loki, a log platform). Events leave as they happen, so a SIEM sees a failed sign-in in seconds, and the audit trail exists somewhere an administrator of this instance can’t rewrite.

A destination is one place. Make as many as you need:

curl -X POST $CASCADE/api/v1/log-destinations -H "X-CSRF-Token: ..." -d '{
  "name": "plant syslog", "kind": "syslog",
  "address": "syslog.plant.example:6514", "network": "tls",
  "caCert": "-----BEGIN CERTIFICATE-----..."
}'
curl -X POST $CASCADE/api/v1/log-destinations -H "X-CSRF-Token: ..." -d '{
  "name": "collector", "kind": "otlp", "endpoint": "https://otel.plant.example/v1/logs",
  "headers": {"Authorization": "Bearer ..."}
}'

Making, changing and removing destinations is for instance owners and admins (logstream.manage), by a person signed in and not by an API key, because a destination is where the audit log goes. The calls are in the audit log as logstream.destination_created, _updated and _deleted.

What is sent

• Audit events: every row of the audit log, with its sequence number and its hash. The sequence number and the hash are how a receiver spots a repeat, and the hash is the row’s own, so the chain can be checked from what arrived. A failed or refused sign-in is a warning, everything else a notice.
• Execution events: one for each run that finishes, whatever way it ends: the run’s id, the workflow’s id and name, its project, status, mode, when it was created, started and finished, how long it ran, and its error and the step it failed at. A failed run is an error, a stopped one a notice, a good one informational. They are sent a couple of seconds after the run ends.
• Never item data. What a run’s steps saw and made stays in the run, where redaction policies protect it. The audit log never holds a secret.

events picks one or both (["audit"], ["execution"], the default both). start says where a new destination begins: now, the default, sends what happens after it is made, and beginning sends the whole audit log and every finished run first.

Syslog

A message is RFC 5424: <priority>1 time host hotloop-cascade pid msgid - {json}, with the event as one line of JSON. The priority is the facility times eight plus the severity. The msgid is the audit action (workflow.saved) or the run’s status.

Field	What it does
address	host:port.
network	tcp (the default), udp or tls.
framing	For tcp and tls: octet (RFC 6587 octet counting, the default) or newline. UDP is one datagram each.
facility	local0 by default; any syslog facility name.
appName	The header’s APP-NAME, hotloop-cascade by default.
caCert, serverName	For tls: the authority to trust, and the name the certificate has to be for when it isn’t the host. There is no setting that skips the check.

OTLP

Logs go to endpoint as OTLP over HTTP with a JSON body: a log record for each event, its body the event as JSON, and its parts as attributes (cascade.event, cascade.action, cascade.status, cascade.project, cascade.actor.name and so on) so a receiver can filter without parsing. The resource says service.name=hotloop-cascade, its version and the host. headers are sent with every request, for a token or a tenant. They are sealed under the instance key, rewrapped by a key rotation, and never shown: the list says only their names. caCert and serverName work as for syslog.

How delivery behaves

• At least once, in order. A destination remembers the last audit row and the last finished run it sent. One that is down gets everything it missed when it comes back, in order, and a restart carries on where it stopped. If the sender is cut off between sending and noting it, those events are sent again, which is what the sequence numbers are for.
• A failing destination backs off, one second and doubling to five minutes, and its status says what went wrong and when: lastError, lastErrorAt. A success clears it. Nothing is dropped while it is down, so the audit log is the limit, not a queue.
• One sender at a time. With several mains, whichever holds a destination’s lease sends to it; the others wait and take over if it stops.
• Changing a destination keeps its place. Moving it to another address never resends or skips anything. Headers left out of a change are kept, and an empty object clears them. The name and kind can’t change.
• POST /api/v1/log-destinations/{destination}/test sends one made-up event through the same path and says whether it arrived, with the reason when it didn’t. It moves nothing.
• GET /api/v1/log-destinations shows each one’s status: how many events it has sent, the last audit sequence number delivered, when it last succeeded, and its last error.

Where large data lives

Files and big payloads that a run carries (a downloaded PDF, a camera frame, a spreadsheet) are stored as blobs, apart from the database. By default they sit on the instance’s disk. Set HOTLOOP_CASCADE_BLOB_STORE to put them in an S3 bucket or an Azure blob container instead, so the main can be rebuilt without carrying them and several mains share one copy.

Setting	Meaning
HOTLOOP_CASCADE_BLOB_STORE	fs (the default), s3, or azure.
HOTLOOP_CASCADE_BLOB_SPOOL_DIR	Where a blob waits while it is hashed before upload. Defaults to blob-spool in the data directory.
HOTLOOP_CASCADE_BLOB_S3_BUCKET, _ACCESS_KEY, _SECRET_KEY	Required for s3. The secret also reads _SECRET_KEY_FILE.
HOTLOOP_CASCADE_BLOB_S3_ENDPOINT, _REGION	Leave the endpoint out for AWS. Set it for Garage, MinIO or Ceph.
HOTLOOP_CASCADE_BLOB_S3_PATH_STYLE	true or false. Path style is the default whenever an endpoint is set.
HOTLOOP_CASCADE_BLOB_S3_CA_FILE	A CA bundle for an endpoint with a private certificate.
HOTLOOP_CASCADE_BLOB_S3_PREFIX	A folder inside the bucket, so one bucket can hold more than one instance.
HOTLOOP_CASCADE_BLOB_AZURE_CONTAINER	Required for azure.
HOTLOOP_CASCADE_BLOB_AZURE_CONNECTION_STRING, or _ACCOUNT with _KEY, or _SAS	One way to sign in. Secrets also read a _FILE form.
HOTLOOP_CASCADE_BLOB_AZURE_ENDPOINT, _PREFIX	An endpoint for Azurite or a private cloud, and a folder inside the container.

On the chart, set these through env and extraEnv, with the secrets coming from a Kubernetes Secret.

• Addressed by content. A blob is stored under its SHA-256 (sha256/ab/cdef...), so the same file is stored once however many runs carry it. Putting it again is a no-op, including when two runs do it at the same moment.
• Hashed before upload. The data is read once into the spool file, hashed, and then sent, so memory stays flat for any size. A blob over the size limit is refused, and a cancelled upload leaves nothing behind in the bucket.
• Checked on the way out. A read is verified against its hash as it streams, so a corrupted object fails loudly instead of handing a run bad data.
• Size. A single upload goes up as a multipart transfer on S3 and in blocks on Azure, so the usual 5 GiB single-request limit does not apply.
• Changing store later. Existing blobs stay where they were written. Move them to the new store before switching, because a reference to a blob the new store lacks is a missing file.

These two backends are tried against Garage and Azurite in the test stack, the same conformance suite as the disk store: put and read, empty blobs, dedupe under concurrency, corruption, the limit, bad ids, delete, list, a cancelled put, and a 192 MiB streamed upload with a heap check.

What it means for you: a rebuilt or replaced main finds every file its runs carried, with no copying.