Design and Rationale

Status: Draft · Companion to the Workfile specification · Non-normative

Workfile has been designed to make composed work describable, so that an ordinary workflow needs no code. The specification defines the format, and the manifests that make that possible. This document explains some of the design decisions that have informed Workfile’s creation, but is not intended to add any requirement to the specification.

---

Why a format at all

Most work is composed of small, discrete steps. Consider an email that must be classified, recorded as a lead, opened as a ticket, or ignored as spam. Teams have several options to turn this into a workflow:

A script. Code can express all of it. The author then owns everything that is not the work itself: acceptance of the email,the formal classification process, integration with a ticketing and CRM system, retries, backoff, error classification, credentials, a place to run, and a way to resume after a failure. The description of the work ends up in source code, mixed with the machinery that runs it.

A builder tool. A no-code drag-and-drop tool removes that machinery, and holds the workflow as a graph in one vendor’s database. An export is a serialization of the editor’s state. It is not a document that a person writes, reviews, or generates.

An agent’s own tool calls. A model can call each action in turn. Every intermediate result then passes through the model’s context, which costs tokens and time. The model decides the sequence again on every run, so two runs of one request can differ. A transcript is all that remains afterwards.

The same artifact is missing from all three. In the script, the description of the work exists, but it is mixed with the code that executes it. In the builder, it belongs to one vendor. With the agent, it is never written down at all.

A format supplies that artifact. A Workfile states the work and nothing else, so:

• a person can read it, and review it before it runs;
• a model can write it, because the grammar is small and one construct has one form;
• any engine that implements this standard can run it;
• it lives in version control, and a diff shows what changed.

The two authors of a composed workflow

The format divides knowledge between two authors.

The workflow author states what should happen as a series of composed units of work. This lives in the Workfile.

The catalog author declares how a given API behaves: its errors, its retry policy, its idempotency, its correlation keys. The catalog author has a responsibility to track upstream vendor changes and maintain the catalog.

Every design decision in the format follows from keeping these separate.

Why not an existing workflow format

Workfile is not the first format that describes a workflow. Five families of prior work each solved part of this problem. The ones that are easy to use are incomplete, and the ones that are complete require an engineer.

Consumer automation — Zapier, Make, n8n. These are the easy systems. A person who does not program can connect two products in an afternoon, because the catalog is a maintained product: authentication, pagination, and webhooks are already handled. Workfile takes that catalog, and takes the plain language that made it usable. These systems stop at their own edges. The workflow is the editor’s state, held by the platform, so nobody can review it, diff it, generate it, or run it somewhere else. Work that the builder does not cover goes back into code.

CI pipelines — GitHub Actions, GitLab CI, Tekton. These are also easy, and they proved that a workflow can be a file. Workfile takes YAML steps with ids, results referenced by id, and reusable units that declare their inputs. Their execution model is a job on a runner, started by an event in a repository. A run that waits a week for a customer to reply is outside it. Each file also runs on one platform, and moving it means rewriting it.

Data and science pipelines — Airflow, Dagster, CWL, Nextflow. These are complete, and they are the source of the execution model. Workfile takes the single-assignment graph, immutable results, and replay from a record. The price is the engineer. The author writes Python and operates a scheduler. The domain is also computation over data, where a repeated step costs time. A repeated step here can charge a customer twice, so idempotency has to be declared rather than assumed.

Durable execution — Temporal, Azure Durable Functions, AWS Step Functions. These are the most complete. A run suspends for a month, resumes, and replays from its record exactly. Workfile takes both properties. The price is that the workflow is a program. The author writes code, and keeps it deterministic by following rules that the language does not enforce. Step Functions is declarative instead, and it runs in one cloud, and each state carries its own retry and error handling.

Engine-independent standards — BPMN, CNCF Serverless Workflow. This family aimed at both ends, and it is the closest relative. Workfile takes the goal directly: a published format, with a conformance suite, that more than one engine can execute. BPMN’s element set is large, so engines implement different parts of it, and authors need a modeling tool. A task names the code that implements it, and BPMN does not describe that code. The diagram is portable, and its implementations are not, so the same file on another engine still needs the work to be rewritten.

The trade-off has one cause. A description of work is not enough on its own, because something must still state how each dependency behaves. The easy systems keep that statement inside the platform, where an author cannot read it, change it, or take it elsewhere. The complete systems give it to the author, who then has to be an engineer to write a workflow. Workfile declares it in the manifest, and the catalog author owns it. A file can then stay small enough for a non-programmer to read, and exact enough for any conforming engine to run the same way.

Keeping the language small

Workfile is a small language, and it will stay small. The expression layer is pure, total, and not Turing-complete. It has no user-defined functions, no macros, no recursion, no unbounded iteration, and no mutation. Logic that needs more power belongs in run, in an agent, or behind a connector.

Because of this design:

• a file is checkable before any side effect occurs;
• a run can replay from its record without contacting a dependency;
• a run can suspend and resume correctly;
• a model can generate a file mechanically, and a validator can tell it what is wrong.

Why store exists, and why it is small

A workflow often needs one small piece of state that outlives a run: a cursor, a deduplication flag, a round-robin counter.

store therefore holds get, put, compare-and-set, and increment on single keys, and nothing more. It defines no enumeration and no query, because those would make it a database and invite schema, indexing, and migration into the format.

Why into exists on triggers

Two sources of one logical event are common: an RSS item and a spreadsheet row that both mean “a post to publish”. Without a projection, every step that reads the payload has to branch on which trigger fired, and the conditional is repeated in each step.

into moves that reconciliation to one place, before the first step. The requirement that every entry bind the same names with the same structure is what makes the result checkable: a path that resolves for one trigger resolves for all.

Triggers that differ in behavior rather than in payload shape are a different problem, and are best addressed via separate Workfiles.