Authoring Guide

Status: Draft · Companion to the Workfile specification · Non-normative

This guide is not part of the specification. Nothing here adds to or overrides the spec; where the two disagree, the spec wins. What this document does is answer the two questions the spec deliberately does not: where should this piece of logic live? and how do I express the workflow shape I already have in my head?

Where logic lives

A Workfile is the efficient form for composed actions, even when the task runs once. An agent that composes actions through its own tool calls moves every intermediate result through its context, as tokens. A Workfile moves the same data through the engine, and a model contributes only the file. The validator checks the file before any side effect occurs. The engine holds the catalog, the connections, and the credentials in one standard environment. An agent uses that environment by submitting a file instead of calling each tool itself. One case needs no file: a question that a model answers by reading the data itself.

Workfile has no single escape hatch. It has four valves, and each has a jurisdiction. Reaching for the wrong one is the most common authoring mistake, so start here:

You need…	Reach for…
A pure computation over values already in scope	An expression, in a `value` step or inline
Prose or markup the workflow owns — an email body, an alert message	A local template, inline in a `render` step or in files beside the Workfile
A pure computation the core filters can’t express, reused across files	An extension filter package
A small piece of state that must survive across runs	The `store` namespace
A loop where the next call depends on the last result	A self-transitioning state in the `states` profile
Anything that touches the outside world	A connector from the catalog
A pure computation too involved for a filter chain	`run`
A sub-task whose steps can’t be specified in advance	`agent`

Two consequences of this layout are worth internalizing:

run cannot compensate for a missing connector. run is sandboxed pure compute: it cannot invoke actions, has no network access by default, and holds no state. If the catalog lacks a connector for the system you need to reach, the workflow is not harder to write — it is unwritable until the connector exists. The fix for a missing integration is always a connector (or a wf/http canonical manifest), never a code step. Plan catalog coverage first; it is existential, not incremental.

Most “I need code” moments don’t need run. Deriving a field per element is each. Filtering on a field is where(path, op, value); filtering on a computed test is keep(expr). Pulling repeated matches out of text is extract_all. A lookup table is a map literal with default. A counter is store.increment. run is for genuine data reshaping — a computation over dynamic keys, a nontrivial algorithm — and in a typical portfolio of workflows it should appear in well under one file in five.

Translating imperative habits

Workflows usually arrive as imperative pseudocode. Several common shapes have a Workfile rendering that is not the literal translation — and the non-literal version is the one that validates, replays, and scales. These are the recurring ones.

“Poll on a schedule, fetch what’s new, save the cursor”

TRIGGER: schedule (hourly)
items = source.fetch_since(last_check)
FOR item IN items: ...
set(last_check, now)

Don’t build this. The schedule-plus-cursor loop is what a poll trigger is: the trigger owns the cursor, the implementation manages it, and your file receives one event per item. The Workfile version has no cursor, no store, and no batch loop:

on:
  reviews.received:            # kind: poll — cursor lives with the trigger
    connection: acme-yelp

steps:
  - log: { sheet.append_row: { table: Reviews, row: { rating: "{{ review.rating }}" } } }
  - alert:
      when: "{{ review.rating <= 2 }}"
      slack.post: { channel: "#reputation", text: "{{ review.url }}" }

Reach for store-held cursors only when no poll trigger can exist for the source.

“Loop over services”

FOR platform IN [x, linkedin, facebook]:
    platform.publish(adapt(post, platform))

There is no dynamic connector dispatch — connector.action names are static so that every call is validated against a manifest before the run starts. The loop unrolls into named parallel branches, which is also where per-platform differences (length limits, hashtag style) stop pretending to be uniform:

  - publish:
      parallel:
        x:        [{ post: { x.publish:        { text: "{{ content.body | truncate(280) }}" } } }]
        linkedin: [{ post: { linkedin.publish: { text: "{{ content.body }}" } } }]
        facebook: [{ post: { facebook.publish: { text: "{{ content.body }}" } } }]
      on_error: continue

“For each X where …”

FOR attendee IN event.attendees WHERE attendee.domain != our_domain:
    crm.log_activity(contact, type="meeting", ...)

Filter at the source of the for_each, not inside its body — a when repeated on every step of the body is the loop telling you the list was wrong. A test on a field as it stands is where; a computed test — here the domain has to be derived from the email — is keep:

  - followups:
      for_each: "{{ event.attendees | keep((current.email | split('@') | last) != inputs.our_domain) }}"
      as: attendee
      max: 50
      steps:
        - contact:  { crm.find_or_create: { email: "{{ attendee.email }}" } }
        - activity: { crm.log_activity:  { contact: "{{ contact.id }}", type: meeting } }

Prefer where when it can say the thing (where('status', '==', 'open')) — it is the stronger, statically-closed form, and the validator will suggest it when a keep is doing a where’s job.

“Wait, re-check, maybe stop” (drip sequences)

WAIT 1.hour;  IF cart.completed: STOP
email(...)
WAIT 23.hours; IF cart.completed: STOP
...

The poll-and-recheck stanza is a workaround for not having events. Workfile has events: wait_for with a timeout inverts the logic — the completion interrupts the wait, and the timeout is what advances the drip:

  - first_window:
      wait_for: commerce.checkout_completed
      match: { checkout_id: "{{ cart.id }}" }
      timeout: 1h
  - done:
      when: "{{ first_window.status == 'received' }}"
      stop: "customer completed checkout"
  - reminder_1:
      mail.send: { to: "{{ cart.customer_email }}", template: cart_reminder_1 }

For longer sequences, the states profile expresses the same thing as states with timeout: { after: ..., goto: ... } and an on: subscription to commerce.checkout_completed that transitions to a final state — one place to see the whole lifecycle.

“A second trigger for the callback”

TRIGGER: deal moves to "Proposal"   → send envelope
TRIGGER 2: esign webhook (envelope.completed) → move stage

The second trigger exists only because the pseudocode’s runtime can’t suspend. A Workfile run can, and correlation routes the callback to the suspended run — so the deal context is still in scope when the answer arrives, and the two halves can’t drift apart:

  - envelope: { esign.send: { file: "{{ pdf.file }}", signer: "{{ deal.contact_email }}" } }
  - signed:
      wait_for: esign.envelope_completed
      match: { envelope_id: "{{ envelope.id }}" }
      timeout: 14d
  - advance:
      when: "{{ signed.status == 'received' }}"
      crm.move_stage: { deal: "{{ deal.id }}", stage: "Contract Signed" }
  - expired:
      when: "{{ signed.status == 'timeout' }}"
      crm.log: { deal: "{{ deal.id }}", note: "proposal expired" }

“TRIGGER: A OR B”

TRIGGER: new item in RSS feed OR new row in "Content Calendar" sheet
post = {title, body, link}

Two sources of one logical event are a single file with two triggers, and into is where the payload shapes reconcile: each trigger projects into the same shared binding, so the steps see one shape and never branch on trigger.name:

on:
  - rss.item_published: { feed: https://blog.acme.com/feed.xml }
    as: item
    into:
      post: { title: "{{ item.title }}", body: "{{ item.summary }}", link: "{{ item.url }}" }
  - sheet.row_added: { sheet: Content Calendar }
    as: row
    into:
      post: { title: "{{ row.title }}", body: "{{ row.body }}", link: "{{ row.link }}" }

steps:
  - publish:
      parallel:
        x:        [{ posted: { x.publish:        { text: "{{ post.title }} {{ post.link }}" } } }]
        linkedin: [{ posted: { linkedin.publish: { text: "{{ post.body }}" } } }]
      on_error: continue

If the triggers call for different behavior rather than different field names, that is two Workfiles sharing their common tail via call — into reconciles shapes, not logic.

“If it exists, update; else create”

Predicate actions (contact_exists) split the read from the branch. Do the lookup once, bind the result, and branch on it with choose — the null-on-absent convention makes has/!= null the test:

  - existing: { crm.find_contact: { email: "{{ lead.email }}" } }
  - upsert:
      choose:
        - when: "{{ existing != null }}"
          then: [{ updated: { crm.update_contact: { id: "{{ existing.id }}", fields: "{{ lead }}" } } }]
      else:
        - created: { crm.create_contact: { fields: "{{ lead }}" } }
        - staged:  { crm.add_to_pipeline: { contact: "{{ created.id }}", stage: "New Lead" } }

“Round-robin assignment”

Round-robin needs a counter that survives across runs — pure expressions can’t hold one, and run can’t either. store.increment is the primitive built for exactly this:

  - cursor: { store.increment: { key: "sales_round_robin" } }
  - owner:
      value: "{{ inputs.sales_team[cursor.value % (inputs.sales_team | length)] }}"
  - assign: { crm.assign_owner: { contact: "{{ created.id }}", owner: "{{ owner }}" } }

If the CRM offers round-robin assignment natively, prefer the connector — the catalog owning operational detail beats the file re-implementing it.

“Fetch every page”

rows = []; cursor = ""
DO:
    page = api.list(cursor)
    rows += page.items
    cursor = page.next
WHILE cursor != null

Usually, don’t write this loop at all. A cursor walk is transport, and the catalog absorbs transport: a well-made list action returns the complete result for its filters, and a poll trigger owns its own cursor. Meeting this loop in a file usually means the manifest is unfinished.

The literal translation is also unwritable on purpose: repeat iterations share no bindings, so no iteration can read the cursor the last one fetched. The format has exactly one loop that carries a value forward — a states file whose state transitions to itself, with the carried values crossing in into and declared in accepts:

initial: start

states:
  start:                       # the initial state cannot declare accepts,
    goto: fetch                # so a seed state supplies the first carry
    into: { cursor: "", rows: [] }

  fetch:
    accepts:
      cursor: { type: "string", required: true }
      rows:   { type: "list[json]", required: true }
    steps:
      - page: { helpdesk.list_tickets: { cursor: "{{ cursor }}" } }
    route: "{{ 'more' if page.next != null else 'done' }}"
    cases:
      more: { goto: fetch, into: { cursor: "{{ page.next }}", rows: "{{ [rows, page.items] | flatten }}" } }
      done: { goto: emit,  into: { rows: "{{ [rows, page.items] | flatten }}" } }
    else:   { goto: emit,  into: { rows: "{{ [rows, page.items] | flatten }}" } }

  emit:
    accepts: { rows: { type: "list[json]", required: true } }
    final: true

outputs:
  tickets: "{{ rows }}"

Three habits of the shape:

A transition routes on values, so a boolean decision routes on a computed label — 'more' if … else 'done' — never on true as a case name.
Appending is a list literal plus flatten: [rows, page.items] | flatten. There is no + on lists.
Nothing bounds this loop except the API running out of pages. for_each and repeat declare max; a self-transitioning state declares nothing. That is one more reason the catalog should own pagination.

Keep the loop behind a call, so the sequential caller stays sequential and binds the loop file’s outputs. A callee cannot itself contain a call, so the loop file is a leaf.

“On failure, try the next provider”

FOR provider IN [primary, backup]:
    IF provider.send(msg): BREAK

There is no break, and one iteration cannot see another. For a known, small set of alternatives, unroll into a fallback chain: mark each attempt optional, and guard each later attempt on the earlier binding. A reference to a failed step resolves to null, so the guard is a null test:

  - via_primary:
      sms_a.send: { to: "{{ lead.phone }}", text: "{{ notice }}" }
      optional: true
  - via_backup:
      when: "{{ via_primary == null }}"
      sms_b.send: { to: "{{ lead.phone }}", text: "{{ notice }}" }

The chain is sequential steps, not a choose: a choose selects its arm before anything runs, and cannot observe an attempt’s failure. Each alternative is also its own static action — there is no dynamic connector dispatch (“Loop over services”, above).

When the alternatives are data rather than steps — a list of mirror URLs, a ladder of page sizes for one action — the chain becomes the loop-with-carry of the previous entry, with an index as the carry (into: { i: "{{ i + 1 }}" }) and alternatives[i] as the argument.

“Halve the batch and try again”

size = 100
TRY bulk.send(records, size)
ON overflow: size = size / 2; RETRY

The retry modifier re-issues the step with the same evaluated arguments, and the manifest classifies which failures that can fix. A retry that changes its arguments is a new call, and it is the loop-with-carry again: the state carries what changes, and the route reads what happened. Two choices shape it:

Adapt on a declared result field (sent.truncated) where the API offers one. A failed optional step is only null, so a route after a failure cannot see the error code — only that the attempt failed.
Carry a position on a ladder of literals rather than computing the next argument: [100, 25, 5][i] shows its bound, and its exhaustion is a route arm. Halving with arithmetic hides the bound and needs an int conversion.

  attempt:
    accepts: { i: { type: "int", required: true } }
    steps:
      - sent:
          bulk.send: { records: "{{ inputs.records }}", page_size: "{{ [100, 25, 5][i] }}" }
          optional: true
    route: "{{ 'done' if sent != null else ('smaller' if i < 2 else 'exhausted') }}"
    cases:
      done:      { goto: confirmed }
      smaller:   { goto: attempt, into: { i: "{{ i + 1 }}" } }
      exhausted: { goto: alert_ops }
    else:        { goto: alert_ops }

“email(user, template=…, data=…)”

A template has two possible homes, and the pseudocode doesn’t say which. When the external system owns the content — an ESP campaign, a DocuSign envelope, a CMS layout — pass data to the template-holding connector and let it render. When the workflow owns the content — transactional mail, alerts, chat messages — the template is local files beside the Workfile, rendered with render:

templates/order_confirmation/
  subject.txt
  body.txt
  body.html

<p>Thanks, {{ order.customer_name }}!</p>
<table>
{{#each order.items as item}}
  <tr><td>{{ item.name }}</td><td>{{ item.qty }}</td></tr>
{{/each}}
</table>

  - confirmation:
      render: templates/order_confirmation
      with: { order: "{{ order }}" }
  - confirmed:
      mail.send:
        to:      "{{ order.customer_email }}"
        subject: "{{ confirmation.subject }}"
        text:    "{{ confirmation.body_txt }}"
        html:    "{{ confirmation.body_html }}"

A short body goes inline in the step instead — render: { text: "…" }, or a parts: map for several — and the checks are identical, so nothing forces a second file.

There is no schema file. The variables are inferred from the template body, with is checked against them in both directions before any run starts, and .html parts escape every interpolation with no opt-out. Optional variables are the ones the template guards itself — {{ discount | default('') }}, or an {{#if discount != null }} block.

Per-audience templates are chosen statically — a route over the audience with one render per case, never a computed path. Over a validated enum, that turns “we added a department with no welcome email” into a compile error.

Idioms

Small patterns that come up often enough to name, but not often enough to deserve syntax.

Ordered enums (the rank idiom)

Enums have no order, so MAX(urgency, tier_priority) doesn’t typecheck. Map each enum to a rank, compare numbers, map back:

  - urgency_rank:
      value: "{{ {'low': 1, 'normal': 2, 'high': 3}[classify.urgency] | default(2) }}"
  - tier_rank:
      value: "{{ 3 if account.tier == 'enterprise' else 2 }}"
  - priority:
      value: "{{ {1: 'low', 2: 'normal', 3: 'high'}[[urgency_rank, tier_rank] | max] }}"

Verbose, but every table is visible, every case is checkable, and there is no hidden collation. If a rank scale recurs across files, it belongs in a filter package — the canonical rank package holds the reusable form.

Guarded groups (the one-armed `choose`)

when guards a single step. To guard a block of steps on one condition, use a choose with one arm and an empty else:

  - big_deal:
      choose:
        - when: "{{ classify.category == 'sales' and enrich.deal_size > 50000 }}"
          then:
            - exec_ping: { slack.post: { channel: "#big-deals" } }
            - task:      { crm.create_task: { title: "White-glove follow-up" } }
      else: []

The spec makes this the representation of a guarded block: the empty else records that the no-op path was considered, references into the unexecuted arm resolve to null like any skipped step, and implementations are barred from warning about the single arm.

Null before a comparison

== and != treat null as just another value, but an ordered comparison against null is an evaluation fault: it fails the step rather than quietly picking a branch. When a value can legitimately be absent, decide its stand-in at the point of use:

  - notify_exec:
      when: "{{ (enrich.crm.lookup.deal_size | default(0)) > 50000 }}"
      slack.post: { channel: "#big-deals" }

default supplies the stand-in; has is the alternative when absence should take its own branch instead of a default value.

Rendering a list into text

Reports and notifications often need “one line per item.” each plus format plus join is the pattern; no template engine is involved:

  - summary_lines:
      value: "{{ deltas | each([current.name, current.pct] | format('{}: {}% WoW')) | join('\n') }}"

When the output is a rich document rather than lines of text, don’t build markup in expressions: render a local template when the workflow owns the content, or hand the data to a template-holding connector (docs.create_from_template) when the external system does.

Parenthesize where the parse isn’t obvious

Filters bind tighter than comparison, so {{ a | length > 0 }} parses as {{ (a | length) > 0 }}. That’s the useful reading, but it isn’t the one a reader guesses. Write the parentheses. Chained comparison is a syntax error (a < b < c), so there is nothing to guess there — but mixed filter-and-arithmetic chains benefit from the same habit, and a formatter should insert them rather than rely on precedence.

`route` dispatches on behavior, not on values

When branches differ only in a value — a channel name, a queue, a label — a route with one-step cases is ceremony around a lookup table. Use a map literal in a value step:

  - channel:
      value: "{{ {'sale': '#wins', 'ticket': '#support'}[event.type] | default('#general') }}"

Keep route for branches that differ in what they do. Over a validated enum it earns its verbosity, because the validator then proves no case was forgotten.

`queue_by` orders runs; it doesn’t share anything between them

A queued run inherits no binding from the run before it. What it sees is whatever effect the earlier run left in the external system. If you wanted a value carried forward, you wanted store.

Ordering is also durable in the awkward direction: a run holds its key until it reaches a terminal outcome, including while suspended. A file with wait_for: ... timeout: 14d and a queue_by will serialize that queue for up to fourteen days. That is usually not what the author pictured.

batch and queue_by can’t be combined — a batch holds many events, and a per-event key doesn’t identify it.

Declare `outputs` on any file you intend to `call`

A call step binds exactly the callee’s declared outputs. A callee with no outputs returns nothing, and every path into the call step is a compile error — which is the right failure, but a confusing one if you expected the callee’s step ids to be visible. They never are.

inputs:
  account: { type: "string", required: true }

outputs:
  refund_id: "{{ issued.id }}"
  amount:    "{{ issued.amount }}"

  - refund:
      call: workflows/refunds/process.yaml
      with: { account: "{{ enrich.billing.lookup.id }}" }
  - confirm:
      slack.post: { text: "Refunded {{ refund.amount }} ({{ refund.refund_id }})" }

The outputs block is the callee’s public surface. Treat renaming a step inside it as a private change, and renaming an output as a breaking one.

Failed iterations are ordinary data

Under on_error: continue, a for_each or parallel scope carries a reserved error binding: null when the sibling completed, and { step, code, message } when it didn’t. Reporting on partial failure needs no special construct:

  - failed:
      value: "{{ attachments | keep(current.error != null) }}"
  - report:
      when: "{{ (failed | length) > 0 }}"
      slack.post:
        channel: "#ops"
        text: "{{ failed | each([current.error.step, current.error.message] | format('{}: {}')) | join('\n') }}"

Under the default fail_fast, none of this applies — the first failure fails the construct.