Follow workflow identity, serialization, durable execution, continuation, and terminal outcomes.
An execution is Alfred's durable record of one logical workflow request. The HTTP request creates or finds that record; Workflow SDK carries the accepted work to a terminal state.
Every workflow request requires an Idempotency-Key. Alfred also canonicalizes
the request body into a logical request identity.
| Mechanism | Behavior |
|---|---|
| Same key, same body | Returns the existing execution. |
| Same key, different body | Returns 409 idempotency_conflict. |
| New key, equivalent normalized body | Reuses a matching execution when continuation is supported. If the matching execution is rolled_back, creates a new execution linked to it. |
| New key, different logical body | Creates a new execution. |
Replaying the original key always returns the execution accepted for that
delivery, including when it is rolled_back. Starting again after rollback
therefore requires a genuinely new delivery key.
Canonicalization is workflow-specific. For add-certificate, domain case,
whitespace, defaults, ticket IDs, Kirby mode, dry-run behavior, and the
certificate override participate in the identity.
A successful trigger returns 202 Accepted with an execution resource. That
means Alfred has durably accepted the intent, not that provider work has
completed.
Acceptance records:
The response provides links for the execution, events, and supported controls. Clients should follow those links and wait for a terminal execution status.
onboard-domain, add-domain, and add-certificate use the
aws-infrastructure concurrency group. One root command owns it at a time.
update-lambda-version uses a target-and-environment lane, so unrelated fleets
can progress concurrently while conflicting versions for one fleet fail closed.
add-subdomain has no shared global lane.
Queued work remains durable. When the active command finishes, reconciliation releases the owner and dispatches the next eligible command. Rollback, continuation, and finalization recovery for the active execution can retain or reacquire priority when releasing the lane would expose inconsistent infrastructure.
Failed Workflow SDK runs retain ownership by default. Alfred releases the lane for a failed run only when the execution has also persisted a structured readiness failure. Readiness failures are explicit, precondition-style outcomes that are safe to retry later and do not require the global lane to remain blocked. Cancellations, unclassified failures, missing runs, inspection errors, and contradictory terminal state remain conservative and require recovery.
The API process applies the same rule to a lane that was already persisted as
recovery-required, including after a process restart. It atomically releases
only a failed owner whose planning stage contains a schema-valid readiness
failure. The concurrency group, owner execution, and lane state must still
match at the write boundary. Missing, malformed, nonterminal, or later-stage
evidence leaves the lane blocked for operator recovery.
Child certificate runs do not acquire a second global slot. They are awaited by the owning root workflow and remain part of its serialized operation.
Execution insertion and Workflow SDK start cannot be one atomic transaction. Alfred therefore records explicit start state:
| Start state | Meaning |
|---|---|
start_pending | Intent exists, but no durable run is bound yet. |
started | A Workflow SDK run ID is bound to the execution. |
start_failed | Starting the run failed and the error is projected. |
The command reconciler compares the Alfred command with the bound Workflow SDK
run. It retains live or suspended owners, releases commands with matching safe
terminal projections, and marks ambiguous terminal results as
recovery-required. Lane transitions happen from this observed state rather
than from assumptions inside a provider step.
Workflow functions contain deterministic orchestration. Side effects are
isolated in "use step" functions, while waits use durable sleep() and
operator checkpoints use deterministic hooks.
After a process restart, Workflow SDK replays orchestration and reuses persisted
outputs for completed steps in the same run. A retryable step can back off
without blocking an HTTP connection. A terminal safety failure uses
FatalError to stop retries.
Active-run hooks protect against duplicate-sensitive work even if two starts are observed. The second run reports the conflicting run rather than executing the same provider sequence.
The add-subdomain workflow makes approval part of durable orchestration. It
reads the exact hosted zone, apex alias, and current records into a normalized
plan whose SHA-256 hash covers the proposed and existing DNS state. Missing
and already-correct aliases continue without operator input. A CNAME or
different A record projects the execution as paused and waits on a hook
derived from the execution ID and plan hash.
Approval resumes only that exact plan. The workflow reads Route53 again before
mutation; a different hash creates a new approval checkpoint instead of using
stale consent. An accepted plan becomes one Route53 change batch containing
the complete deletes and alias upserts, and the workflow waits durably until
the provider reports INSYNC. Retrying a failed or canceled execution reuses
safe durable state, but never treats an old plan hash as approval for changed
DNS.
| Status | Meaning |
|---|---|
queued | Accepted and waiting to start or reacquire the command lane. |
running | Workflow SDK is executing a stage. |
waiting | The workflow is durably waiting for an external condition. |
paused | An active run is waiting for an operator resume hook. |
succeeded | Every required stage completed. |
failed | A stage or terminal reconciliation failed. |
canceled | The accepted execution was canceled. |
rolled_back | Compensation completed and the execution is terminal. |
Stage projections have their own running, waiting, succeeded, or failed
status plus structured output and error fields.
Retry behavior is available for failed or canceled executions of all five runnable workflows. A retry validates the persisted input against the current workflow schema and queues a new durable run on the same execution. A rolled-back execution is never retried.
Equivalent add-certificate submissions additionally use completed stage state
to decide whether contract work remains:
A rolled-back execution is permanently terminal. Alfred preserves it as an
audit record and rejects retry controls. To run the workflow again, submit a
fresh request with a new delivery key; Alfred creates a new execution linked
to the rolled-back one through predecessor lineage. Replaying the original
Idempotency-Key returns the rolled-back execution instead of starting work.
Continuation inspects successful stage projections and starts a new durable run only when required. It skips stages whose required output is already persisted; within the new run, Workflow SDK also reuses completed step outputs during replay.
Platform routing has two completion boundaries. Alfred first commits Helm routing and waits for the relevant ArgoCD applications to converge. It then checks the nginx booking configuration. Missing booking entries are validated and committed atomically to the configured nginx base branch. The commit SHA is persisted in the platform-routing stage output, while production nginx deployment is represented as structured Platform assistance. Entries already present on the base branch are a no-op and require no deployment assistance.
An nginx booking commit changes only the Calipso CORS map and next-deimos server-name list. Alfred bases the two-file commit on the observed branch head, retries concurrent ref updates without force, and recovers a lost GitHub response through the operation identity in the commit body. The commit uses a domain-focused Conventional Commit subject and records the entries added to each file in its description.
The Helm commits use Conventional Commit subjects that identify whether Alfred is adding domains to nginx hosts or Mystique Kirby routing. Alfred stores the workflow operation identity and the exact entries it owns in the commit body so a retry can recover a commit after a lost GitHub response and a rollback can remove only those entries.
A rollback run is different: it does not inherit the source run's step cache. It loads validated mutation receipts from the retained source run and executes explicit compensation.
Workflow steps update projections and emit stable outbox events. Event keys deduplicate retried emission, while execution sequence numbers provide ordering for polling, SSE replay, and webhooks.
Treat only a terminal execution status as the workflow result. Provider state is useful for validation, but it cannot replace the execution because later stages may still be waiting, retrying, or reporting manual cleanup.
See Monitor And Control Executions for the operator workflow and Execution Reference for the resource contract.