Attributes - A unified solution for idempotency and observability

[pranaygp]

Attributes (f.k.a. Tags)

Attributes Tags are piece of metadata that are associated with runs, steps, hooks, and other future entities. It's a simple key/value solution that addresses two problems

1. Meaningful Visibility - on the o11y tab, we need to provide users a way to search and filter for runs, steps, etc.. Run IDs/step IDs are generated and not meaningful. Tags provide a way to add attributes to these entities which are expected to be indexed by the world for filtering and search. This way, worlds don't need to index all the run/step inputs/outputs (which contains sensitive data)
2. Idempotency - Support for atomically creating a tag, or throwing on conflict, is the most versatile way to do run idempotency (explained further down)

Let's first see it in use:

import { setTag } from "workflow";

async function myWorkflow() {
  "use workflow";

  setTag("foo", "bar"); // returns promises, but you don't need to await them
  await setTag("baz", 1);
}

This is the simplest use of tags - it upserts two tags on the active run - "bar", and "baz".

Signature/Spec:

type Value = string | number | boolean | null | Date;
type SetTagOptions = {
    // prevent multiple entities having the same key:value pair. 
    // uniqueness is enforced globally across the project
    unique: boolean
}
async function setTag(
  key: string,  // bounded to 256 characters (or left to world)
  value: Value | (current: Value | undefined) => Value, // bounded to 256 bytes (or left to world)
  options: SetTagOptions
): Promise<Value>

Key Points

• These are simple key:value pairs that are intended to be short and will show up in the observability when inspecting the entity
• The keys and values should not contain sensitive data
• Keys and values are expected to be indexed by the world to enable search and filtering in the UI
• Worlds can (and probably should) limit the number of tags and/or the size of the tags, to prevent unbounded growth and overindexing
• No nested values. Keep it simple.

Usage for idempotency

Most alternatives implement idempotency purely as an "idempotency key" that needs to be set when the run is invoked, or allow you to customize the Run ID. The main challenge here is that you don't always know what the idempotency key is when starting the run itself

Take this example of a code review workflow:

async function reviewPR(repo: string, pr: number) {
  "use workflow";

  const { checksum } = await getLatestCommit(repo, pr);
  
  // we want run idempotency on the commit SHA, not the PR itself
  // so we create a unique tag
  await setTag("commitSHA", checksum, {unique: true});
  // this throws a `TagConflictError` if there's already a run for this commit
}

Having idempotency controlled within the workflow is more flexible than leaving it to the caller only. Caller based idempotency actually comes for free if we also allow users to set tags at creation time. For instance, with runs, start can be extended to allow passing in tags at invoke time. Example syntax:

start(
  reviewPR,
  ["vercel/workflow", 42],
  {
    tags: {
      commitSHA: { value: "...", unique: true } // if we happened to have it already
    }
  }
)

Doing the "idempotencyKey" is a slippery slope to having to support various kinds of behaviors (for example, Temporal has a whole doc on ID reuse and conflicts to support multiple behaviours. I find the whole thing too complicated). setTag gives all the power to the user.

Let's model various possible behaviours

Fail the current run if an existing run already exists with the same tag

(example already shown above)

Fail the current run only if the conflicting run did not fail

async function reviewPR() {
  "use workflow";
  
  try {
    await setTag("idempotencyKey", "...", {unique: true});
  } catch (error) {
    if (error instanceof TagConflictError) {
      const status = await error.existingRun.status
     if (status !== 'failed') {
        // only throw if the conflicting run failed. otherwise we can continue this new run
        throw e;
     }
    }
  }
}

Cancel the existing run, and continue the current one

async function reviewPR() {
  "use workflow";
  
  try {
    await setTag("idempotencyKey", "...", {unique: true});
  } catch (error) {
    if (error instanceof TagConflictError) {
      await error.existingRun.cancel(); // throws again if cancel fails
    }
  }

  // if execution got here, it means the old run was in a running state, and was successfully cancelled
}

Do anything really...

async function reviewPR() {
  "use workflow";
  
  let existingRun: Run | null = null;
  try {
    await setTag("idempotencyKey", "...", {unique: true});
  } catch (error) {
    if (error instanceof TagConflictError) {
      run = getRun(error.existingRunId)
    }
  }

 // we can now continue this workflow run but adapt behaviour based on this.
 // we can also choose to not use `unique` and simply list all the runs by a given tag
 // and use that result here to model all sorts of complex/custom handling
}

Tags are really powerful 🤯

An Implementation Gotcha!

It's tempting to implement setTag, and especially getTag, directly in the runtime without forking out to a step (for instance, getTag can be completely synchronous since the run data is available already). But calls to setTag and getTag must have the return values cached in the event log to preserve determinism. Otherwise unqiue would always throw, and even simple examples like this would fail:

async function myWorkflow() {
  "use workflow";

  console.log(await getTag("foo")); // should be `undefined` on replays too
  await setTag("foo", true)
  console.log(await getTag("foo")); // should always be true
}

This is not a problems if these APIs are simply implemented as steps themselves, although an optimization later is running these as "local steps" - i.e. run in the workflow process itself while recording them in event log

Special Tags / Reserved Namespace

Tags names beginning with $ are reserved for special functionality and metadata. Here are some proposed tags -

• $name (type: string). If set, the observability CLI and UI will prefer this instead of the ID as a more "human friendly" name.
  • Open question: should these automatically enforce unique? The benefit is we can make commans like npm wf cancel <run-name> work as expected
  • Pranay's 2c - yes, we make them unique and lax that requirement in the future
    *$id (type: string. enforced uniqueness). 💡 insight: this could be particularly useful for solving the steps "stable ID" problem - whereby we would use $id over the implicit generated step IDs during replay
• Insert other ideas? $ai.model/$ai.provider/etc. (for pretty UI icons, etc),

Other examples of using tags

Steps

Since uniqueness is enforced globally across a project (across all runs/workflows), this tags solution extends naturally to providing a layer of step idempotency. This can be particularly helpful to preserve idempotency for steps that is guaranteed across re-runs/upgrades

async function sendThankYouEmail(orderID: string) {
  "use step"

  // When a service provider accepts idempotency keys, this is the natural thing people do:
  const { stepId } = getStepMetadata();
  await card.charge({idempotencyKey: stepId})

  // However, if you, for any reason, cancelled the workflow run and restarted it
  // or you upgraded a run using the "cancel and restart" strategy (RFC pending)
  // then `stepId` is not stable. However, orderId here is. So the recommended
  // approach would actually be:
  await card.charge({idempotencyKey: orderId})

  // However, not every service has an idempotency key solution. This often means
  // people have to roll their own idempotency using redis. However, tags solve that
  // already.
  await setTag("send-thank-you-email-unique-id", orderID, { unique: true })

  // now you can do whatever compute you want here and are guaranteed
  // idempotency. 
}

Another Implementation Gotcha / Open Question
Using unique setTags inside steps would cause every retry attempt to fail.
Also, what is the intended behaviour for tags if a step fails anyway? should tags accumulate across all retries, or do users only care about the final state of the tags (i.e. on the only successful attempt)

Pranay's 2c. Using setTag in a run is actually scoped to the current attempt (note: we don't currently have an "attempt" entity). The state of the tags on the step entity is just a (materialized) pointer to the latest "attempt"'s tags. unique doesn't follow the pointers, so it won't throw across attempts, but will throws across steps

(Also Pranay: Is this getting too complex? There's got to be an occam's razor solution to this that still lets us use tags across runs, steps, and hooks)

Hooks

Tagging hooks is only really meant for o11y, and can only be set when creating the hook. Unique unique when setting tags on a hook is pretty redundant since the token is already unique - the only exception is if for some reason you do want to use autogenerated hook tokens for aesthetics, but you want to fail the workflow run on a unique tag. Pretty esoteric 🤷🏼

For completeness, here's hook usage

async function documentReviewFlow() {
  "use workflow";

  const webhook = createWebhook({
    tags: {
      $name: "Pranay's NDA Review"
      document: "NDA",
      reviewer: "[email protected]"
    }
  })
  await sendEmail("[email protected]", "NDA.pdf", webhook.url);
  await webhook;
}

Now, on he o11y UI, I would have a more useful/readable view in the hook so I can identify what each hook actually is rather than the currently non-descriptive hook ID

Related APIs

getTag (get value of a specific tag inside a run)
listTags (list all the tags and values inside a run)
start(...) can be extended to allow setting tags at call time
createHook/createWebhook needs to be extended to set tegs on hook creation time
notably, steps don't have a way to set keys at invoke time, but this limitation seems fine to me since steps can always set them as the first thing. We can also solve this later by having a run(step, args, opts) api as an alternative to calling steps directly if we need it

Related World Spec Changes

<Not filling this in - should be easy to infer from the above and we can sort out as we go in PR reviews>

Open Questions

• What should the limits be? Size of keys/values? Or should that be left to the world? One benefit of limiting them in the spec instead of leaving it to the world is that the UI does not have to handle arbitrarily sizes/number of tags - but maybe that's not worth worrying about
• Should these be made available at workflow (root), or workflow/api?
  • Pranay's 2c - workflow for setTag, getTag, and listTags (which are all automatically applied to the run/step scope in which they are called), and workflow/api for listRunsByTags (or similar o11y APIs)
• Should we support different uniqueness scopes? (unique across all workflow in a project, across all runs in a workflow, all steps in a run, etc). OR, do we just do globally unique and leave it to the developer to include workflowName, runId, etc. in the key names?

[hugo082]

I really appreciate the flexibility of the tags solution—it's powerful and enables a lot of creative use cases. However, I have concerns about using this API as the primary mechanism for workflow ID/idempotency management:

1. Atomicity Issues

The try/catch pattern for implementing common idempotency policies is fundamentally non-atomic:

try {
  await setTag("idempotencyKey", "...", {unique: true});
} catch (error) {
    if (error instanceof TagConflictError) {
      // RACE CONDITION WINDOW OPENS HERE
      await error.existingRun.cancel();
      // What if the existing run completes successfully during cancel?
      // What if another deployment with the same ID starts right here?
      // What if the cancel fails but we continue anyway?
    }
}
// We are guaranteed to be alone here...

Operations like "terminate existing and start new" need to be atomic at the server level to avoid race conditions.

2. Boilerplate for Common Cases

Looking at real-world usage, 80% of idempotency use cases fall into three patterns:

• FAIL - reject if a run exists (default behavior)
• USE_EXISTING - return handle to existing run
• TERMINATE_EXISTING - cancel existing and proceed with new

Having to write 5-10 lines of try/catch boilerplate for each of these common cases seems tedious when they could be expressed as simple configuration options.

3. Explicit APIs vs Reserved Tags

For core workflow semantics like ID management and conflict resolution, I believe explicit API parameters are clearer than reserved tags. When someone reads code or configuration, seeing workflowIdConflictPolicy: TERMINATE_EXISTING immediately communicates intent, whereas the policy being buried in try/catch blocks inside the workflow makes it harder to understand the behavior at a glance.

Reserved tags like $id or $name make sense for metadata, but conflating them with behavioral policies feels like mixing concerns.

4. Future continueAsNew Considerations

Have you thought through how this interacts with a future continueAsNew API? Typically, continueAsNew keeps the same workflow ID but starts fresh execution. Questions that arise:

• Do tags carry over to the new execution?
• If tags are the idempotency mechanism, does the new execution inherit the same idempotency constraints?
• How do unique tags behave across the continuation boundary?

Just want to make sure we're designing with this use case in mind, even if continueAsNew comes later.

---

To be clear: I'm not against tags as a feature—they're great for metadata and observability. I'm specifically concerned about them being the primary solution for workflow idempotency and ID management, which feels like a core workflow primitive that deserves first-class API support.

[pranaygp]

1. Can you show me a real world use case or two where you need atomic "cancel existing and proceed with current"? (not saying it's not important, I just want to see what you're thinking here in code to find where this breaks)
2. We (or a third party) can ship the wrapping boilerplate logic as a helper function
3. Same answer as above 👆. Think import { startOrCancelRun, startOrGetRun } from "workflow/utils" or similar
4. We don't plan to ship a continueAsNew or reuse ID across multiple runs
   • imo an ID should only ever point to one entity and never change. Lots of properties come from being able to trust that an ID is static (const vs let)
   • This ID reuse policy, and continue as new, are both Temporalism because of a limitation in that system with long event log replays. We have a different solution in mind for that problem when someone hits that limit

[pranaygp]

In general, the goal with workflow is to be extremely lightweight with very strong and flexible primitives. By reducing the surface area of the framework, we can focus on making a few things extremely stable, performant, and versatile. When we can solve multiple birds with one stone, I lean towards implementing fewer "primitives" and building more things on top of those. I'm not at all averse to shipping those helper utilities from the main workflow package if they are common patterns :)

For example, "webhook" is pretty much just built on top of "hook" and uses an ephemeral stream for the callback. defineHook is also just a typed helper on top of hook. Anyone can build them.

sleep was initially just implemented as a step using RetryableError till it made sense to pull it into the runtime natively, while maintaining the exact same user signature so framework updates just get better for the user

[hugo082]

One concrete example where "cancel existing and proceed with current" must be atomic is order fulfillment / shipping orchestration like:

async function fulfillOrder(orderId: string) {
  "use workflow";

  // 1. Reserve inventory
  // 2. Choose carrier, service level, warehouse
  // 3. Purchase shipping label
  // 4. Notify warehouse / WMS
  // 5. Update order status, tracking, etc.
}

This workflow might be (re)started whenever:

• The customer changes address or shipping speed.
• A support agent edits the order (different warehouse, item removed, etc.).

The business rule is: there is exactly one active "fulfillment plan" for a given orderId. The most recent plan wins. = "cancel existing and proceed with current"

If we implement this with attributes and userland cancel, we get an unavoidable race.

• Run A (old address) is in the middle of:
  • creating a label
  • notifying Warehouse A
• Run B (new address) starts, sees the tag conflict, calls existingRun.cancel() on A.
• While cancel is in flight, A can still buy a label or trigger a pick in Warehouse A.
• B then creates a second label and notifies Warehouse B.

Now both “plans” partially win: 1) Two warehouses think they should ship order 123. 2)Two labels may exist with different destinations.

For this kind of workflow, correctness literally means:

• At any time, there is only one authoritative fulfillment run per orderId.
• When a new one starts, the previous one must be definitively "out of the game."

That is why I think "cancel existing and proceed with current" belongs as an atomic, server‑side start policy (for example idempotencyKey + conflictPolicy: MOST_RECENT_WINS), rather than something we approximate with setTag + cancel() inside the workflow body.

[mikearnaldi]

In complete agreement with @hugo082 I would also add: if there is no standard way to call a workflow idempotently (and given workflows body is always idempotent it makes close to no sense ever calling a workflow non idempotently) any higher order abstraction becomes impossible, think for example an API where the user passes in a workflow and it gets executed behind the scenes, such API would have no knowledge of the specific way idempotency is implemented for the specific workflow.

Overall I think this RFC is very good, just not for idempotency of workflow invocation.

[pranaygp]

Would a special/reserved tag like $idempotency achieve this? (see the special tags/reserved namespace section in the RFC)

[pranaygp]

Perhaps start can also support an idempotentKey property in the signature, but it's essentially syntactic sugar on top of tags. This endorses/encourages a convention for the standard way to call a workflow idempotently

[Schniz]

Should we call it annotations instead of tags, like OpenTelemetry does?

[tobiaslins]

@Schniz I think attributes are even better?

[ethshea]

+1 annotations are just the java way of adding attributes

[Schniz]

:claude-thinking: You're absolutely right!

Attributes it is! sorry for the mixup

[pranaygp]

Yup! Aligned on attributes! will update the RFC :)

[1042422227]

Gracias