Helix Agents

Appearance

DBOS Runtime ​

The DBOS runtime (@helix-agents/runtime-dbos) executes agents as durable DBOS Transact workflows backed by Postgres. Workflow state — which steps have completed, their cached results, in-flight recv messages — lives in ordinary Postgres tables managed by DBOS, so you get crash recovery, automatic step retries, and durable interrupt/abort without standing up a separate workflow server.

When to Use ​

Good fit:

  • You already run Postgres and don't want to operate separate Temporal infrastructure
  • Production workloads requiring durable execution and crash recovery
  • Long-running agents that must survive process restarts
  • Chat assistants or stateful sessions that benefit from a hibernating, always-addressable workflow (persistent mode)
  • Teams that want to leverage existing Postgres tooling for observability and querying

Not ideal for:

  • Quick development iteration where the JS runtime's zero-infra story is simpler
  • Agents that need workspaces — DBOS does not support them (see Known Limitations)
  • Deployments with no Postgres and no appetite to add one

Prerequisites ​

You need a Postgres database. DBOS uses it as its system database (workflow status, step cache, in-flight messages) and @helix-agents/store-postgres shares the same instance for session state.

bash
# Local development with Docker
docker run -d --name dbos-postgres \
  -e POSTGRES_PASSWORD=dbos \
  -p 5432:5432 \
  postgres:16

Token-by-token streaming requires Redis (RedisStreamManager) — Postgres write latency is too high for per-token chunk fan-out.

Installation ​

bash
npm install @helix-agents/runtime-dbos @helix-agents/store-postgres @helix-agents/store-redis @helix-agents/llm-vercel @dbos-inc/dbos-sdk

Architecture ​

mermaid
graph TB
    subgraph App ["Your Application Process"]
        Executor["<b>DBOSAgentExecutor</b><br/>execute / resume / retry / submitToolResult<br/>Binds step deps · registers workflows"]
        Workflow["<b>DBOS Workflow Body</b> (deterministic)<br/>planStepProcessing() · buildMessagesForLLM()<br/>shouldStopExecution()"]
        Steps["<b>@DBOS.step()s</b><br/>loadState · callLLM · executeTool<br/>checkpoint · client-tool · approval-gate"]
        Executor --> Workflow --> Steps
    end

    Steps --> PG["Postgres<br/>(session state + DBOS system tables)"]
    Steps --> LLM["LLM API"]
    Steps --> Redis["RedisStreamManager<br/>(token streaming)"]

The workflow body must be deterministic so DBOS can replay it on crash recovery. All I/O (LLM calls, Postgres reads/writes, tool execution, streaming) is wrapped in @DBOS.step() calls whose results are checkpointed in DBOS system tables. See Determinism Rules below and the architecture deep-dive.

Setup Guide ​

The DBOS runtime registers its workflow bodies via module-level decorator evaluation. The critical ordering rule is: register workflows and bind your LLM adapter before DBOS.launch(), because DBOS.launch() asynchronously recovers any pending workflows, and each recovered workflow body reads the live LanguageModel registry at entry.

typescript
import { DBOS } from '@dbos-inc/dbos-sdk';
import { defineAgent } from '@helix-agents/core';
import { DBOSAgentExecutor, registerDBOSAgentWorkflows } from '@helix-agents/runtime-dbos';
import { PostgresStateStore } from '@helix-agents/store-postgres';
import { RedisStreamManager } from '@helix-agents/store-redis';
import { VercelAIAdapter } from '@helix-agents/llm-vercel';
import { openai } from '@ai-sdk/openai';

const agent = defineAgent({
  name: 'assistant',
  systemPrompt: 'You are a helpful assistant.',
  tools: [],
  llmConfig: { model: openai('gpt-6-luna') },
  maxSteps: 10,
});

// 1. Register workflow bodies (evaluates @DBOS.workflow() decorators). Idempotent.
registerDBOSAgentWorkflows();

// 2. Launch DBOS (begins async recovery of pending workflows).
await DBOS.launch();

// 3. Construct the executor. This binds every @DBOS.step's static deps
//    (state store, stream manager, LLM adapter, hooks, etc.) and calls
//    registerDBOSAgentWorkflows() again internally (idempotent).
const executor = new DBOSAgentExecutor({
  stateStore: new PostgresStateStore({ connectionString: process.env.DATABASE_URL! }),
  streamManager: new RedisStreamManager({ url: process.env.REDIS_URL! }),
  llmAdapter: new VercelAIAdapter({ model: openai('gpt-6-luna') }),
  // runStartFailSafeMs: 120_000, // run-start fail-safe (default)
});

// 4. Execute. sessionId is REQUIRED on the DBOS runtime.
const handle = await executor.execute(agent, { message: 'Hello' }, { sessionId: 'session-1' });
const result = await handle.result();

Constructor config differs from { registry }

Unlike Temporal's AgentRegistry-based wiring, DBOSAgentExecutor takes { stateStore, streamManager, llmAdapter } directly. There is no registry argument — agents are registered lazily (per execute() / resume() / retry() call) via the internal registerAgent helper. The { registry } snippet in some older overview tables is not the actual constructor shape.

Registering agents for recovery ​

DBOSAgentExecutor.execute() / resume() / retry() register the agent internally before starting a workflow. You only need to call registerAgent explicitly when a process may host a recovered workflow on DBOS.launch() without first going through one of those calls — for example, a process whose only job is delivering executor.submitToolResult(...):

typescript
import { registerAgent } from '@helix-agents/runtime-dbos';

// Before DBOS.launch(), in a submit-only / recovery-only process:
registerAgent(agent);

registerAgent is idempotent. If the same agent.name is later re-registered with a different live LanguageModel, it logs a warning and overwrites (last writer wins) to surface config drift.

bindCallLLMStep ​

bindCallLLMStep(llmAdapter, streamManager) wires the LLM adapter into the DBOS CallLLMStep static slot. Application code should not call this directly — the DBOSAgentExecutor constructor calls it for you from your config. It is exported only because e2e test infrastructure needs to swap the adapter between runs.

Execution Model ​

The DBOS runtime is architecturally closest to the Temporal runtime — both derive workflow IDs from sessionId, wrap I/O in durable step functions, and checkpoint intermediate results. The key architectural difference is the suspension primitive: where the v7 stateless-suspension runtimes (JS, Temporal, Cloudflare) use a durable-state suspensionContext and exit the workflow at every HITL boundary, DBOS uses its native DBOS.recv / DBOS.send primitives over Postgres-backed workflow replay. The workflow suspends in-place on await DBOS.recv(toolCallId) and wakes when a submission is delivered via DBOS.send. This is functionally equivalent for callers but architecturally separate from the unified suspensionContext model.

Standard vs Persistent Mode ​

DBOS supports two operational modes. The mode is fixed at the first execute() for a session and cannot change (mixing throws DBOSModeMismatchError).

Standard mode (default)Persistent mode
Workflow lifetimeOne short-lived workflow per execute() / resume() / retry(), runs to completion then exitsOne long-lived DBOS.recv()-loop workflow per session, forever
Workflow IDagent__{name}__{sessionId}__run__{runId}; resume() and retry() use …__resume__{runId}agent__{name}__{sessionId}; a restart after a terminal turn …__p__{runId}
Multi-turnEach turn starts a fresh workflow that loads accumulated stateSubsequent turns route into the existing workflow via DBOS.send(workflowId, msg, 'inbox')
HibernationN/A (workflow exits between turns)Blocks on DBOS.recv — zero compute, instant wake via Postgres LISTEN/NOTIFY
Best forRequest-response, batch, long analysesChat assistants, agents holding resources/locks across turns

Opt into persistent mode per-call ({ mode: 'persistent' }), per-agent (defaultMode: 'persistent'), or rely on the framework default ('standard'). Other runtimes ignore mode / defaultMode — they are no-ops there. See the persistent-mode deep-dive for the full lifecycle, idle-TTL semantics, and interrupt-vs-abort behavior.

Crash Recovery & Determinism ​

On process restart, DBOS replays the workflow body from the last completed step. For replay to be correct, the workflow body must produce the same sequence of step calls in the same order every time:

  • Inside the workflow body: use only pure functions (planStepProcessing, buildMessagesForLLM, shouldStopExecution from @helix-agents/core), DBOS.now(), DBOS.randomUUID(), and await @DBOS.step() calls. Avoid Date.now(), Math.random(), crypto.randomUUID(), fetch(), and process.env reads.
  • Inside a @DBOS.step(): non-determinism is fine. LLM calls, Postgres/Redis I/O, tool execution, Date.now(), and crypto.randomUUID() are all allowed because the step result is checkpointed and replayed from cache.

Two independent checkpoint layers coexist: the DBOS step cache (replay optimization, in DBOS system tables) and Helix checkpoints (user-facing time-travel / branching / retry, in store-postgres). They do not conflict.

Run start and ownership ​

Every entry (execute(), resume(), retry(), a recv-driven continuation, a child or companion turn) starts its run in the workflow's first step: the run-start commit, which appends the entry's input rows, applies any rewind (from_checkpoint, retry()), and creates the run record and its input checkpoint atomically. Nothing is written before it, so a crash between execute() and the workflow start leaves no orphan run, and an entry that loses writes nothing (no input row, no resumeCount, no stream change). Only the winner claims the stream, makes it live and writes run_started.

  • Run ids and owner. The executor mints the run id (a UUID) and names the workflow after it, so naming it writes nothing. A run's ownerToken is its workflow id (one execution). An entry into a session whose current run's owner workflow is still pending is refused with AgentAlreadyRunningError; once the owner workflow is no longer live, the next entry supersedes the stale run.
  • The call waits for the outcome. execute() / resume() / retry() return once their workflow's run-start has an outcome: a winner gets its handle, a loser the typed AgentAlreadyRunningError (RunStartConflictError) or RunStartRejectedError, synchronously.
  • Fail-safe. With no outcome within runStartFailSafeMs (default 120 000 ms), the call gives the entry up atomically with the store (abandonRunStart), cancels the workflow and throws core RunStartAbandonedError (transport_error, retryable; nothing was persisted), or RunStartOutcomeUnknownError (non-retryable: read the session before retrying) when the tombstone could not be written. Both are exported by @helix-agents/core; they replace the DBOS-only RunStartFailSafeError.
  • Hooks. A run that fails at its entry (the run-start commit, the stream claim or run_started) fires no lifecycle hook. onAgentFail fires only for a run whose onAgentStart / onAgentResumed fired, keyed on a durable start record (agent-start-fired-<runId>).
  • resumeCount is bumped by the winning resume() / retry() entry, never by a loser.
  • Superseded runs. Every write carries the run id and is fenced to it; a tool, spawn or registration first checks that the run still owns the session. A run that lost its session stops with AgentResult.status === 'superseded' (errorDetail.code === 'state_run_superseded') and writes nothing more; the error is excluded from step retries. onAgentComplete / onAgentFail do not fire (one exception: a supersede that lands after onAgentComplete fired and before the run's terminal commit leaves onAgentComplete then onAgentSuperseded; the last terminal hook is authoritative); onAgentSuperseded does.
  • Retries exhausted. A step that runs out of DBOS retries (DBOSMaxStepRetriesError) — the LLM call, a tool, a commit, any step — settles the turn: the terminal commit with failed and the last attempt's error, the run failed, one error chunk, the failed stream and onAgentFail once. It used to leave the session and its stream active.

Conversation history and checkpoints ​

  • Complete history. getMessagesStep loads the whole log with core loadLLMHistory. It used to call getMessages({ limit: 10_000 }), which returned the oldest 10,000 messages of a longer session (RM-11). A short read fails the run with errorDetail.code === 'state_history_incomplete' (retryable: false) and no model call. A store throw while loading is retryable: the step re-attempts the load up to 3 times (1 s, then 2 s back-off), so a transient Redis / Postgres blip does not fail the run; only when every attempt throws does the run fail with the same code and retryable: true.
  • Step-atomic commits. A step's assistant message, its tool results, its state writes and its checkpoint land in ONE commit (step), with the live stream head as its streamSequence; tool results are staged and committed in the step's promote, never appended outside it (RM-39, RM-40). Every checkpoint's messageCount is the live row count (RM-12).
  • Terminal checkpoints. Every terminal exit — completion, failure and stop — writes the terminal commit (a state checkpoint with the status, the error and the terminal hooks' writes) after the terminal hooks and before the run status and the stream end. Earlier releases wrote none on a failure or stop (ruling R28); user ruling D1 changed that. retry() never targets it (see below). DBOS currently treats a content_filter / refusal / max_tokens / error / unknown finish of an agent without outputSchema as completed, not failed (DI-26).
  • No content-less rows. retry() and resume() start the workflow with no input, and the body appends nothing for it (RM-28). Previously each retry/resume wrote a { role: 'user', content: undefined } row.
  • resume({ mode: 'from_checkpoint', checkpointId }) resolves the checkpoint first (another session's id throws state_checkpoint_foreign, an unknown id state_checkpoint_not_found, before any state change), then the new workflow's run-start commit performs the rewind atomically with the new run (startRun.truncateTo, CP-32). If the session is suspended on a client tool with its workflow still waiting in DBOS.recv, that workflow is cancelled first and a fresh one starts instead of attaching to it. Discarded pending calls get an aborted tool_end, and the winner writes one stream_resync (RM-60). A completed or failed session is rewound too, and a workflow starts from the checkpoint (C7; it used to throw an untyped "not in a resumable state" error).
  • Branch with checkpointId resolves it against the source session before cloneSession (which does not check ownership).
  • retry() honours RetryOptions (CP-30). A retry is a run-start commit in a new …__resume__{runId} workflow. By default it targets the failed run's last committed step (pass message, or the retry throws validation_error), or, when the run failed on its first step, its run-start commit (the triggering message is re-run, then message if given), through core planRetry, as on every runtime. checkpointId is restored as given (truncateTo). It used to ignore RetryOptions and continue from the persisted history. See JS runtime: retry() for the full rules.
  • Cost: the loaded history is a DBOS step output, so it is written to the DBOS system database once per step. For very long sessions this grows as O(N²); see FU-B2-01 in docs/dev/follow-ups.md.

Drain in-flight workflows before deploying this release

This release changes the workflow input (runMeta became entry) and the step sequence (the run-start commit is the first step, steps were added and reordered, a step commits its tool results in its promote, terminal turns add the terminal commit), and workflow ids changed (…__resume__{runId}). No DBOS.patch marker guards the change, so a workflow that is pending when you deploy cannot replay on the new code (nor the reverse, so a rollback needs the same drain). A paused (suspended) session is a pending workflow blocked in DBOS.recv, so it counts as in-flight too. Let running and suspended workflows finish, or cancel them, before rolling out. Persistent-mode session workflows never finish on their own: cancel them (DBOS.cancelWorkflow); the next execute() on the session starts a fresh workflow. See the deploy checklist (and, for the previous release, the v0.50 migration guide).

Multi-Turn Conversations ​

In standard mode, multi-turn works exactly as on the JS / Temporal runtimes: call execute() again with the same sessionId after the previous run completes. Each call starts a new workflow that loads accumulated state.

typescript
await (
  await executor.execute(agent, { message: 'My name is Alice.' }, { sessionId: 's1' })
).result();

// Continue the conversation — same sessionId, new run, accumulated state.
const h2 = await executor.execute(agent, { message: 'What is my name?' }, { sessionId: 's1' });

A turn started while the previous run is still finishing (CP-85)

The new turn's run-start winner claims the session's stream (a compare-and-set), and the previous run's endStream / failStream / pauseStream are fenced to its own run id, so a finishing run no longer ends the new turn's stream. The finding record stays open until its conformance cell lands; see CP-85 in the conformance findings.

In persistent mode, subsequent execute() calls route the message into the already-running workflow rather than starting a new one.

Routed turns skip structured-output validation (MA-21)

A turn delivered through the persistent workflow's inbox (a routed follow-up execute(), or a companion sendMessage to a persistent child) runs without the agent's outputSchema. For an outputSchema agent, __finish__ arguments are then saved unvalidated, and a turn that ends without calling it completes with no output instead of entering forced completion. The first turn and a message drained at idle timeout are not affected. See MA-21 in the conformance findings.

An interrupt right after execute() (CP-79, CP-57)

The user's input is now part of the run-start commit, the workflow's first step, and execute() returns its handle only once that commit has an outcome, so an interrupt through the returned handle can no longer drop the turn's input (it used to be persisted several steps in, CP-79). interrupt() on that handle, and on a getHandle() handle, records the stop on the run (WA MR2), sends the workflow its control message, cancels it (DBOS.cancelWorkflow) and then, after 500 ms, settles the run by the stop (interrupted; a K5 that landed first stands). Whether result() now reports interrupted rather than failed (CP-57) is re-checked by its conformance cell. The finding records stay open until their cells land; see the interrupt guide and the conformance findings.

abort() with no live run. An abort over a session with no live run has nothing to stop. This covers an interrupted session, a paused one whose run already ended, and a finished one. The abort answers not_stoppable and writes nothing: the session keeps its status, so an interrupted session stays resumable (logged runtime-dbos: abort not_stoppable). Only the handle's own named workflow is cancelled, best-effort. The exception is a persistent loop idling between turns whose workflow is still open: the abort sends it a shutdown, cancels it and marks the session failed, so the next execute() starts a fresh loop. An abort with no reason records core's DEFAULT_ABORT_REASON ('Aborted by user'), and an interrupt records DEFAULT_INTERRUPT_REASON.

Human-in-the-Loop (HITL) ​

DBOS supports the full HITL surface on its DBOS-native suspension primitive.

Client-Executed Tools ​

A tool declared with execute: 'client' suspends the run while the client computes the result. The DBOS flow:

  1. The LLM emits a client-tool call. The workflow suspends in a fixed order: the run's suspended_* status, then the pending calls' frames (tool_start / tool_approval_request), then the suspension commit (pendingClientToolCalls[toolCallId] with the step's rows and checkpoint), then pauseStream (run_paused). The stream's paused status means the suspension is committed. onAgentSuspended fires after the commit.
  2. The workflow body calls await DBOS.recv(toolCallId, deadlineSec) and suspends. The process can be restarted here — the recv re-arms on replay.
  3. The client posts to POST /submit-tool-result; the server calls executor.submitToolResult(...), which routes a DBOS.send(workflowId, payload, toolCallId) to the owner workflow.
  4. The recv returns; the workflow clears pending state, appends a tool-result message, and the loop continues.

Submission is exactly-once: concurrent or retried submits on the same toolCallId are serialized by an OCC-stamped submittedAt marker (first writer wins; losers return already_completed), and the DBOS.send idempotency key is scoped by workflow ID. A durable completedClientToolCalls marker makes already_completed survive executor restart and persistent-workflow exit. Deadline timeouts append a synthetic failed tool result ({"error": ...} with the humanized client_tool_timeout text, helixToolFailed: true), emit a tool_end chunk, and fire the timeout hooks — matching the cross-runtime contract.

A lapsed deadline is reported, not skipped. Once its suspension is complete (after pauseStream), the run publishes it as a DBOS event. When every suspended call's deadline lapses unanswered, the workflow continues the session inline as a new run with no caller action. The suspended run's handle still reports suspended_client_tool for it, even when that continuation finished before the handle's pending-client-tool poll looked, and a plain resume() afterwards returns the continuation's outcome (as a JS or Temporal resume after the deadline drains the expired calls). An answered round keeps the auto-continue contract: handle.result() reports the final outcome.

See the client-tools deep-dive for sub-agent ownership routing and the full race-condition analysis.

Approval-Gated Tools ​

A tool with requireApproval: true (or a function-form predicate) suspends on a tool_approval_request chunk and shares the same pendingClientToolCalls + DBOS.recv primitive as client tools. Resume by calling executor.submitToolResult({ kind: 'approval-response', toolCallId, approved, reason? }):

  • Approve (approved: true) runs the tool's execute() with the original input.
  • Deny (approved: false) persists a synthetic failed tool result ({"error": "Tool call was not approved by the user"}, metadata.helixToolFailed: true) and skips execute(). Other runtimes currently persist the deny differently (CP-66).

Known regression: approved tools do not stream live through handleChatStream (W1 MR2b)

Until FU-DBOS-CONTINUATION-ENTRY-ORDER lands, an approved tool runs under the suspended run, before the continuation run's run_started, and handleChatStream's submit attach goes live only on that run. So (a) the approved tool's live frames (state_patch, custom ctx.emit events, subagent_*) are not streamed: they reach the client only through the snapshot (the re-anchor, retry() or a reload); and (b) an approved tool that runs past the 30 s live-wait bound gets the non-retryable 500 run_not_live, and the client must call retry(). Client-tool continuations are unaffected. This is a regression MR2b introduces.

Checkpointed requireApproval predicate: function-form predicates are evaluated inside a @DBOS.step (ApprovalGateStep.evaluateApprovalGatePredicateStep), so the boolean result is checkpointed in workflow history. This makes the suspend-vs-run decision replay-deterministic even for non-pure predicates (a predicate that reads the clock or a DB row is safe). Fail-closed semantics live inside the step — an exception is treated as requireApproval = true — so the checkpointed value is always a boolean (parity with Temporal's activity-wrapped predicate evaluation; GL #111 Batch C).

Driving the loop after submitToolResult ​

submitToolResult delivers the answer to the parked workflow (DBOS.send → its DBOS.recv), which continues the turn itself: a complete batch answers continuation: { kind: 'new_run', runId } (W1 MR2b): the run the woken workflow continues the turn in, <suspended run>:continuation (or the current run once that continuation committed); caller_resumes only when no run is current, a partial one awaiting_batch. The caller attaches to the run's stream instead of calling resume(): handleChatStream does this for you (with useChat / useResumeClientTools); a direct caller reads the session stream or getHandle(). See Hooks for how the continuation run links to the suspended one.

Resume before submit is refused (R44). resume() in continue / with_message mode on a session whose client-tool batch is still partial (some awaited call has no submitted result) throws a retryable state_not_resumable and writes nothing. It used to resume while a call was still awaited, and the late result was lost. Instead:

  • submit every result of the batch (submitToolResult); once it is complete the run continues, or let a call's deadline lapse (a lapsed deadline counts as its timeout result), then resume if needed;
  • to abandon the batch, call interrupt() (it ends the paused run at once, interrupted) or submit error results for the calls;
  • a from_checkpoint rewind that discards the unanswered calls is allowed.

A submit for a call of a stopped run (WA MR2 §7.1). The submit reads the stop of the call's run first and answers { status: 'unknown_tool_call', reason: 'run_stopped' }; nothing is stored or sent to the workflow. A stop that is recorded but not yet resolved is settled by this submit: the parked workflow is signalled and cancelled, and the run is settled by the stop after a fixed wait (DEFAULT_STATUS_UPDATE_DELAY_MS, 500 ms, for a step already in flight), so a submit over a stopped run whose owner workflow is still live (parked in recv) answers about 500 ms later than an ordinary submit (FU-DBOS-STOP-CANCEL-EXECUTING-OWNER). A stop that already ended the run answers at once.

See the Client-Executed Tools and Approval Gates guides for cross-runtime concepts.

Hooks ​

Per-call hooks fire on the DBOS runtime with full parity. agent.hooks (the AgentConfig field), plus ExecuteOptions.hooks and ExecuteOptions.hookManager, fire on every execute() / resume() / retry(). The executor computes the merged HookManager (constructor → agent.hooks → options.hooks order) before DBOS.startWorkflow and registers it in a process-local HookManagerRegistry indexed by DBOS.workflowID. The hook and tool steps resolve the manager per-invocation, falling back to the constructor-bound static on cross-worker recovery (a documented split-brain risk mirroring Temporal/CF replaceAgent semantics).

Canonical firing order matches all five runtimes (JS, Temporal, Cloudflare Workflows, Cloudflare DO, and DBOS — the first four use the v7 stateless-suspension model; DBOS uses its own DBOS-native DBOS.recv/DBOS.send primitives but converges on the same observable hook order):

beforeTool → execute → onStateChange → onMessage → afterTool

beforeTool + onStateChange fire inside the @DBOS.step tool body (so they are checkpointed by the step boundary); onMessage + afterTool are deferred to the workflow body so they land after the phase-1 Promise.all settles and after each tool-result message is appended.

onMessage, onStateChange, onAgentSuspended, and onAgentResumed all fire — including for both client-tool and approval-gate suspend/resume cycles.

onAgentResumed.previousRunId ​

onAgentResumed fires with { runId, previousRunId, sessionId, resumedFromCheckpointId }. DBOS has two resume branches:

  • Branch 2 (fresh resume workflow): runId !== previousRunId — a new run is allocated and a fresh workflow starts (same as JS / Temporal / CF).
  • Branch 1 (recv-driven continuation in the same workflow): the client-tool submit path detects a still-PENDING workflow blocked on DBOS.recv, returns a handle wrapping the same workflow, and the body wakes there. No new workflow. A client-tool / approval round still continues as a new run: the woken workflow writes a run-start commit (a continuation entry carrying the answered calls) and runId !== previousRunId. When only children were awaited, the same run continues and runId === previousRunId.

In both branches previousRunId is always populated with the suspended run's runId. Hook consumers should use previousRunId as the linkage signal for span stitching — do not rely on runId !== previousRunId to detect a resume, because that test fails when only children were awaited.

See the Hooks (Observability) guide for the full hook reference.

State ​

stateSchema.default() values are seeded into customState at session start so tools that append to a key (e.g. customState.items via an Immer /items/- push) see the default-initialized container rather than undefined. Seeding is best-effort and validated; dropped keys log a warning.

Parallel-tool in-step state visibility ​

Each DBOS tool runs as an independent durable step that loads state.customState fresh from Postgres at the top of its execution. A parallel sibling tool that calls ctx.getState() to read a key written by another sibling within the same LLM step sees the pre-sibling value, not the cumulative state. This is by design — DBOS steps must be idempotent for replay determinism.

Tools that need to read sibling writes within a single step should either be sequenced through a single parent tool that orchestrates the work, or run on the JS / Cloudflare DO runtimes (which apply in-memory state merging between parallel tool calls before persisting).

Remote Sub-Agent Handling ​

DBOS is a first-class consumer of remote sub-agents: a parent running on DBOS can delegate to a child agent hosted on a separate HTTP or Cloudflare service via createRemoteSubAgentTool. Remote calls run through a dedicated durable step (executeRemoteSubAgentCall), separate from both regular tool steps and local sub-agent handling.

The step provides:

  • Deterministic session IDs — {parentSessionId}-remote-{toolCallId}, so a replayed step re-addresses the same child instead of starting a new run.
  • Crash recovery — on step retry it checks transport.getStatus() to avoid re-executing an already-completed child (Postgres-backed workflow replay).
  • Ref-cadence reconnection — seeds lastSequence from the persisted SubSessionRef at step entry and persists it every N proxied chunks (and on every failure path), so a retry resumes the SSE stream mid-flight rather than replaying from zero.
  • Stream proxying — the remote child's chunks flow through the parent's stream, stamped with remoteSource provenance.
  • SubSessionRef tracking — registers with remote metadata.
  • Interrupt propagation — interrupts abort the in-flight remote call, and StreamDropError triggers step retry.
  • State & usage fold — on completion the child's terminal customState and usage rollup fold into the parent via the shared processRemoteSubAgentCompletion path (output is re-validated against the tool's outputSchema).

Client-executed and approval-gated tools inside a remote child are not supported across the service boundary (a v1 limitation shared by all runtimes). See the Remote Agents guide for setup and the cross-service guide for the state/usage/ reconstruction contracts.

Persistent Sub-Agent Handling ​

DBOS supports persistent sub-agents: a parent that declares persistentAgents gets the same companion__* tools as every other runtime, with identical LLM-facing semantics (terminate-truth, exactly-once completion delivery, deterministic child ids).

Re-spawning a completed persistent child continues it on its preserved session (memory retained) rather than recreating it — see Re-consulting a persistent companion (the critic loop).

  • Dispatch. companion__listChildren / getChildStatus / terminateChild route through the shared core dispatcher. spawnAgent, sendMessage, and waitForResult are handled by DBOS-local logic so they can use durable primitives: a child runs as its own durable workflow (startPersistentWorkflow), sendMessage appends to the child's durable inbox via DBOS.send, and waitForResult polls with durable DBOS.sleep (the wait survives a crash).
  • Delivery is truthful. companion__sendMessage reports delivered: true only when the message reached the child (its run-start committed with the message as its input row, or the child's inbox loop took it). Otherwise the result is delivered: false with a typed reason — child_busy (another run owns the child), child_exited (its loop exited before taking the message), child_not_running, child_start_unconfirmed (the child's run-start outcome did not arrive in time; the message may still be delivered) or child_entry_failed — so the model can retry. A consult that reopens a completed child and loses or fails its start returns an error result instead of claiming a spawn.
  • Terminate is truthful. companion__terminateChild records an abort stop on the child's current run and wakes its loop with a shutdown. It reports terminated: true only once the stop is recorded. A failure after the record, such as the stop's terminal or the shutdown send, is logged and the terminate still counts, because the record ends the child. A stop that was not recorded is a tool failure, and the parent's ref goes back to what it was. For an idle loop with no live run, the shutdown is the stop, so a failed send is a tool failure.
  • Completion delivery. A completion notifier (a @DBOS.step) injects a finished non-blocking child's outcome into the parent's next turn, deduplicated by the durable completionDelivered flag on the SubSessionRef.
  • Blocking spawn (caveat). Non-blocking spawn is fully supported. The blocking spawn path currently blocks until the workflow is idle and can mis-report a failed child as completed (tracked as FU-DBOS-BLOCKING-SPAWN-SEMANTICS). Prefer non-blocking spawn + companion__waitForResult on DBOS until that is resolved.
  • No workspaces on persistent children (C8). A persistent child that declares a workspace fails fast at spawn — workspaces are unsupported on DBOS (see below). Use inheritWorkspace-free children with no workspace, or run workspace-bearing children on the JS / Cloudflare DO runtimes.

Known Limitations ​

  • Approved tools do not stream live through handleChatStream (a W1 MR2b regression, until FU-DBOS-CONTINUATION-ENTRY-ORDER lands). (a) An approved tool's live frames (state_patch, custom ctx.emit events, subagent_*) are not streamed; they reach the client only through the snapshot (the re-anchor, retry() or a reload). (b) An approved tool that runs past the 30 s live-wait bound gets the non-retryable 500 run_not_live, and the client must call retry(). Client-tool continuations are unaffected. See Approval-Gated Tools.
  • Workspaces are unsupported. Unlike Temporal and CF Workflows (which fail-fast at run-start), DBOS silently passes workspaces: undefined. describeCapabilities() returns an empty workspaceProviderKinds list, so AgentServer surfaces a RUNTIME_NO_WORKSPACE_SUPPORT 404 on the HTTP workspace route. Agents needing workspaces should run on the JS or Cloudflare DO runtimes.
  • The history: input prefix is not honored. executor.execute(agent, { message, history: [...] }, { sessionId }) silently drops the history: prefix on DBOS — the message log starts at the new user message. Tracked as the open follow-up FU-DBOS-HISTORY-PREFIX-NEVER-HONORED (low priority; parity gap with JS / CF / Temporal).
  • Doubly-nested HITL is unsupported. A top-level agent's client-executed or approval-gated tool suspends and resumes correctly. The doubly-nested case — an ephemeral child sub-agent itself calling an execute: 'client' tool and expected to suspend the parent — is not supported; the parent returns 'completed' rather than suspending. Hoist the client tool to the top-level agent, or use a persistent sub-agent instead. Tracked in GitLab #73 (documented-as-unsupported).
  • Submit-side outputSchema validation is skipped. The DBOS agent registry stores SerializedAgent records, which strip Zod schemas, so submitted client-tool results are accepted without server-side schema validation. Client-side validators should be the primary source of correctness.
  • In-step parallel-sibling state visibility — see State above.
  • Memory auto-injection/extraction is not supported. The memory: config field is accepted at the type level but is never invoked on DBOS. Agents requiring memory recall or storage should run on the JS or Cloudflare runtimes.

Forced completion on the 5.5 generation still fails on DBOS (support planned)

DBOS does not resolve completion strategies yet. Every forced call uses the legacy request (a named tool choice on the one completion tool, a trimmed system prompt, reasoning disabled) whatever the adapter reports, so on Claude Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1, which reject forced tool choice, each forced call fails with HTTP 400. A forced call's provider error is still a recoverable miss here, so the run ends framework_forced_completion_exhausted with the provider's code lost. There is also no early refusal of an unrecognised model and no prompt-prefix check. DBOS also drops providerOptions and maxOutputTokens from LLMConfig across its step boundary (only temperature survives), so thinking and effort settings do not reach the model, and it renders a systemPrompt function once, with empty state, when the agent is serialized. Unlike the other runtimes, a skills provider that throws does not fail the run on DBOS: the workflow body logs a warn (dbos.skills.catalog_unavailable) and runs the turn without the skills catalog or any preloaded skill body (before 0.50 a getSkill failure for one preloaded name skipped only that body), because the body resolves the catalog outside any step and failing there could strand a recovering run. The planned DBOS support moves resolution into a step and fails closed (FU-COMPLETION-DBOS-SKILLS-RECOVERY-STRAND). An outputSchema agent on those models that finishes by calling __finish__ or its finishWith tool itself is not affected. Until DBOS support lands, run outputSchema agents on those models on runtime-js, a Cloudflare runtime or Temporal. Ordered thinking parts are still captured and replayed verbatim (that is adapter-side).

stopWhen is never evaluated on DBOS (DI-29)

The workflow calls the stop check (shouldStopExecution) with maxSteps only, so an agent's stopWhen predicate never runs: after it would have fired, the agent keeps calling the LLM and running tools until the model stops on its own or maxSteps is reached. Observed live (a stopWhen: () => true agent made 3 LLM calls where JS made 1). JS, the Cloudflare DO and Cloudflare Workflows honour it. Use a finishWith tool or an outputSchema to end the loop. See the stopWhen guide and DI-29 in the conformance findings.

A maxSteps cut-off completes on DBOS (DI-25)

An agent without outputSchema that runs out of maxSteps mid-tool-loop ends completed with no output and no error code here. JS, the Cloudflare DO and Cloudflare Workflows fail it with framework_max_steps_exhausted and let retry() continue it; on DBOS retry() refuses the session because it is not failed. Wiring core resolveStopDecision and the run's completionReason into the workflow is FU-CFD1C-DI29-DI25-DBOS (CF-D1 MR 3). See When maxSteps runs out.

A blocked, refused or truncated final reply completes the run (DI-26)

For an agent without outputSchema, the workflow never classifies a text step's stop reason. A final step that stops with content_filter, refusal, max_tokens, error or unknown ends the run completed with no error, where JS and both Cloudflare runtimes fail it. Observed live for content_filter; the other reasons follow from source. The run's completionReason does not flag it: DBOS writes 'failed' there for every text answer, ordinary ones included (DI-27). Agents with an outputSchema are not affected, except on persistent-mode follow-up turns, which run without the schema (MA-21, above). See the error handling guide and DI-26 in the conformance findings.

Stream parity in this release (RM-55, RM-56, RM-57, RM-58)

From source, the DBOS stream gaps observed on the previous release are closed:

  • The LLM step streams every chunk live as the model produces it — text_delta (also beside tool calls), thinking, tool-argument streaming, sources, files, step_end — through one writer stamped with the run and the attempt, like runtime-js and Temporal (RM-55). A retried step re-streams under a new attemptId; readers drop the superseded attempt.
  • A suspension pauses the stream (run_paused) after the suspension commit (RM-56).
  • An interrupt() that lands while a step's LLM call is in flight discards the step: nothing of it is committed and its content is marked step_discarded; a cancelled owner's settle writes the same marker (RM-57).
  • Every chunk carries the 1-based step number of the step commit it belongs to (RM-58).

The finding records stay open until their conformance cells land. See the stream protocol parity table and the conformance findings.

Prompt caching is supported. LLMConfig.cache strategies (anthropicCache, openaiCache, xaiCache, or a custom CacheStrategy) are applied on DBOS at parity with the JS, Temporal, and Cloudflare runtimes. Strategies are not JSON-serializable, so the executor registers the live strategy in a process-local registry (alongside the live model) and the workflow body applies it before each LLM call; a strategy that throws is logged and the step continues un-annotated.

Best Practices ​

  • Use store-postgres for state — it shares the same Postgres instance DBOS uses for its system database, keeping infrastructure to one service.
  • Use RedisStreamManager for streaming — Postgres write latency is too high for per-token chunk fan-out.
  • Register before launch — call registerDBOSAgentWorkflows() and construct the executor (or call registerAgent) before DBOS.launch() so recovered workflows find their live LanguageModel.
  • Keep workflow bodies deterministic — push all I/O into @DBOS.step() calls. See Determinism.
  • Choose persistent mode deliberately — only when subsequent messages need to arrive into a running workflow context (in-memory state across turns); otherwise standard mode is simpler.

See Also ​

These deep-dive docs live alongside the package source:

Doc-site cross-links:

Released under the MIT License.

Copyright 2024-2026 Helix AI