LLM Integration Overview

LLM adapters connect Helix Agents to language model providers. The adapter interface abstracts provider-specific APIs, allowing you to switch models without changing agent code.

The LLMAdapter Interface

Every LLM adapter implements the LLMAdapter interface:

interface LLMAdapter {
  generateStep(input: LLMGenerateInput): Promise<StepResult>;

  // Optional: what the configured model supports (see Model capabilities below).
  // Absent ⇒ the `legacy` profile, exactly the behaviour before capabilities existed.
  resolveCapabilities?(config: LLMConfig): CapabilityResolution;

  // Optional: can this schema be sent as native structured output to this model?
  // Absent ⇒ always ok. Called at run entry (early refusal), never during a step.
  checkOutputFormat?(
    format: LLMOutputFormat,
    config: LLMConfig
  ): { ok: true } | { ok: false; reason: string };
}

The adapter is responsible for:

1. Message Conversion - Translating framework messages to provider format, replaying an assistant message's ordered parts verbatim when present (see Preserved thinking)
2. Tool Conversion - Converting Zod schemas to provider tool format
3. Streaming - Invoking callbacks for real-time chunk delivery
4. Response Parsing - Converting provider responses to StepResult
5. Stop Reason Mapping - Normalizing provider finish reasons
6. Capabilities (optional) - Telling the framework which completion strategy the model supports

Available Adapters

Adapter	Package	Providers
VercelAIAdapter	@helix-agents/llm-vercel	OpenAI, Anthropic, Google, Cohere, Mistral, and more

The Vercel AI SDK adapter is the recommended choice for most applications, as it supports a wide range of providers through a unified interface.

LLMGenerateInput

The input passed to generateStep():

interface LLMGenerateInput {
  // Conversation history (including system prompt)
  messages: Message[];

  // Available tools (framework format, adapter converts)
  tools: Tool[];

  // LLM configuration
  config: LLMConfig;

  // Optional cancellation
  abortSignal?: AbortSignal;

  // Streaming callbacks
  callbacks?: LLMStreamCallbacks;

  // Agent context
  agentId: string;
  agentType: string;

  // Forced-completion directive fields — omitted on ordinary calls, set by the
  // framework during forced completion (see Finishing Agents → Forced completion).
  executionPhase?: ExecutionPhase; // 'work' | 'forced_completion' — omitted ⇒ 'work'
  // `forced_tool` strategy only:
  toolChoice?: LLMToolChoice; // 'auto' | 'none' | 'required' | { type: 'tool'; toolName: string }
  reasoningMode?: ReasoningMode; // 'default' | 'disabled' — omitted ⇒ 'default'
  // `native_output` strategy only:
  outputFormat?: LLMOutputFormat; // { type: 'json_schema', name, schema: ZodType, description? }
  minOutputTokens?: number; // use max(config.maxOutputTokens ?? default, minOutputTokens)
}

An adapter must honour every directive field it receives: forward a named toolChoice to the provider, and send outputFormat as the provider's native structured output (a response constrained to the JSON schema of schema). Never fall back silently (auto instead of a named tool, plain text instead of the format): report what the model supports through resolveCapabilities instead, and the framework sends each model only a directive it can honour.

LLMConfig

LLM configuration options:

interface LLMConfig {
  // Required: The language model to use
  model: LanguageModel;

  // Generation parameters
  maxOutputTokens?: number;
  temperature?: number; // 0-2
  topP?: number;
  topK?: number;
  presencePenalty?: number;
  frequencyPenalty?: number;

  // Control parameters
  stopSequences?: string[];
  seed?: number; // For deterministic outputs
  maxRetries?: number; // Default: 3

  // Prompt caching (opt-in; pass a provider-specific strategy)
  cache?: CacheStrategy | CacheStrategy[];

  // HTTP configuration
  headers?: Record<string, string>;

  // Provider-specific options
  providerOptions?: Record<string, Record<string, JSONValue>>;
}

Prompt Caching

Prompt caching is opt-in. Pass a provider-specific strategy helper from @helix-agents/core that matches your model:

import { anthropicCache, openaiCache, xaiCache } from '@helix-agents/core';

// Anthropic
const agent = defineAgent({
  name: 'my-agent',
  systemPrompt: 'You are a helpful assistant.',
  llmConfig: {
    model: anthropic('claude-sonnet-5-5'),
    cache: anthropicCache({ ttl: '1h' }),
  },
});

// OpenAI
llmConfig: { model: openai('gpt-6-luna'), cache: openaiCache() }

// xAI/Grok
llmConfig: { model: xai('grok-3'), cache: xaiCache() }

// Google Gemini — no strategy needed; caching is built-in
llmConfig: { model: google('gemini-2.5-pro') }

Each helper applies the provider-specific mechanism before each LLM call:

Provider	Mechanism	What the Strategy Does
Anthropic	Explicit cache_control markers	Places breakpoints on the system prompt (which also covers tool definitions) and the latest turn's tool-result batch
OpenAI	Implicit prefix caching with routing hints	Sets promptCacheKey from sessionId for cache affinity
Google Gemini	Automatic prefix caching	No action needed (strategy not required)
xAI/Grok	Implicit with conversation routing	Sets x-grok-conv-id header from sessionId

Cache token usage is tracked through the standard usage tracking system and Langfuse tracing.

See the Vercel AI SDK Adapter - Prompt Caching section for detailed provider examples.

Provider Options

Enable provider-specific features:

// OpenAI reasoning summaries (reasoning models, e.g. gpt-6-luna)
const config: LLMConfig = {
  model: openai('gpt-6-luna'),
  providerOptions: {
    openai: {
      reasoningSummary: 'detailed',
      reasoningEffort: 'high',
    },
  },
};

// Anthropic extended thinking (Claude)
const config: LLMConfig = {
  model: anthropic('claude-sonnet-5-5'),
  providerOptions: {
    anthropic: {
      thinking: { type: 'adaptive' },
    },
  },
};

Streaming Callbacks

Real-time streaming during generation:

interface LLMStreamCallbacks {
  // Called for each text token
  onTextDelta?: (delta: string) => void | Promise<void>;

  // Called for thinking/reasoning content
  onThinking?: (content: string, isComplete: boolean) => void | Promise<void>;

  // Called when LLM requests a tool
  onToolCall?: (toolCall: ParsedToolCall) => void | Promise<void>;

  // Called on generation errors
  onError?: (error: Error) => void | Promise<void>;
}

These callbacks enable real-time UI updates as the model generates content.

StepResult

The response from generateStep():

type StepResult = TextStepResult | ToolCallsStepResult | ErrorStepResult;

Text Response

interface TextStepResult {
  type: 'text';
  content: string;
  thinking?: ThinkingContent;
  parts?: AssistantMessagePart[]; // ordered reasoning / text / tool-call parts
  shouldStop: boolean;
  stopReason: StopReason;
}

Tool Calls

interface ToolCallsStepResult {
  type: 'tool_calls';
  toolCalls: ParsedToolCall[];
  subAgentCalls: ParsedSubAgentCall[];
  content?: string; // Text accompanying tool calls
  thinking?: ThinkingContent;
  parts?: AssistantMessagePart[]; // ordered reasoning / text / tool-call parts
  shouldStop: false;
  stopReason: StopReason;
}

Structured Output

A structured-output completion is a tool_calls step carrying the __finish__ call (FINISH_TOOL_NAME), whose arguments are the output. The adapter returns it like any other call, with every call co-emitted with it and the step's text. Core then decides the step:

• the first __finish__ completes the run, keeps its assistant message, and gets the acknowledged result;
• every other call in the step gets a typed not_executed result (superseded_by_terminal_call);
• a forced completion call carrying extra calls is a recoverable protocol miss.

No step result ends a run without an assistant message (RM-35).

Error

interface ErrorStepResult {
  type: 'error';
  error: Error; // always the original LLM error
  shouldStop: boolean;
  stopReason: StopReason;
  onErrorFailure?: Error; // the adapter's awaited onError rejected
}

onErrorFailure is set when the adapter's awaited callbacks.onError rejected while reporting error — typically because the run's stream had already ended, so the error chunk could not be written. It never replaces error (the runtime's failure handling still acts on the LLM error); every runtime logs it at warn as llm.on_error_callback_failed. Adapters build it with reportLLMError (see Custom Adapters).

Stop Reason Normalization

Different LLM providers return different finish reasons. The framework normalizes them:

type StopReason =
  | 'end_turn' // Natural completion
  | 'tool_use' // Tool calls requested (not a stop)
  | 'max_tokens' // Token limit reached (FAILS agent)
  | 'content_filter' // Safety filter (FAILS agent)
  | 'refusal' // Model refused (FAILS agent)
  | 'stop_sequence' // Stop sequence hit (completes normally)
  | 'error' // Generation error (FAILS agent)
  | 'unknown'; // Unrecognized (FAILS agent)

Error stop reasons complete the run on Temporal and DBOS (DI-26)

The documented failure for these reasons holds only on JS, the Cloudflare DO and Cloudflare Workflows. For an agent without outputSchema, Temporal and DBOS end a run whose final text step stops with content_filter, refusal, max_tokens, error or unknown as completed with no error. Observed live for content_filter on both and for max_tokens on Temporal; the other reasons follow from source. Agents with an outputSchema are not affected, except on DBOS persistent-mode follow-up turns, which run without the schema (MA-21). See DI-26 in the conformance findings.

Provider Mappings

Framework	OpenAI	Anthropic	Vercel AI SDK
end_turn	stop	end_turn	stop
tool_use	tool_calls	tool_use	tool-calls
max_tokens	length	max_tokens	length
content_filter	content_filter	-	content-filter
refusal	-	refusal	-

Terminal vs Continuation

Stop reasons determine agent behavior:

// Terminal conditions (agent fails)
function isErrorStopReason(reason: StopReason): boolean {
  return (
    reason === 'max_tokens' ||
    reason === 'content_filter' ||
    reason === 'refusal' ||
    reason === 'error' ||
    reason === 'unknown'
  );
}

// Recoverable errors (retryable for outputSchema agents)
function isRecoverableErrorStopReason(reason: StopReason): boolean {
  return reason === 'max_tokens';
}

// Continue execution (not a real stop)
// tool_use -> execute tools, then continue

// Normal completion
// end_turn, stop_sequence -> agent completes successfully

Forced completion for max_tokens (and other triggers): When an agent has an outputSchema and the LLM returns text with a max_tokens stop reason — or ordinary terminal text (end_turn/stop_sequence) without a completion-tool call, or exhausts maxSteps mid-tool-loop — the framework does not fail immediately or silently drop the output. It enters a deterministic forced-completion phase: the next call(s) enforce the expected completion tool with the completion strategy the model supports (a named toolChoice, or native structured output on models that reject forced tool choice), bounded by maxCompletionRetries (default 2, independent of maxSteps). The phase either succeeds with schema-valid output or the run fails with a classified framework_* error — it never silently completes with missing output. Non-recoverable stop reasons (content_filter, refusal, error, unknown) still fail immediately without entering forced completion. See Finishing Agents → Forced completion for the full lifecycle and @helix-agents/core reference for the API.

Thinking/Reasoning Content

Some models provide reasoning traces:

interface ThinkingContent {
  content: string; // The reasoning text
}

Supported providers:

• Anthropic Claude - Extended thinking via providerOptions.anthropic.thinking
• OpenAI o-series - Reasoning summaries via providerOptions.openai.reasoningSummary

Thinking content is:

• Streamed via onThinking callback
• Stored in StepResult.thinking
• Persisted in message history (if your state store supports it)
• Streamed as thinking chunks to clients

Preserved thinking

On some models a thinking block is bound to the request that produced it: its signature covers the system prompt, the tool set and every earlier message, and a later request whose prefix differs is rejected (Anthropic's Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1, for accounts with enforcement on). Such a model needs every block replayed exactly as it was produced, in order, each with its own signature, including empty and redacted blocks.

StepResult and AssistantMessage therefore carry an optional ordered parts array:

type AssistantMessagePart =
  | { type: 'reasoning'; text: string; providerOptions?: Record<string, Record<string, unknown>> }
  | { type: 'text'; text: string; providerOptions?: Record<string, Record<string, unknown>> }
  | { type: 'tool-call'; toolCallId: string }; // refers into toolCalls

The representation is the AI SDK's: an Anthropic block's signature is providerOptions.anthropic.signature, a redacted block's data providerOptions.anthropic.redactedData. An adapter that captures parts must replay them verbatim when a message has them; content, toolCalls and thinking stay populated for every other consumer. The Vercel adapter records parts only for responses that contain reasoning. On models that bind thinking to history the framework also refuses a call whose system prompt or tool set changed (framework_prompt_prefix_changed); see Preserved thinking for the contract.

Model capabilities

Forced completion needs a mechanism that guarantees a structured final answer, and not every model supports the same one. An adapter reports what the configured model supports with the optional resolveCapabilities(config):

interface ModelCapabilities {
  forcedToolChoice: boolean; // a named tool choice can be forced
  nativeStructuredOutput: boolean; // the response can be constrained to a JSON schema
  historyBoundThinking: boolean; // thinking blocks bind the request prefix
}

type CapabilityResolution =
  | {
      status: 'resolved';
      capabilities: ModelCapabilities;
      profile: string; // e.g. 'legacy', 'anthropic-5.5' (recorded in chunks, hooks and traces)
      source: 'override' | 'table' | 'default';
    }
  | { status: 'unresolved'; reason: string }; // e.g. an unrecognised model id

Both methods are synchronous, pure and do no I/O, so they are safe inside Temporal activities, DBOS steps and Cloudflare step.do bodies. On runtime-js, the CF DO, CF Workflows and Temporal (inside the worker's runLLMStep activity), core resolves capabilities on every LLM call (a resolver that throws counts as unresolved) and picks the strategy for each forced call: forcedToolChoice → forced_tool, else nativeStructuredOutput → native_output, else framework_completion_strategy_unavailable. historyBoundThinking turns on the prompt-prefix check and keeps forced calls from changing the system prompt or tool set. Core never reads a model id to decide anything: model knowledge lives in the adapter. An adapter without resolveCapabilities gets LEGACY_CAPABILITIES ({ forcedToolChoice: true, nativeStructuredOutput: false, historyBoundThinking: false }, profile legacy), which is exactly the behaviour before capabilities existed. DBOS does not call resolveCapabilities / checkOutputFormat yet (support is planned): it always uses the legacy forced request, so an adapter's capabilities (and the Vercel adapter's capabilities override) have no effect there for now.

@helix-agents/llm-vercel ships a capability table for Anthropic models and an override for proxies and new models: see Vercel adapter → Model capabilities. For a custom adapter, see Custom adapters → Capabilities.

Composability

The adapter interface enables composability:

// Development: Use mock adapter
import { MockLLMAdapter } from '@helix-agents/core/testing/mocks';

const adapter = new MockLLMAdapter({
  responses: [{ type: 'text', content: 'Test response' }],
});

// Production: Use Vercel adapter
const adapter = new VercelAIAdapter();

// Both work with any runtime
const executor = new JSAgentExecutor(stateStore, streamManager, adapter);

You can swap adapters without changing agent definitions or runtime code.

Next Steps

• Vercel AI SDK Adapter - Using the recommended adapter
• Custom Adapters - Building your own adapter