LLM Integration Overview

LLM adapters connect Helix Agents to language model providers. The adapter interface abstracts provider-specific APIs, allowing you to switch models without changing agent code.

The LLMAdapter Interface

Every LLM adapter implements the LLMAdapter interface:

interface LLMAdapter {
  generateStep(input: LLMGenerateInput): Promise<StepResult<unknown>>;
}

The adapter is responsible for:

1. Message Conversion - Translating framework messages to provider format
2. Tool Conversion - Converting Zod schemas to provider tool format
3. Streaming - Invoking callbacks for real-time chunk delivery
4. Response Parsing - Converting provider responses to StepResult
5. Stop Reason Mapping - Normalizing provider finish reasons

Available Adapters

Adapter	Package	Providers
VercelAIAdapter	@helix-agents/llm-vercel	OpenAI, Anthropic, Google, Cohere, Mistral, and more

The Vercel AI SDK adapter is the recommended choice for most applications, as it supports a wide range of providers through a unified interface.

LLMGenerateInput

The input passed to generateStep():

interface LLMGenerateInput {
  // Conversation history (including system prompt)
  messages: Message[];

  // Available tools (framework format, adapter converts)
  tools: Tool[];

  // LLM configuration
  config: LLMConfig;

  // Optional cancellation
  abortSignal?: AbortSignal;

  // Streaming callbacks
  callbacks?: LLMStreamCallbacks;

  // Agent context
  agentId: string;
  agentType: string;

  // Forced-completion directive fields — omitted on ordinary calls, set
  // by the framework during forced completion (see Finishing Agents →
  // Forced completion). Custom adapters must forward `toolChoice` to the
  // provider and throw a classified `framework_completion_tool_choice_unsupported`
  // error if they cannot honor named tool selection — never silently fall
  // back to `auto`.
  toolChoice?: LLMToolChoice; // 'auto' | 'none' | 'required' | { type: 'tool'; toolName: string }
  reasoningMode?: ReasoningMode; // 'default' | 'disabled' — omitted ⇒ 'default'
  executionPhase?: ExecutionPhase; // 'work' | 'forced_completion' — omitted ⇒ 'work'
}

LLMConfig

LLM configuration options:

interface LLMConfig {
  // Required: The language model to use
  model: LanguageModel;

  // Generation parameters
  maxOutputTokens?: number;
  temperature?: number; // 0-2
  topP?: number;
  topK?: number;
  presencePenalty?: number;
  frequencyPenalty?: number;

  // Control parameters
  stopSequences?: string[];
  seed?: number; // For deterministic outputs
  maxRetries?: number; // Default: 3

  // Prompt caching (opt-in; pass a provider-specific strategy)
  cache?: CacheStrategy | CacheStrategy[];

  // HTTP configuration
  headers?: Record<string, string>;

  // Provider-specific options
  providerOptions?: Record<string, Record<string, JSONValue>>;
}

Prompt Caching

Prompt caching is opt-in. Pass a provider-specific strategy helper from @helix-agents/core that matches your model:

import { anthropicCache, openaiCache, xaiCache } from '@helix-agents/core';

// Anthropic
const agent = defineAgent({
  name: 'my-agent',
  systemPrompt: 'You are a helpful assistant.',
  llmConfig: {
    model: anthropic('claude-sonnet-4-20250514'),
    cache: anthropicCache({ ttl: '1h' }),
  },
});

// OpenAI
llmConfig: { model: openai('gpt-4o'), cache: openaiCache() }

// xAI/Grok
llmConfig: { model: xai('grok-3'), cache: xaiCache() }

// Google Gemini — no strategy needed; caching is built-in
llmConfig: { model: google('gemini-2.5-pro') }

Each helper applies the provider-specific mechanism before each LLM call:

Provider	Mechanism	What the Strategy Does
Anthropic	Explicit cache_control markers	Places breakpoints on the system prompt (which also covers tool definitions) and the latest turn's tool-result batch
OpenAI	Implicit prefix caching with routing hints	Sets promptCacheKey from sessionId for cache affinity
Google Gemini	Automatic prefix caching	No action needed (strategy not required)
xAI/Grok	Implicit with conversation routing	Sets x-grok-conv-id header from sessionId

Cache token usage is tracked through the standard usage tracking system and Langfuse tracing.

See the Vercel AI SDK Adapter - Prompt Caching section for detailed provider examples.

Provider Options

Enable provider-specific features:

// OpenAI reasoning summaries (o1, o3, o4-mini)
const config: LLMConfig = {
  model: openai('o1'),
  providerOptions: {
    openai: {
      reasoningSummary: 'detailed',
      reasoningEffort: 'high',
    },
  },
};

// Anthropic extended thinking (Claude)
const config: LLMConfig = {
  model: anthropic('claude-sonnet-4-20250514'),
  providerOptions: {
    anthropic: {
      thinking: { type: 'enabled', budgetTokens: 10000 },
    },
  },
};

Streaming Callbacks

Real-time streaming during generation:

interface LLMStreamCallbacks {
  // Called for each text token
  onTextDelta?: (delta: string) => void;

  // Called for thinking/reasoning content
  onThinking?: (content: string, isComplete: boolean) => void;

  // Called when LLM requests a tool
  onToolCall?: (toolCall: ParsedToolCall) => void;

  // Called on generation errors
  onError?: (error: Error) => void;
}

These callbacks enable real-time UI updates as the model generates content.

StepResult

The response from generateStep():

type StepResult<TOutput> =
  | TextStepResult
  | ToolCallsStepResult
  | StructuredOutputStepResult<TOutput>
  | ErrorStepResult;

Text Response

interface TextStepResult {
  type: 'text';
  content: string;
  thinking?: ThinkingContent;
  shouldStop: boolean;
  stopReason: StopReason;
}

Tool Calls

interface ToolCallsStepResult {
  type: 'tool_calls';
  toolCalls: ParsedToolCall[];
  subAgentCalls: ParsedSubAgentCall[];
  content?: string; // Text accompanying tool calls
  thinking?: ThinkingContent;
  shouldStop: false;
  stopReason: StopReason;
}

Structured Output

interface StructuredOutputStepResult<TOutput> {
  type: 'structured_output';
  output: TOutput; // Validated by outputSchema
  thinking?: ThinkingContent;
  shouldStop: true;
  stopReason: StopReason;
}

Error

interface ErrorStepResult {
  type: 'error';
  error: Error;
  shouldStop: boolean;
  stopReason: StopReason;
}

Stop Reason Normalization

Different LLM providers return different finish reasons. The framework normalizes them:

type StopReason =
  | 'end_turn' // Natural completion
  | 'tool_use' // Tool calls requested (not a stop)
  | 'max_tokens' // Token limit reached (FAILS agent)
  | 'content_filter' // Safety filter (FAILS agent)
  | 'refusal' // Model refused (FAILS agent)
  | 'stop_sequence' // Stop sequence hit (completes normally)
  | 'error' // Generation error (FAILS agent)
  | 'unknown'; // Unrecognized (FAILS agent)

Provider Mappings

Framework	OpenAI	Anthropic	Vercel AI SDK
end_turn	stop	end_turn	stop
tool_use	tool_calls	tool_use	tool-calls
max_tokens	length	max_tokens	length
content_filter	content_filter	-	content-filter
refusal	-	refusal	-

Terminal vs Continuation

Stop reasons determine agent behavior:

// Terminal conditions (agent fails)
function isErrorStopReason(reason: StopReason): boolean {
  return (
    reason === 'max_tokens' ||
    reason === 'content_filter' ||
    reason === 'refusal' ||
    reason === 'error' ||
    reason === 'unknown'
  );
}

// Recoverable errors (retryable for outputSchema agents)
function isRecoverableErrorStopReason(reason: StopReason): boolean {
  return reason === 'max_tokens';
}

// Continue execution (not a real stop)
// tool_use -> execute tools, then continue

// Normal completion
// end_turn, stop_sequence -> agent completes successfully

Forced completion for max_tokens (and other triggers): When an agent has an outputSchema and the LLM returns text with a max_tokens stop reason — or ordinary terminal text (end_turn/stop_sequence) without a completion-tool call, or exhausts maxSteps mid-tool-loop — the framework does not fail immediately or silently drop the output. It enters a deterministic forced-completion phase: the next call(s) expose exactly the expected completion tool under a named toolChoice directive, with model reasoning disabled where the adapter supports it, bounded by maxCompletionRetries (default 2, independent of maxSteps). The phase either succeeds with schema-valid output or the run fails with a classified framework_* error — it never silently completes with missing output. Non-recoverable stop reasons (content_filter, refusal, error, unknown) still fail immediately without entering forced completion. See Finishing Agents → Forced completion for the full lifecycle and @helix-agents/core reference for the API.

Thinking/Reasoning Content

Some models provide reasoning traces:

interface ThinkingContent {
  content: string; // The reasoning text
}

Supported providers:

• Anthropic Claude - Extended thinking via providerOptions.anthropic.thinking
• OpenAI o-series - Reasoning summaries via providerOptions.openai.reasoningSummary

Thinking content is:

• Streamed via onThinking callback
• Stored in StepResult.thinking
• Persisted in message history (if your state store supports it)
• Streamed as thinking chunks to clients

Composability

The adapter interface enables composability:

// Development: Use mock adapter
const adapter = new MockLLMAdapter({
  responses: [{ type: 'text', content: 'Test response' }],
});

// Production: Use Vercel adapter
const adapter = new VercelAIAdapter();

// Both work with any runtime
const executor = new JSAgentExecutor(stateStore, streamManager, adapter);

You can swap adapters without changing agent definitions or runtime code.

Next Steps

• Vercel AI SDK Adapter - Using the recommended adapter
• Custom Adapters - Building your own adapter