GenAgent.Backends.OpenAI (GenAgentOpenAI v0.2.2)

Copy Markdown View Source

GenAgent.Backend implementation backed by the OpenAI Responses API (POST /v1/responses).

Unlike the CLI-backed backends (GenAgent.Backends.Claude, GenAgent.Backends.Codex), this backend talks directly to an HTTP API. Unlike GenAgent.Backends.Anthropic, it does not resend the full conversation history on every turn -- OpenAI holds conversation state server-side, and the backend threads one previous_response_id value across turns.

This makes the OpenAI backend structurally simpler than the Anthropic one: there's no messages array to manage, and multi-turn just works as long as update_session/2 runs after each terminal :result event.

How the two halves of a turn land in the session

  1. prompt/2 POSTs {input: [{role: "user", content: prompt}], previous_response_id: session.previous_response_id, instructions, store: true}, parses the response, and returns {:ok, events, session}. The session itself is not mutated here -- the request body is built from session.previous_response_id as it stood entering the turn.
  2. When the state machine delivers the terminal :result event, it calls update_session/2 with the event's data. The event's :response_id becomes session.previous_response_id for the next turn.
  3. The next prompt/2 uses the new previous_response_id and OpenAI replays all prior context on the server side.

Instructions do not persist across previous_response_id chains

OpenAI's documentation is explicit: "the instructions used on previous turns will not be present in the context." This backend therefore resends :instructions on every request when the option is set. The per-request token cost is tiny, but the invariant is load-bearing -- a future optimization that tries to "save tokens by only sending instructions once" would silently break system-prompt behavior on every turn after the first.

Reasoning models

For reasoning models (o1/o3/o4/gpt-5), the response body contains output[] items with type: "reasoning" alongside the type: "message" item. This backend ignores the reasoning items entirely when extracting text -- it walks output[] looking for the message item -- but surfaces reasoning_tokens (from usage.output_tokens_details.reasoning_tokens) in the :usage event when present, so patterns can reason about cost.

Set :reasoning_effort to :low, :medium, or :high to request a specific effort level; leave it nil for the model default.

Options

  • :api_key -- OpenAI API key. Defaults to System.get_env("OPENAI_API_KEY").
  • :model -- model name. Defaults to "gpt-5".
  • :instructions -- system prompt (string). Resent every turn; see note above.
  • :reasoning_effort -- one of :low | :medium | :high | nil. When set, adds {"reasoning": {"effort": ...}} to each request.

  • :max_output_tokens -- cap on output tokens per turn. Defaults to nil (model default).
  • :receive_timeout -- HTTP receive timeout in milliseconds. Defaults to 60_000; increase it for long-context or slow reasoning turns.
  • :connect_timeout -- HTTP connect timeout in milliseconds. Defaults to Req's default when unset.
  • :http_fn -- a 1-arity function (request_map) -> {:ok, response_map} | {:error, term} that replaces the default Req-backed HTTP call. Intended for tests.

Summary

Types

reasoning_effort()

@type reasoning_effort() :: :low | :medium | :high | nil

t()

@type t() :: %GenAgent.Backends.OpenAI{
  api_key: String.t() | nil,
  client_session_id: String.t(),
  connect_timeout: timeout() | nil,
  http_fn: (map() -> {:ok, map()} | {:error, term()}),
  instructions: String.t() | nil,
  max_output_tokens: pos_integer() | nil,
  model: String.t(),
  previous_response_id: String.t() | nil,
  reasoning_effort: reasoning_effort(),
  receive_timeout: timeout()
}