GenAgent.Backends.OpenAI (GenAgentOpenAI v0.3.0)

Copy Markdown View Source

GenAgent.Backend implementation backed by the OpenAI Responses API (POST /v1/responses).

Unlike the CLI-backed backends (GenAgent.Backends.Claude, GenAgent.Backends.Codex), this backend talks directly to an HTTP API. Unlike GenAgent.Backends.Anthropic, it does not resend the full conversation history on every turn -- OpenAI holds conversation state server-side, and the backend threads one previous_response_id value across turns.

This makes the OpenAI backend structurally simpler than the Anthropic one: there's no messages array to manage, and multi-turn just works as long as update_session/2 runs after each terminal :result event.

How the two halves of a turn land in the session

  1. prompt/2 POSTs {input: [{role: "user", content: prompt}], previous_response_id: session.previous_response_id, instructions, store: true}, parses the response, and returns {:ok, events, session}. The session itself is not mutated here -- the request body is built from session.previous_response_id as it stood entering the turn.
  2. When the state machine delivers the terminal :result event, it calls update_session/2 with the event's data. The event's :response_id becomes session.previous_response_id for the next turn.
  3. The next prompt/2 uses the new previous_response_id and OpenAI replays all prior context on the server side.

Instructions do not persist across previous_response_id chains

OpenAI's documentation is explicit: "the instructions used on previous turns will not be present in the context." This backend therefore resends :instructions on every request when the option is set. The per-request token cost is tiny, but the invariant is load-bearing -- a future optimization that tries to "save tokens by only sending instructions once" would silently break system-prompt behavior on every turn after the first.

Reasoning models

For reasoning models (o1/o3/o4/gpt-5), the response body contains output[] items with type: "reasoning" alongside the type: "message" item. This backend ignores the reasoning items entirely when extracting text -- it walks output[] looking for the message item -- but surfaces reasoning_tokens (from usage.output_tokens_details.reasoning_tokens) in the :usage event when present, so patterns can reason about cost.

Set :reasoning_effort to request a specific effort level, for example :low, :medium, or :high; leave it nil for the model default. The value is sent as reasoning.effort without validation, and the accepted values depend on the model, so check the model's documentation. An unsupported value is reported by the API as an error on the first prompt.

Options

  • :api_key -- OpenAI API key. Defaults to System.get_env("OPENAI_API_KEY"). start_session/1 returns {:error, :missing_api_key} when neither provides a non-empty key, unless a one-arity :http_fn is supplied.
  • :model -- model name. Defaults to "gpt-5".
  • :instructions -- system prompt (string). Resent every turn; see note above.
  • :reasoning_effort -- an atom or string passed through as reasoning.effort (for example :low, :medium, :high), or nil for the model default. Accepted values depend on the model. When set, adds {"reasoning": {"effort": ...}} to each request.
  • :max_output_tokens -- cap on output tokens per turn. Defaults to nil (model default).
  • :receive_timeout -- HTTP receive timeout in milliseconds. Defaults to 60_000; increase it for long-context or slow reasoning turns.
  • :connect_timeout -- HTTP connect timeout in milliseconds. Defaults to Req's default when unset.
  • :http_fn -- a 1-arity function (request_map) -> {:ok, response_map} | {:error, term} that replaces the default Req-backed HTTP call. Intended for tests.

Summary

Types

Passed through as reasoning.effort; accepted values depend on the model.

t()

Types

reasoning_effort()

@type reasoning_effort() :: atom() | String.t() | nil

Passed through as reasoning.effort; accepted values depend on the model.

t()

@type t() :: %GenAgent.Backends.OpenAI{
  api_key: String.t() | nil,
  client_session_id: String.t(),
  connect_timeout: timeout() | nil,
  http_fn: (map() -> {:ok, map()} | {:error, term()}),
  instructions: String.t() | nil,
  max_output_tokens: pos_integer() | nil,
  model: String.t(),
  previous_response_id: String.t() | nil,
  reasoning_effort: reasoning_effort(),
  receive_timeout: timeout()
}