OpenAI Chat Completions and Spring AI

The OpenAI Chat Completions API is a widely implemented JSON-over-HTTP interface for conversational language models. A client sends an ordered list of messages and generation options; the server returns one or more assistant messages with metadata and token usage.

Its basic shape is:

HTTP client
  -> POST /v1/chat/completions
  -> request JSON containing messages and options
  -> model inference
  <- response JSON containing choices and usage

Many gateways and model servers implement an OpenAI-compatible endpoint even when the underlying model is not hosted by OpenAI. This makes the format a useful de facto interoperability API, but it is not an independent formal standard like HTTP, JSON-RPC, or MCP. Compatible providers may support different fields, models, limits, error formats, or extensions.

Spring AI gives Java applications a provider-neutral model above this wire format. Application code works with Prompt, Message, ChatOptions, and ChatResponse; an OpenAI or OpenAI-compatible adapter converts those objects to and from the provider’s JSON.

The HTTP Request #

A simplified request is:

POST /v1/chat/completions HTTP/1.1
Host: api.example.com
Authorization: Bearer ...
Content-Type: application/json

{
  "model": "gpt-4.1",
  "messages": [
    {
      "role": "system",
      "content": "Answer concisely."
    },
    {
      "role": "user",
      "content": "Explain optimistic locking."
    }
  ],
  "temperature": 0.3,
  "max_tokens": 1000,
  "stream": false
}

model and messages identify what should run and what conversation context it should see. The remaining fields control generation. Some gateways select the model through configuration, a deployment name, URL, or header instead of requiring model in the body.

Messages #

messages is ordered. Its order is part of the prompt:

[
  {
    "role": "system",
    "content": "You are a support assistant."
  },
  {
    "role": "user",
    "content": "Where is order 42?"
  },
  {
    "role": "assistant",
    "content": "I need to look up that order."
  },
  {
    "role": "user",
    "content": "It belongs to account 7."
  }
]

Common roles are:

Text is commonly represented by a string. Models with multimodal support may accept an array of typed content parts containing text, images, audio, or other media.

The endpoint is normally stateless. A model does not remember a previous HTTP request merely because the caller reuses a conversation ID. To continue a conversation, the client must send the selected previous messages again:

load conversation history
  -> append current user message
  -> send all selected messages
  -> receive assistant message
  -> save new user and assistant messages

The history may be truncated, summarized, or retrieved from external storage to stay within the model’s context limit.

Generation Options #

Frequently implemented request fields include:

Provider support and semantics vary. A field accepted by one compatible endpoint may be rejected, ignored, renamed, or interpreted differently by another. Even model versions behind the same endpoint may differ.

Sampling options also interact. Applications commonly tune either temperature or top_p rather than changing both without evaluation.

The HTTP Response #

A simplified non-streaming response is:

{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1784778339,
  "model": "gpt-4.1",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Optimistic locking detects concurrent updates..."
      },
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 18,
    "total_tokens": 43
  }
}

The main fields are:

Each choice normally contains an index, assistant message, finish_reason, and optional log probabilities. Common finish reasons include:

• stop: normal completion;
• length: output or context limit reached;
• tool_calls: the model requested one or more tools; and
• content_filter: generation was stopped by a safety filter.

A successful HTTP response does not imply that the answer is correct. It means the provider accepted the request and produced a syntactically valid model result.

Provider Extensions #

OpenAI-compatible does not mean byte-for-byte identical. For example:

• Azure-compatible responses may include prompt_filter_results and content_filter_results.
• A gateway may add latency checkpoints, routing details, cache statistics, or service-tier metadata.
• Token details may distinguish reasoning, audio, cached, accepted prediction, or rejected prediction tokens.
• Error responses and rate-limit headers differ among providers.

Applications should ignore unknown fields when possible and avoid binding their domain model to every provider extension. When an extension is required, represent it explicitly in provider-specific configuration or metadata.

Tool Calls in the Schema #

A request can include tool definitions:

{
  "model": "gpt-4.1",
  "messages": [
    {
      "role": "user",
      "content": "What is the status of order 42?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "find_order",
        "description": "Look up an order by ID",
        "parameters": {
          "type": "object",
          "properties": {
            "order_id": {
              "type": "string",
              "description": "Canonical order ID"
            }
          },
          "required": [
            "order_id"
          ]
        }
      }
    }
  ]
}

The model does not execute the function. It can return an assistant message containing a structured request:

{
  "role": "assistant",
  "content": null,
  "tool_calls": [
    {
      "id": "call_123",
      "type": "function",
      "function": {
        "name": "find_order",
        "arguments": "{\"order_id\":\"42\"}"
      }
    }
  ]
}

The host application validates the arguments, invokes its local or remote operation, and sends the result in a subsequent request:

{
  "role": "tool",
  "tool_call_id": "call_123",
  "content": "{\"status\":\"SHIPPED\"}"
}

The usual traffic is therefore:

model request with tool schemas
  <- assistant tool-call request
application executes tool
  -> model request with tool result
  <- final assistant answer

Tool calling often needs at least two model round trips. The application, not the model, remains responsible for authorization, validation, idempotency, timeouts, and side effects.

Streaming Responses #

With "stream": true, the server normally returns Server-Sent Events whose data contains incremental chat.completion.chunk objects:

data: {"object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant"}}]}

data: {"object":"chat.completion.chunk","choices":[{"delta":{"content":"Optimistic"}}]}

data: {"object":"chat.completion.chunk","choices":[{"delta":{"content":" locking"}}]}

data: [DONE]

Chunks contain deltas rather than complete messages. A client must aggregate content, tool-call fragments, finish reasons, and usage where provided. Streaming improves time to first visible output but does not necessarily reduce total latency or token cost.

Spring AI’s Object Model #

Spring AI separates application-facing chat concepts from provider wire DTOs. The main types are:

The low-level application flow is:

ChatOptions options = OpenAiChatOptions.builder()
    .model("gpt-4.1")
    .temperature(0.3)
    .maxTokens(1000)
    .build();

Prompt prompt = new Prompt(
    List.of(
        new SystemMessage("Answer concisely."),
        new UserMessage("Explain optimistic locking.")
    ),
    options
);

ChatResponse response = chatModel.call(prompt);
AssistantMessage answer = response.getResult().getOutput();

ChatClient provides the higher-level fluent equivalent:

ChatResponse response = chatClient.prompt()
    .system("Answer concisely.")
    .user("Explain optimistic locking.")
    .options(options)
    .call()
    .chatResponse();

call() is the synchronous terminal operation. Prompt-building methods only configure the request; call() invokes the Advisor chain and model adapter and waits for completion. stream() returns a reactive stream instead.

How Spring AI Serializes a Request #

Spring AI does not normally serialize Prompt.toString() as the HTTP body. The provider adapter performs a mapping:

Spring AI Prompt
  messages:
    SystemMessage
    UserMessage
    AssistantMessage
    ToolResponseMessage
  options:
    ChatOptions
       |
       v
OpenAI adapter
  OpenAI request DTOs
  role/content conversion
  option and tool-schema conversion
       |
       v
JSON serializer
       |
       v
POST /v1/chat/completions

For the Spring AI OpenAI integration, transport types are represented under OpenAiApi, including request, message, choice, and completion structures. Applications should normally use the portable types rather than constructing those transport DTOs directly.

The response follows the reverse path:

HTTP response JSON
  -> OpenAI response DTOs
  -> AssistantMessage
  -> Generation
  -> ChatResponseMetadata and Usage
  -> ChatResponse

This boundary explains why logging a ChatClientRequest may display a Java record or Prompt representation while HTTP wire logging displays concrete OpenAI-compatible JSON. The former is the provider-neutral request before conversion; the latter is the serialized transport body after conversion.

Advisors and Conversation Memory #

ChatMemory is not part of the OpenAI wire schema. It is an application-side abstraction. A memory Advisor loads messages before model invocation and saves new messages afterward:

ChatMemory memory = MessageWindowChatMemory.builder()
    .maxMessages(20)
    .build();

ChatClient client = builder
    .defaultAdvisors(
        MessageChatMemoryAdvisor.builder(memory).build()
    )
    .build();

Each request supplies the conversation key:

ChatResponse response = client.prompt()
    .user(question)
    .advisors(spec -> spec.param(
        ChatMemory.CONVERSATION_ID,
        conversationId
    ))
    .call()
    .chatResponse();

Conceptually, the Advisor performs:

ChatMemory.get(conversationId)
  -> add historical messages to Prompt
  -> ChatModel.call(Prompt)
  -> ChatMemory.add(current user and assistant messages)

Consequently, the serialized messages array can contain the complete selected conversation window. Conversation IDs, storage, retention, and access control belong to the host application; the Chat Completions API does not define them.

Spring AI Tool Serialization #

Spring AI represents an executable capability as a ToolCallback. Annotated Java methods can be discovered and converted into callbacks:

public class OrderTools {

    @Tool(description = "Look up an order by ID")
    public OrderStatus findOrder(
            @ToolParam(description = "Canonical order ID") String orderId) {
        return orderService.find(orderId);
    }
}

Registering the object:

String answer = chatClient.prompt()
    .user("Where is order 42?")
    .tools(orderTools)
    .call()
    .content();

causes Spring AI to:

1. inspect the annotated method;
2. create a ToolCallback;
3. derive a JSON Schema for its arguments;
4. map the callback definition to the provider’s tools array;
5. map returned tool_calls into AssistantMessage.ToolCall values;
6. invoke the callback in the host application when automatic tool execution is enabled; and
7. serialize the result as a tool message for the next model request.

The LLM sees only names, descriptions, and schemas. It cannot reflect over the JVM or invoke arbitrary methods.

Logging the Two Representations #

Spring AI Advisor logging and HTTP wire logging answer different questions.

An Advisor such as SimpleLoggerAdvisor can show:

• Spring AI messages after earlier Advisors have modified the request;
• ChatOptions;
• Advisor context such as a conversation ID; and
• the provider-neutral ChatResponse.

Its request may look like Java toString() output because it is logging ChatClientRequest, not the eventual HTTP body.

HTTP client logging can show:

• the URL and headers;
• the final serialized JSON request;
• provider-specific fields and defaults;
• the raw JSON response; and
• streaming chunks.

Wire logging occurs after the provider adapter has created its transport DTO. It is the right layer for diagnosing serialization and provider compatibility. It is also dangerous: messages, retrieved documents, tool results, authorization headers, cookies, and personal data can appear in logs. Enable full payload logging only in a controlled development environment and apply redaction and retention limits.

Compatibility Boundaries #

An OpenAI-compatible endpoint should be treated as a provider contract, not as proof of identical behavior. Verify:

• endpoint paths and authentication;
• supported message roles and content types;
• model naming and deployment selection;
• request option names and limits;
• tool-call and structured-output support;
• streaming event format;
• token-usage accounting;
• safety and refusal fields;
• error bodies and HTTP status codes; and
• provider-specific extensions.

Spring AI reduces application coupling by keeping these details in a model adapter. It cannot make incompatible provider behavior identical. Test each supported gateway and model as a separate execution path.

See Spring AI for the broader application model, including Advisors, RAG, vector stores, memory repositories, observability, and production architecture.