Skip to content

English · 简体中文

Content Blocks

Messages in PAR carry structured content through content_block list rather than flat strings. When you call Runtime.invoke, the response message contains a list of these blocks — text, tool calls, tool results, or images. Each block in the list can independently hold a cache_control field, which provider adapters (`Anthropic, `Openai) emit in the wire format for prompt caching.

The message type in Types looks like this:

type message = {
  role : message_role;
  content_blocks : content_block list;
  tool_calls : tool_call list option;
  tool_call_id : string option;
  name : string option;
}

content_blocks replaced the old content : string option field. Every block is a variant of content_block, each with its own cache_control slot.

Block types

type content_block =
  | Text_block of {
      text : string;
      cache_control : cache_control option;
    }
  | Tool_use_block of {
      id : string;
      name : string;
      arguments : Yojson.Safe.t;
      cache_control : cache_control option;
    }
  | Tool_result_block of {
      tool_use_id : string;
      content : string;
      cache_control : cache_control option;
    }
  | Image_block of {
      source : image_source;
      media_type : string;
      data : string;
      detail : string option;
      cache_control : cache_control option;
    }
  | Reasoning_block of {
      text : string;
      signature : string option;
      cache_control : cache_control option;
    }
  | Redacted_reasoning_block of {
      data : string;
      cache_control : cache_control option;
    }

Text_block holds plain text. The LLM's response text and user input both land here.

Tool_use_block represents an LLM's request to call a tool. The id matches the provider's tool-use identifier; name is the tool name; arguments is the JSON payload.

Tool_result_block carries the tool handler's output back to the conversation. tool_use_id links it to the originating Tool_use_block.

Image_block holds image data, either as a URL reference or base64-encoded bytes. media_type is the MIME type (e.g. "image/png").

Reasoning_block holds a reasoning model's chain of thought (o1/o3, DeepSeek-R1, Claude extended thinking). It carries the reasoning text, not the user-facing answer, and text_of_message deliberately leaves it out. signature is Anthropic's thinking signature: the block must be sent back verbatim, signature included, whenever the assistant message also carries a tool_use block, so the engine prepends reasoning blocks to the assistant message it records.

Redacted_reasoning_block holds the opaque payload Anthropic substitutes when it redacts thinking. It has no text and must also be echoed back for the round-trip to stay valid.

Reasoning blocks only reach the wire when the provider can replay them: the Anthropic adapter drops an unsigned Reasoning_block (one that came from another provider, or from a conversation persisted before reasoning became a block) because Anthropic rejects a thinking block without a signature. The OpenAI adapter never sends reasoning back — o1/o3 and most OpenAI-compatible APIs reject it on input.

llm_response.reasoning carries the same blocks for a single response, and the engine copies them into the recorded assistant message ahead of the text.

Construction

Build each variant by hand with record syntax:

(* Simple text block *)
let block = Text_block {
  text = "Hello, world!";
  cache_control = None;
}

(* Text block with prompt caching *)
let cached = Text_block {
  text = "System prompt content";
  cache_control = Some { type_ = `Ephemeral; ttl = Some `Five_min };
}

(* Tool result from a handler *)
let result = Tool_result_block {
  tool_use_id = "call_abc123";
  content = "42";
  cache_control = None;
}

(* Tool use from LLM response *)
let tool_use = Tool_use_block {
  id = "call_abc123";
  name = "calculator";
  arguments = `Assoc [("expression", `String "6 * 7")];
  cache_control = None;
}

(* Image from URL *)
let img = Image_block {
  source = Url "https://example.com/photo.png";
  media_type = "image/png";
  data = "";
  detail = None;
  cache_control = None;
}

(* Image from base64 *)
let img_b64 = Image_block {
  source = Base64 "iVBORw0KGgo...";
  media_type = "image/png";
  data = "";
  detail = None;
  cache_control = None;
}

Prefer the Message helpers shown in Image blocks over hand-building these records.

Build a full message by populating the record:

let msg : Types.message = {
  role = User;
  content_blocks = [
    Text_block { text = "What is 2+2?"; cache_control = None };
  ];
  tool_calls = None;
  tool_call_id = None;
  name = None;
}

Helper functions

The Message module provides three helpers for common string-to-block conversions.

content_of_string

val Message.content_of_string : string -> content_block list

Wraps a string in a single Text_block with cache_control = None. Returns an empty list for the empty string.

Message.content_of_string "Hello"
(* [Text_block { text = "Hello"; cache_control = None }] *)

Message.content_of_string ""
(* [] *)

string_of_content

val Message.string_of_content : content_block list -> string

Extracts text from all Text_block and Tool_result_block variants and concatenates them. Non-text blocks (Tool_use_block, Image_block) are silently skipped.

Message.string_of_content [
  Text_block { text = "Hello "; cache_control = None };
  Tool_use_block { id = "x"; name = "t"; arguments = `Assoc []; cache_control = None };
  Text_block { text = "world"; cache_control = None };
]
(* "Hello world" *)

text_of_message

val Message.text_of_message : Types.message -> string

Convenience wrapper. Equivalent to Message.string_of_content msg.content_blocks.

Message.text_of_message msg
(* Extracts all text and tool-result content from the message *)

content_opt

val Message.content_opt : Types.message -> string option

Returns Some text if the message has any extractable text, or None if the content is empty. Useful for pattern matching on responses.

match Message.content_opt resp with
| Some text -> Printf.printf "Got: %s\n" text
| None -> Printf.printf "No text in response\n"

cache_control

Every block variant carries an optional cache_control field:

type cache_control = {
  type_ : [`Ephemeral];
  ttl : cache_ttl option;
}

type cache_ttl = [`Five_min | `One_hour]

type_ is currently always `Ephemeral. The type is a variant, not a plain alias, so PAR can introduce persistent cache modes later without breaking changes. ttl is the time-to-live hint. The Anthropic wire format recognizes both values; other providers ignore cache_control gracefully.

Setting cache_control

There are three ways to mark blocks for caching.

Direct construction. Build the cache_control value inline:

Text_block {
  text = "Expensive system prompt";
  cache_control = Some { type_ = `Ephemeral; ttl = Some `Five_min };
}

Cache_breakpoint.mark_message. Marks the last block in a message's content_blocks list. This is the recommended user-facing API for caching a whole message:

val Cache_breakpoint.mark_message :
  ttl:Types.cache_ttl ->
  Types.message ->
  Types.message
let marked_msg = Cache_breakpoint.mark_message ~ttl:`One_hour msg
(* The last block in msg.content_blocks now has cache_control set *)

The function walks to the last block and applies cache_control to it regardless of variant. If content_blocks is empty, it's a no-op.

Engine auto-marking. The Engine calls apply_breakpoints internally before each LLM round trip. It evaluates breakpoint candidates (system prompt, per-message, per-tool) against the provider's cache capabilities, then stamps the chosen blocks automatically. You don't need to call this yourself; it runs as part of the ReAct loop.

Provider support

Prompt caching is currently supported by the `Anthropic wire format. When the provider adapter encounters a block with cache_control set, it emits the corresponding field in the JSON payload. The `Openai and `Ollama adapters ignore cache_control silently, so setting it on those providers is harmless.

Image blocks

type image_source =
  | Url of string
  | Base64 of string

Image_block carries a reference to image data along with its MIME type. Url points to an image hosted at a remote URL; the provider fetches it. Base64 embeds the raw bytes directly in the block. detail is the provider-specific resolution hint — OpenAI accepts "auto", "low" or "high"; other providers ignore it.

Constructing image blocks

Use the Message helpers rather than building the record by hand — they keep cache_control and detail consistent:

open Par

(* Remote image — the provider fetches it, bytes never pass through PAR *)
let remote = Message.image_url "https://example.com/photo.png" ()

(* Explicit MIME type and resolution hint *)
let remote_hd = Message.image_url "https://example.com/photo.jpg"
  ~media_type:"image/jpeg" ~detail:"high" ()

(* Bytes already in hand — checked, so the call can fail *)
let inline = match Message.image_base64 ~data:"iVBORw0KGgo..." ~media_type:"image/png" () with
  | Ok block -> block
  | Error msg -> failwith msg

(* Local file — read and base64-encoded; MIME type inferred from the extension *)
let local = match Message.image_file "assets/photo.png" () with
  | Ok block -> block
  | Error msg -> failwith msg

Message.media_type_of_extension maps .png, .jpg, .jpeg, .gif and .webp. .bmp is not inferred because neither supported provider accepts it; pass ~media_type for anything else, or the call is an error rather than a silent guess.

Limits and validation

The constructors that carry image bytes are the boundary where a caller hands PAR data it did not produce, so they check it instead of trusting it:

Rule Value
Decoded payload limit 5 MiB (Message.max_image_bytes; override per call with ?max_bytes)
Accepted media types image/png, image/jpeg, image/gif, image/webp (Message.supported_image_media_types)
Base64 payloads must decode; an oversized string is rejected on its encoded length before anything is decoded
Empty payloads rejected
Unknown file extension rejected unless ?media_type is given

image_file checks the file size before reading, so a large file is refused without being loaded into memory. image_base64 returns a result for the same reason; image_file always did.

image_url is not checked: it carries a reference, not bytes, and its media_type is a hint the provider may ignore.

Five mebibytes is the per-image cap Anthropic documents; OpenAI allows more, so raise ?max_bytes when the provider in use accepts more. These limits are PAR's own contract — they are not a claim that a provider accepted any particular image.

Sending an image to a model

Runtime.invoke takes a plain string. For multimodal input use Runtime.invoke_content, which takes a list of content blocks:

let result = Runtime.invoke_content rt ~agent_id:"vision"
  ~content:[
    Message.text "What is in this picture?";
    Message.image_url "https://example.com/photo.png" ();
  ] ()

Runtime.invoke is a thin wrapper around Runtime.invoke_content — it wraps the string in a single text block. There is one implementation, so the two entry points cannot drift apart.

Python input

The Python binding exposes the same synchronous operation and content constructors:

from par_runtime import Runtime, text, image_url, image_base64, image_file

# rt is an initialized Runtime with the "vision" agent already registered.
response_json = rt.invoke_content("vision", [
    text("Describe this image"),
    image_url("https://example.com/photo.png", detail="low"),
], save=True, update_current=True)

local_image = image_file("assets/photo.jpg")
inline_image = image_base64("iVBORw0KGgo...", media_type="image/png")

The constructors return the SDK's existing tagged-array JSON blocks; there is no second provider-specific input schema. invoke_content returns the same JSON envelope as invoke. save=True persists this call's conversation; update_current=False leaves shared current-conversation state unchanged. File helpers raise OSError on I/O failures and ImageError (a ValueError) when the content fails a check: unknown extension, unsupported media type, malformed or oversized base64, or a file over MAX_IMAGE_BYTES. MAX_IMAGE_BYTES and SUPPORTED_IMAGE_MEDIA_TYPES are exported so a caller can pre-check, and max_bytes= overrides the limit per call. Malformed blocks raise PARInvokeError before the provider is called. User input accepts text and image blocks, not Tool_use_block or Tool_result_block.

Persisted image blocks written before detail was added still decode: an absent detail defaults to None. New blocks preserve the hint when saved and loaded. The OCaml image_file helper also closes its channel on failed reads.

Provider wire formats

Provider Image encoding
`Openai Message content switches to the array form ([{type:"text",...}, {type:"image_url",...}]) only when the message carries an image. Messages without images keep the plain string form, so existing request payloads are unchanged. Base64 sources are sent as a data URI (data:<media_type>;base64,<data>), which is the only inline form OpenAI accepts. detail is emitted as image_url.detail.
`Anthropic Emits {"type":"image","source":{...}}. For base64, media_type and data live inside source, matching Anthropic's schema.

The `Ollama adapter inherits the OpenAI encoding.

Tool results that carry content

A tool used to return one JSON payload, which the runtime flattened into text before the model saw it. A tool that produces something the model has to look at — a screenshot, a chart, a rendered page — can return content blocks instead:

let tb = Runtime.register_tool rt ~name:"shot" ~description:"Takes a screenshot"
  ~input_schema:(`Assoc [])
  ~handler:(fun _ _ ->
    Success_content [
      Message.text "a screenshot of the dashboard";
      Message.image_file "assets/dashboard.png" ();
    ]) ()

Success_content is a new handler_result constructor. The blocks become the tool result in the conversation, so the next model call carries them. A tool that returns Success json — the usual case — is unchanged: its payload is still flattened into a text block.

Provider wire formats for a tool result that carries an image:

Provider Tool result encoding
`Openai The tool message switches to the array content form, with the image as an image_url part. The model must support vision.
`Anthropic tool_result.content becomes the array form: a text part plus an {"type":"image","source":{...}} part.

Middleware applies to rich content as well: pii_mask and sanitize_tool_output scrub the text blocks (image payloads are binary and are left alone), and logging records the success. A tool that declares an output_schema and returns rich content is logged as unvalidated — the schema describes a JSON payload, and there is none to check.

Python handlers

A Python tool handler returns JSON, so rich content travels through one envelope that both languages read:

import json
from par_runtime import text, image_file, tool_content

def shot(_input_json: str) -> str:
    return json.dumps(tool_content(
        text("a screenshot of the dashboard"),
        image_file("assets/dashboard.png"),
    ))

rt.register_tool_with_handler("shot", "Takes a screenshot", '{"type": "object"}', shot)

The envelope key is Types.tool_content_key (_par_content) and the blocks use the same tagged-array JSON as everywhere else, so a payload built in Python is read by the OCaml runtime unchanged.

Not supported

Rich content is a tool-result shape. A workflow Tool_call step rejects it with Invalid_input: a workflow step's output is a JSON value the engine stores and returns.

Migration from string

The old message type had content : string option. The new type, used by Runtime.create and all agent interactions, replaces this with content_blocks : content_block list. Here's how to migrate:

(* Old pattern *)
let msg = {
  role = User;
  content = Some "hello";
  tool_calls = None;
  tool_call_id = None;
  name = None;
}

(* New pattern *)
let msg = {
  role = User;
  content_blocks = Message.content_of_string "hello";
  tool_calls = None;
  tool_call_id = None;
  name = None;
}

For multi-block content, build the list explicitly:

let msg = {
  role = User;
  content_blocks = [
    Text_block { text = "Describe this image:"; cache_control = None };
    Image_block {
      source = Url "https://example.com/photo.png";
      media_type = "image/png";
      data = "";
      cache_control = None;
    };
  ];
  tool_calls = None;
  tool_call_id = None;
  name = None;
}

Reading content back out works through the same helpers, so code that previously used msg.content directly should switch to Message.text_of_message msg or Message.content_opt msg.

See also

  • Agent API -- Runtime configuration, agent config, tool registration
  • Streaming API -- Token streaming with content block events
  • Middleware API -- Pipeline hooks that see content blocks