English · 简体中文
Content Blocks¶
Messages in PAR carry structured content through content_block list rather than flat strings. When you call Runtime.invoke, the response message contains a list of these blocks — text, tool calls, tool results, or images. Each block in the list can independently hold a cache_control field, which provider adapters (`Anthropic, `Openai) emit in the wire format for prompt caching.
The message type in Types looks like this:
type message = {
role : message_role;
content_blocks : content_block list;
tool_calls : tool_call list option;
tool_call_id : string option;
name : string option;
}
content_blocks replaced the old content : string option field. Every block is a variant of content_block, each with its own cache_control slot.
Block types¶
type content_block =
| Text_block of {
text : string;
cache_control : cache_control option;
}
| Tool_use_block of {
id : string;
name : string;
arguments : Yojson.Safe.t;
cache_control : cache_control option;
}
| Tool_result_block of {
tool_use_id : string;
content : string;
cache_control : cache_control option;
}
| Image_block of {
source : image_source;
media_type : string;
data : string;
detail : string option;
cache_control : cache_control option;
}
| Reasoning_block of {
text : string;
signature : string option;
cache_control : cache_control option;
}
| Redacted_reasoning_block of {
data : string;
cache_control : cache_control option;
}
Text_block holds plain text. The LLM's response text and user input both land here.
Tool_use_block represents an LLM's request to call a tool. The id matches the provider's tool-use identifier; name is the tool name; arguments is the JSON payload.
Tool_result_block carries the tool handler's output back to the conversation. tool_use_id links it to the originating Tool_use_block.
Image_block holds image data, either as a URL reference or base64-encoded bytes. media_type is the MIME type (e.g. "image/png").
Reasoning_block holds a reasoning model's chain of thought (o1/o3, DeepSeek-R1, Claude extended thinking). It carries the reasoning text, not the user-facing answer, and text_of_message deliberately leaves it out. signature is Anthropic's thinking signature: the block must be sent back verbatim, signature included, whenever the assistant message also carries a tool_use block, so the engine prepends reasoning blocks to the assistant message it records.
Redacted_reasoning_block holds the opaque payload Anthropic substitutes when it redacts thinking. It has no text and must also be echoed back for the round-trip to stay valid.
Reasoning blocks only reach the wire when the provider can replay them: the Anthropic adapter drops an unsigned Reasoning_block (one that came from another provider, or from a conversation persisted before reasoning became a block) because Anthropic rejects a thinking block without a signature. The OpenAI adapter never sends reasoning back — o1/o3 and most OpenAI-compatible APIs reject it on input.
llm_response.reasoning carries the same blocks for a single response, and the engine copies them into the recorded assistant message ahead of the text.
Construction¶
Build each variant by hand with record syntax:
(* Simple text block *)
let block = Text_block {
text = "Hello, world!";
cache_control = None;
}
(* Text block with prompt caching *)
let cached = Text_block {
text = "System prompt content";
cache_control = Some { type_ = `Ephemeral; ttl = Some `Five_min };
}
(* Tool result from a handler *)
let result = Tool_result_block {
tool_use_id = "call_abc123";
content = "42";
cache_control = None;
}
(* Tool use from LLM response *)
let tool_use = Tool_use_block {
id = "call_abc123";
name = "calculator";
arguments = `Assoc [("expression", `String "6 * 7")];
cache_control = None;
}
(* Image from URL *)
let img = Image_block {
source = Url "https://example.com/photo.png";
media_type = "image/png";
data = "";
detail = None;
cache_control = None;
}
(* Image from base64 *)
let img_b64 = Image_block {
source = Base64 "iVBORw0KGgo...";
media_type = "image/png";
data = "";
detail = None;
cache_control = None;
}
Prefer the Message helpers shown in Image blocks over hand-building these records.
Build a full message by populating the record:
let msg : Types.message = {
role = User;
content_blocks = [
Text_block { text = "What is 2+2?"; cache_control = None };
];
tool_calls = None;
tool_call_id = None;
name = None;
}
Helper functions¶
The Message module provides three helpers for common string-to-block conversions.
content_of_string¶
Wraps a string in a single Text_block with cache_control = None. Returns an empty list for the empty string.
Message.content_of_string "Hello"
(* [Text_block { text = "Hello"; cache_control = None }] *)
Message.content_of_string ""
(* [] *)
string_of_content¶
Extracts text from all Text_block and Tool_result_block variants and concatenates them. Non-text blocks (Tool_use_block, Image_block) are silently skipped.
Message.string_of_content [
Text_block { text = "Hello "; cache_control = None };
Tool_use_block { id = "x"; name = "t"; arguments = `Assoc []; cache_control = None };
Text_block { text = "world"; cache_control = None };
]
(* "Hello world" *)
text_of_message¶
Convenience wrapper. Equivalent to Message.string_of_content msg.content_blocks.
content_opt¶
Returns Some text if the message has any extractable text, or None if the content is empty. Useful for pattern matching on responses.
match Message.content_opt resp with
| Some text -> Printf.printf "Got: %s\n" text
| None -> Printf.printf "No text in response\n"
cache_control¶
Every block variant carries an optional cache_control field:
type cache_control = {
type_ : [`Ephemeral];
ttl : cache_ttl option;
}
type cache_ttl = [`Five_min | `One_hour]
type_ is currently always `Ephemeral. The type is a variant, not a plain alias, so PAR can introduce persistent cache modes later without breaking changes. ttl is the time-to-live hint. The Anthropic wire format recognizes both values; other providers ignore cache_control gracefully.
Setting cache_control¶
There are three ways to mark blocks for caching.
Direct construction. Build the cache_control value inline:
Text_block {
text = "Expensive system prompt";
cache_control = Some { type_ = `Ephemeral; ttl = Some `Five_min };
}
Cache_breakpoint.mark_message. Marks the last block in a message's content_blocks list. This is the recommended user-facing API for caching a whole message:
let marked_msg = Cache_breakpoint.mark_message ~ttl:`One_hour msg
(* The last block in msg.content_blocks now has cache_control set *)
The function walks to the last block and applies cache_control to it regardless of variant. If content_blocks is empty, it's a no-op.
Engine auto-marking. The Engine calls apply_breakpoints internally before each LLM round trip. It evaluates breakpoint candidates (system prompt, per-message, per-tool) against the provider's cache capabilities, then stamps the chosen blocks automatically. You don't need to call this yourself; it runs as part of the ReAct loop.
Provider support¶
Prompt caching is currently supported by the `Anthropic wire format. When the provider adapter encounters a block with cache_control set, it emits the corresponding field in the JSON payload. The `Openai and `Ollama adapters ignore cache_control silently, so setting it on those providers is harmless.
Image blocks¶
Image_block carries a reference to image data along with its MIME type. Url points to an image hosted at a remote URL; the provider fetches it. Base64 embeds the raw bytes directly in the block. detail is the provider-specific resolution hint — OpenAI accepts "auto", "low" or "high"; other providers ignore it.
Constructing image blocks¶
Use the Message helpers rather than building the record by hand — they keep cache_control and detail consistent:
open Par
(* Remote image — the provider fetches it, bytes never pass through PAR *)
let remote = Message.image_url "https://example.com/photo.png" ()
(* Explicit MIME type and resolution hint *)
let remote_hd = Message.image_url "https://example.com/photo.jpg"
~media_type:"image/jpeg" ~detail:"high" ()
(* Bytes already in hand — checked, so the call can fail *)
let inline = match Message.image_base64 ~data:"iVBORw0KGgo..." ~media_type:"image/png" () with
| Ok block -> block
| Error msg -> failwith msg
(* Local file — read and base64-encoded; MIME type inferred from the extension *)
let local = match Message.image_file "assets/photo.png" () with
| Ok block -> block
| Error msg -> failwith msg
Message.media_type_of_extension maps .png, .jpg, .jpeg, .gif and .webp. .bmp is not inferred because neither supported provider accepts it; pass ~media_type for anything else, or the call is an error rather than a silent guess.
Limits and validation¶
The constructors that carry image bytes are the boundary where a caller hands PAR data it did not produce, so they check it instead of trusting it:
| Rule | Value |
|---|---|
| Decoded payload limit | 5 MiB (Message.max_image_bytes; override per call with ?max_bytes) |
| Accepted media types | image/png, image/jpeg, image/gif, image/webp (Message.supported_image_media_types) |
| Base64 payloads | must decode; an oversized string is rejected on its encoded length before anything is decoded |
| Empty payloads | rejected |
| Unknown file extension | rejected unless ?media_type is given |
image_file checks the file size before reading, so a large file is refused
without being loaded into memory. image_base64 returns a result for the same
reason; image_file always did.
image_url is not checked: it carries a reference, not bytes, and its
media_type is a hint the provider may ignore.
Five mebibytes is the per-image cap Anthropic documents; OpenAI allows more, so
raise ?max_bytes when the provider in use accepts more. These limits are
PAR's own contract — they are not a claim that a provider accepted any
particular image.
Sending an image to a model¶
Runtime.invoke takes a plain string. For multimodal input use Runtime.invoke_content, which takes a list of content blocks:
let result = Runtime.invoke_content rt ~agent_id:"vision"
~content:[
Message.text "What is in this picture?";
Message.image_url "https://example.com/photo.png" ();
] ()
Runtime.invoke is a thin wrapper around Runtime.invoke_content — it wraps the string in a single text block. There is one implementation, so the two entry points cannot drift apart.
Python input¶
The Python binding exposes the same synchronous operation and content constructors:
from par_runtime import Runtime, text, image_url, image_base64, image_file
# rt is an initialized Runtime with the "vision" agent already registered.
response_json = rt.invoke_content("vision", [
text("Describe this image"),
image_url("https://example.com/photo.png", detail="low"),
], save=True, update_current=True)
local_image = image_file("assets/photo.jpg")
inline_image = image_base64("iVBORw0KGgo...", media_type="image/png")
The constructors return the SDK's existing tagged-array JSON blocks; there is no
second provider-specific input schema. invoke_content returns the same JSON
envelope as invoke. save=True persists this call's conversation;
update_current=False leaves shared current-conversation state unchanged. File
helpers raise OSError on I/O failures and ImageError (a ValueError) when
the content fails a check: unknown extension, unsupported media type, malformed
or oversized base64, or a file over MAX_IMAGE_BYTES. MAX_IMAGE_BYTES and
SUPPORTED_IMAGE_MEDIA_TYPES are exported so a caller can pre-check, and
max_bytes= overrides the limit per call. Malformed blocks raise
PARInvokeError before the provider is called. User input accepts text and
image blocks, not Tool_use_block or Tool_result_block.
Persisted image blocks written before detail was added still decode: an absent
detail defaults to None. New blocks preserve the hint when saved and loaded.
The OCaml image_file helper also closes its channel on failed reads.
Provider wire formats¶
| Provider | Image encoding |
|---|---|
`Openai |
Message content switches to the array form ([{type:"text",...}, {type:"image_url",...}]) only when the message carries an image. Messages without images keep the plain string form, so existing request payloads are unchanged. Base64 sources are sent as a data URI (data:<media_type>;base64,<data>), which is the only inline form OpenAI accepts. detail is emitted as image_url.detail. |
`Anthropic |
Emits {"type":"image","source":{...}}. For base64, media_type and data live inside source, matching Anthropic's schema. |
The `Ollama adapter inherits the OpenAI encoding.
Tool results that carry content¶
A tool used to return one JSON payload, which the runtime flattened into text before the model saw it. A tool that produces something the model has to look at — a screenshot, a chart, a rendered page — can return content blocks instead:
let tb = Runtime.register_tool rt ~name:"shot" ~description:"Takes a screenshot"
~input_schema:(`Assoc [])
~handler:(fun _ _ ->
Success_content [
Message.text "a screenshot of the dashboard";
Message.image_file "assets/dashboard.png" ();
]) ()
Success_content is a new handler_result constructor. The blocks become the
tool result in the conversation, so the next model call carries them. A tool
that returns Success json — the usual case — is unchanged: its payload is
still flattened into a text block.
Provider wire formats for a tool result that carries an image:
| Provider | Tool result encoding |
|---|---|
`Openai |
The tool message switches to the array content form, with the image as an image_url part. The model must support vision. |
`Anthropic |
tool_result.content becomes the array form: a text part plus an {"type":"image","source":{...}} part. |
Middleware applies to rich content as well: pii_mask and
sanitize_tool_output scrub the text blocks (image payloads are binary and are
left alone), and logging records the success. A tool that declares an
output_schema and returns rich content is logged as unvalidated — the schema
describes a JSON payload, and there is none to check.
Python handlers¶
A Python tool handler returns JSON, so rich content travels through one envelope that both languages read:
import json
from par_runtime import text, image_file, tool_content
def shot(_input_json: str) -> str:
return json.dumps(tool_content(
text("a screenshot of the dashboard"),
image_file("assets/dashboard.png"),
))
rt.register_tool_with_handler("shot", "Takes a screenshot", '{"type": "object"}', shot)
The envelope key is Types.tool_content_key (_par_content) and the blocks
use the same tagged-array JSON as everywhere else, so a payload built in Python
is read by the OCaml runtime unchanged.
Not supported¶
Rich content is a tool-result shape. A workflow Tool_call step rejects it
with Invalid_input: a workflow step's output is a JSON value the engine
stores and returns.
Migration from string¶
The old message type had content : string option. The new type, used by Runtime.create and all agent interactions, replaces this with content_blocks : content_block list. Here's how to migrate:
(* Old pattern *)
let msg = {
role = User;
content = Some "hello";
tool_calls = None;
tool_call_id = None;
name = None;
}
(* New pattern *)
let msg = {
role = User;
content_blocks = Message.content_of_string "hello";
tool_calls = None;
tool_call_id = None;
name = None;
}
For multi-block content, build the list explicitly:
let msg = {
role = User;
content_blocks = [
Text_block { text = "Describe this image:"; cache_control = None };
Image_block {
source = Url "https://example.com/photo.png";
media_type = "image/png";
data = "";
cache_control = None;
};
];
tool_calls = None;
tool_call_id = None;
name = None;
}
Reading content back out works through the same helpers, so code that previously used msg.content directly should switch to Message.text_of_message msg or Message.content_opt msg.
See also¶
- Agent API -- Runtime configuration, agent config, tool registration
- Streaming API -- Token streaming with content block events
- Middleware API -- Pipeline hooks that see content blocks