Placeholders like [image 1] are inserted into the terminal input purely for display; the actual media travels as structured content blocks. They must not leak into the canonical model-facing message or LangSmith trace as if the user typed them.

When available, tracked placeholder spans identify the exact display tokens to strip so user-authored literal duplicates with the same token are preserved. The token fallback removes one matching occurrence per tracked media item for callers that only have placeholder text.

function

get_clipboard_image

Attempt to read an image from the system clipboard.

Supports macOS via pngpaste or osascript.

function

get_image_from_path

Read and encode an image file from disk.

function

get_video_from_path

Read and encode a video file from disk.

function

is_media_path

Return whether a path's extension is a known image or video extension.

This is a cheap, extension-only check (no file read or decode). Use it to classify a dropped file path as media without loading it, e.g. to reject a dragged image in a text-only input.

It is a heuristic, not a support check, and disagrees with the loaders in both directions: get_image_from_path accepts anything Pillow can decode regardless of extension, while get_video_from_path additionally requires a recognized magic-byte signature. Call get_media_from_path when the answer must match what can actually be attached.

function

get_media_from_path

Try to load a file as an image first, then as a video.

function

encode_to_base64

Encode raw bytes to a base64 string.

function

create_multimodal_content

Create multimodal message content with text, images, and videos.