FeaturesMultimodal

Images

Send images to vision-enabled models via URL or base64.

Gateway supports two image input formats. Both work across all vision-capable providers — Gateway translates to the right format automatically.

Image URL

The simplest way to send an image. Pass a URL directly with the image_url content block.

1response = client.responses.create(
2    model="openai/gpt-5.1",
3    input=[
4        {
5            "type": "message",
6            "role": "user",
7            "content": [
8                {"type": "text", "text": "Describe this image."},
9                {"type": "image_url", "url": "https://example.com/photo.jpg"},
10            ],
11        }
12    ],
13)

Image with source type

The image block gives you explicit control over the source type and media type. Use this for base64-encoded images or when you need to specify the format.

Base64
1import base64
2
3with open("photo.png", "rb") as f:
4    image_data = base64.standard_b64encode(f.read()).decode("utf-8")
5
6response = client.responses.create(
7    model="anthropic/claude-sonnet-4-20250514",
8    input=[
9        {
10            "type": "message",
11            "role": "user",
12            "content": [
13                {"type": "text", "text": "What do you see?"},
14                {
15                    "type": "image",
16                    "source_type": "base64",
17                    "media_type": "image/png",
18                    "data": image_data,
19                },
20            ],
21        }
22    ],
23)
URL
1response = client.responses.create(
2    model="anthropic/claude-sonnet-4-20250514",
3    input=[
4        {
5            "type": "message",
6            "role": "user",
7            "content": [
8                {"type": "text", "text": "What do you see?"},
9                {
10                    "type": "image",
11                    "source_type": "url",
12                    "media_type": "image/jpeg",
13                    "data": "https://example.com/photo.jpg",
14                },
15            ],
16        }
17    ],
18)

OpenAI SDK

If you’re using the OpenAI SDK pointed at Gateway, use the standard OpenAI image format.

OpenAI SDK
1from openai import OpenAI
2
3client = OpenAI(
4    api_key="YOUR_API_KEY",
5    base_url="https://api-gateway.merge.dev/v1/openai",
6)
7
8response = client.chat.completions.create(
9    model="gpt-5.1",
10    messages=[
11        {
12            "role": "user",
13            "content": [
14                {"type": "text", "text": "What's in this image?"},
15                {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}},
16            ],
17        }
18    ],
19)

Multiple images

Pass multiple image blocks in the same message to compare or analyze several images at once.

Multiple images
1response = client.responses.create(
2    model="openai/gpt-5.1",
3    input=[
4        {
5            "type": "message",
6            "role": "user",
7            "content": [
8                {"type": "text", "text": "Compare these two images."},
9                {"type": "image_url", "url": "https://example.com/before.jpg"},
10                {"type": "image_url", "url": "https://example.com/after.jpg"},
11            ],
12        }
13    ],
14)

Each image is estimated at ~765 tokens for context window and compression calculations. Keep this in mind when sending multiple images in a single request.