Images

Send images to vision-enabled models via URL or base64

Images

Send images to vision-enabled models via URL or base64

Gateway supports two image input formats. Both work across all vision-capable providers - Gateway translates to the right format automatically.

Image URL

The simplest way to send an image. Pass a URL directly with the image_url content block.

1 response = client.responses.create(
2     model="openai/gpt-5.1",
3     input=[
4         {
5             "type": "message",
6             "role": "user",
7             "content": [
8                 {"type": "text", "text": "Describe this image."},
9                 {"type": "image_url", "url": "https://example.com/photo.jpg"},
10             ],
11         }
12     ],
13 )

Image with source type

The image block gives you explicit control over the source type and media type. Use this for base64-encoded images or when you need to specify the format.

Base64

1 import base64
2 
3 with open("photo.png", "rb") as f:
4     image_data = base64.standard_b64encode(f.read()).decode("utf-8")
5 
6 response = client.responses.create(
7     model="anthropic/claude-sonnet-4-20250514",
8     input=[
9         {
10             "type": "message",
11             "role": "user",
12             "content": [
13                 {"type": "text", "text": "What do you see?"},
14                 {
15                     "type": "image",
16                     "source_type": "base64",
17                     "media_type": "image/png",
18                     "data": image_data,
19                 },
20             ],
21         }
22     ],
23 )

URL

1 response = client.responses.create(
2     model="anthropic/claude-sonnet-4-20250514",
3     input=[
4         {
5             "type": "message",
6             "role": "user",
7             "content": [
8                 {"type": "text", "text": "What do you see?"},
9                 {
10                     "type": "image",
11                     "source_type": "url",
12                     "media_type": "image/jpeg",
13                     "data": "https://example.com/photo.jpg",
14                 },
15             ],
16         }
17     ],
18 )

OpenAI SDK

If you’re using the OpenAI SDK pointed at Gateway, use the standard OpenAI image format.

OpenAI SDK

1 from openai import OpenAI
2 
3 client = OpenAI(
4     api_key="YOUR_API_KEY",
5     base_url="https://api-gateway.merge.dev/v1/openai",
6 )
7 
8 response = client.chat.completions.create(
9     model="gpt-5.1",
10     messages=[
11         {
12             "role": "user",
13             "content": [
14                 {"type": "text", "text": "What's in this image?"},
15                 {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}},
16             ],
17         }
18     ],
19 )

Multiple images

Pass multiple image blocks in the same message to compare or analyze several images at once.

Multiple images

1 response = client.responses.create(
2     model="openai/gpt-5.1",
3     input=[
4         {
5             "type": "message",
6             "role": "user",
7             "content": [
8                 {"type": "text", "text": "Compare these two images."},
9                 {"type": "image_url", "url": "https://example.com/before.jpg"},
10                 {"type": "image_url", "url": "https://example.com/after.jpg"},
11             ],
12         }
13     ],
14 )

Each image is estimated at ~765 tokens for context window and compression calculations. Keep this in mind when sending multiple images in a single request.

Gateway supports two image input formats. Both work across all vision-capable providers - Gateway translates to the right format automatically.

Image URL

The simplest way to send an image. Pass a URL directly with the image_url content block.

1 response = client.responses.create(
2     model="openai/gpt-5.1",
3     input=[
4         {
5             "type": "message",
6             "role": "user",
7             "content": [
8                 {"type": "text", "text": "Describe this image."},
9                 {"type": "image_url", "url": "https://example.com/photo.jpg"},
10             ],
11         }
12     ],
13 )

Image with source type

The image block gives you explicit control over the source type and media type. Use this for base64-encoded images or when you need to specify the format.

Base64

1 import base64
2 
3 with open("photo.png", "rb") as f:
4     image_data = base64.standard_b64encode(f.read()).decode("utf-8")
5 
6 response = client.responses.create(
7     model="anthropic/claude-sonnet-4-20250514",
8     input=[
9         {
10             "type": "message",
11             "role": "user",
12             "content": [
13                 {"type": "text", "text": "What do you see?"},
14                 {
15                     "type": "image",
16                     "source_type": "base64",
17                     "media_type": "image/png",
18                     "data": image_data,
19                 },
20             ],
21         }
22     ],
23 )

URL

1 response = client.responses.create(
2     model="anthropic/claude-sonnet-4-20250514",
3     input=[
4         {
5             "type": "message",
6             "role": "user",
7             "content": [
8                 {"type": "text", "text": "What do you see?"},
9                 {
10                     "type": "image",
11                     "source_type": "url",
12                     "media_type": "image/jpeg",
13                     "data": "https://example.com/photo.jpg",
14                 },
15             ],
16         }
17     ],
18 )

OpenAI SDK

If you’re using the OpenAI SDK pointed at Gateway, use the standard OpenAI image format.

OpenAI SDK

1 from openai import OpenAI
2 
3 client = OpenAI(
4     api_key="YOUR_API_KEY",
5     base_url="https://api-gateway.merge.dev/v1/openai",
6 )
7 
8 response = client.chat.completions.create(
9     model="gpt-5.1",
10     messages=[
11         {
12             "role": "user",
13             "content": [
14                 {"type": "text", "text": "What's in this image?"},
15                 {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}},
16             ],
17         }
18     ],
19 )

Multiple images

Pass multiple image blocks in the same message to compare or analyze several images at once.

Multiple images

1 response = client.responses.create(
2     model="openai/gpt-5.1",
3     input=[
4         {
5             "type": "message",
6             "role": "user",
7             "content": [
8                 {"type": "text", "text": "Compare these two images."},
9                 {"type": "image_url", "url": "https://example.com/before.jpg"},
10                 {"type": "image_url", "url": "https://example.com/after.jpg"},
11             ],
12         }
13     ],
14 )

Each image is estimated at ~765 tokens for context window and compression calculations. Keep this in mind when sending multiple images in a single request.

1	response = client.responses.create(
2	model="openai/gpt-5.1",
3	input=[
4	{
5	"type": "message",
6	"role": "user",
7	"content": [
8	{"type": "text", "text": "Describe this image."},
9	{"type": "image_url", "url": "https://example.com/photo.jpg"},
10	],
11	}
12	],
13	)

1	import base64
2
3	with open("photo.png", "rb") as f:
4	image_data = base64.standard_b64encode(f.read()).decode("utf-8")
5
6	response = client.responses.create(
7	model="anthropic/claude-sonnet-4-20250514",
8	input=[
9	{
10	"type": "message",
11	"role": "user",
12	"content": [
13	{"type": "text", "text": "What do you see?"},
14	{
15	"type": "image",
16	"source_type": "base64",
17	"media_type": "image/png",
18	"data": image_data,
19	},
20	],
21	}
22	],
23	)

1	from openai import OpenAI
2
3	client = OpenAI(
4	api_key="YOUR_API_KEY",
5	base_url="https://api-gateway.merge.dev/v1/openai",
6	)
7
8	response = client.chat.completions.create(
9	model="gpt-5.1",
10	messages=[
11	{
12	"role": "user",
13	"content": [
14	{"type": "text", "text": "What's in this image?"},
15	{"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}},
16	],
17	}
18	],
19	)