OpenAI Vision API: send an image as Base64

The payload shape Chat Completions expects

In the Chat Completions API, an image travels as one content part inside a user message. The part has "type": "image_url", and its image_url.url value is the full data URI — data:image/png;base64, followed by the encoded bytes. The optional detail field sits next to url:

{
  "model": "gpt-4o",
  "messages": [{
    "role": "user",
    "content": [
      { "type": "text", "text": "What's in this screenshot?" },
      { "type": "image_url",
        "image_url": {
          "url": "data:image/png;base64,iVBORw0KGgo...",
          "detail": "high"
        } }
    ]
  }]
}

detail controls processing depth: low is a cheap fixed-size pass, high lets the model inspect the image in detail, and auto (the default) lets the model decide. For screenshots with small text, high usually pays for itself.

The Responses API uses input_image instead

The Responses API carries the same image under a different part type: "type": "input_image", with the data URI placed directly in the image_url field as a plain string — not nested in an object:

{
  "model": "gpt-4o",
  "input": [{
    "role": "user",
    "content": [
      { "type": "input_text", "text": "What's in this screenshot?" },
      { "type": "input_image",
        "image_url": "data:image/png;base64,iVBORw0KGgo..." }
    ]
  }]
}

The data URI value itself is identical in both APIs — only the envelope differs. That is why the tool above outputs the standalone data URI too: copy it into whichever envelope your client library sends.

Python and Node.js examples

Python with the standard library only — read the file, encode it, splice the prefix:

import base64
import requests

with open("screenshot.png", "rb") as f:
    b64 = base64.b64encode(f.read()).decode("utf-8")

resp = requests.post(
    "https://api.openai.com/v1/chat/completions",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={
        "model": "gpt-4o",
        "messages": [{
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe this image"},
                {"type": "image_url",
                 "image_url": {"url": f"data:image/png;base64,{b64}"}}
            ],
        }],
    },
)
print(resp.json()["choices"][0]["message"]["content"])

Node.js — fs.readFileSync with the "base64" encoding does it in one call:

import fs from "node:fs";

const b64 = fs.readFileSync("screenshot.png", "base64");

const body = {
  model: "gpt-4o",
  messages: [{
    role: "user",
    content: [
      { type: "text", text: "Describe this image" },
      { type: "image_url",
        image_url: { url: `data:image/png;base64,${b64}` } },
    ],
  }],
};

One subtlety in both languages: the MIME type in the prefix must match the actual bytes. A PNG renamed to .jpg still has PNG bytes, and some client libraries sniff the type from the file extension. This tool detects the type from the file's real signature instead.

Size: the 33% overhead and the 20 MB ceiling

Base64 encodes three bytes into four characters, so the string is about a third larger than the file. OpenAI documents a practical ceiling of roughly 20 MB per encoded image — a 15 MB photo is already there before you have written a word of prompt. Since vision models downscale input internally, a 1024-2048 px image carries essentially all the signal the model can use. Run large photos through the JPG compressor first, and check the finished size with the Base64 size calculator before you ship it.

Why OpenAI wants the prefix (and Google does not)

OpenAI treats the data URI as a URL — the same field accepts https:// image links, so the string has to be a well-formed URI. Google Cloud Vision sits at the other end of the spectrum: its image.content field takes the raw Base64 payload only, and a data: prefix there causes an encoding error. The Cloud Vision Base64 guide covers that shape, and the API payload builder fills both at once.

When the image comes back as Base64

Image generation endpoints also speak Base64 — they return the produced image as an image_url containing a data URI. Paste that string into the Base64 to image decoder to preview and save it as a file.

Failure modes worth knowing

• Missing or doubled prefix. Concatenating data:image/png;base64, onto a string that already carries it produces ...base64,data:image... — the request fails as an invalid image.
• Line breaks in the string. Some encoders (and email-unfriendly MIME contexts) wrap Base64 at 76 characters. Raw newlines are invalid inside a JSON string — either strip them or JSON-escape as \n. This tool emits a single-line string.
• Hand-rolled JSON assembly. Building the body with string concatenation invites escaping bugs. Assemble with a JSON serializer (or paste the payload from this tool).
• Wrong MIME label. The prefix must name the format the bytes actually are.

Why encode in the browser

Vision pipelines routinely process material you cannot paste into an upload-based web tool: internal dashboards, unreleased designs, contracts, medical images. This page encodes with the browser's FileReader API — there is no upload step, which you can confirm in the Network tab: converting fires zero requests.