Vision-Language Models

Combine text and image inputs through the chat interface for image understanding, OCR, chart analysis, and visual question answering.

Vision is not a separate endpoint but an input modality of chat models: you still call POST /v1/chat/completions, only messages[].content changes from a string into an array of text and image parts.

Difference from text-only chat

messages[].content
  ├─ text-only model:  "Describe this image"                      → string
  └─ vision model:     [{type:"text"…}, {type:"image_url"…}]      → array

The gateway only passes through and translates protocols; it does not verify that a model can actually see. Choosing the wrong model usually surfaces as an upstream parameter error or silently ignored image content, so confirm the model has vision support before sending multimodal messages.

Selection criteria

Vision support is determined by tags or capabilities in the model catalog, not by the name:

Dimension	What to confirm
Context length	Multiple images and long document screenshots consume context quickly.
Single-image parsing	Support for high-resolution images and rendered PDF pages.
Multiple images	Allowed image count and total size per request.
Reasoning	For cross-image comparison or chart inference, confirm multi-step reasoning support.

Model identifiers come from the model catalog; this page does not list a fixed set. Entries tagged with vision support accept image input; trust the endpoint:

curl "${BASE_URL}/v1/public/models?availableOnly=true" \
  -H "Authorization: Bearer ${APIQIK_API_KEY}"

Request structure

{
  "model": "<vision-model-id>",
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "Describe the main content and list visible text in reading order." },
        { "type": "image_url", "image_url": { "url": "https://example.com/page.png" } }
      ]
    }
  ]
}

Part type	Field	Description
`text`	`text`	Instruction or question text.
`image_url`	`image_url.url`	HTTPS image URL or `data:image/...;base64,...`.

For multi-image comparison, append more image_url parts to the same content. Image order in the prompt affects understanding, so for comparison tasks label them in text, for example "first image" and "second image".

Input methods

Method	Suitable use	Notes
HTTPS image URL	Public or temporarily signed resources.	The upstream must be able to fetch it; signatures must remain valid throughout the request.
Data URL (Base64)	Small local images, internal system screenshots.	The body grows significantly; watch gateway and model size limits.
Multiple images	Multi-page receipts, before/after comparison, sequence understanding.	More images raise both latency and token cost.

Prompting guidance

State the task first: description, localization, transcription, comparison, or judgment.
For OCR, define reading order, language, and output format; ask for original line breaks where needed.
For charts, distinguish directly visible data from model inference.
Double-check high-risk fields such as amounts, identity numbers, and coordinates; never persist them unverified.
Never put images containing secrets, personal information, or internal data on public URLs.

Common issues

The model ignores the image: the model has no vision support, or the image part does not follow the OpenAI multimodal schema.
Upstream cannot fetch the image: the URL needs authentication, has expired, or is not publicly reachable. Switch to a Data URL or a publicly accessible temporary link.
Truncated output: image descriptions are often long; raise max_tokens accordingly.

Vision-Language Models

Difference from text-only chatCopy Anchor Link

Selection criteriaCopy Anchor Link

Request structureCopy Anchor Link

Input methodsCopy Anchor Link

Prompting guidanceCopy Anchor Link