Forum Discussion

twennemann's avatar
twennemann
Copper Contributor
Apr 14, 2026

Microsoft Foundry Agent via Responses API rejects local image input as Base64 data URL / byte array

Hello everyone,

we are seeing an issue with the new Microsoft Foundry Agents via the Responses API when sending a local image as part of the user message.

What works

  • text-only input
  • image by public URL

What fails

  • local PNG passed as Base64 data URL
  • local PNG passed as raw byte array through SDK methods

Example failing image part:

{ "type": "input_image", "image_url": "data:image/png;base64,...", "detail": "auto" }

Returned error:

{ "code": "invalid_payload", "message": "The provided data does not match the expected schema", "param": "/", "type": "invalid_request_error", "details": [] }

We reproduced this in:

  • C#
  • Python
  • raw REST

So this does not appear to be limited to one SDK.

Also important: the same pattern is used in the sample repo for the Foundry Agent Web App, and this scenario worked for us about one week ago:
https://github.com/microsoft-foundry/foundry-agent-webapp

Could you confirm whether local image input is currently supported for Foundry Agents through the Responses API, or whether this is a regression?

Best regards

1 Reply

  • Jamony's avatar
    Jamony
    MCT
    Jul 16, 2026

    This looks like a regression rather than an error in your payload. The current Microsoft Responses API documentation now shows the same input_image structure with a data:image/...;base64,... value, and the Foundry project Responses endpoint is documented as OpenAI-compatible. The related Microsoft Q&A thread also records that the GitHub issue was fixed a few days later.

    I would first update the OpenAI and azure-ai-projects packages and retry against the current project endpoint: {project_endpoint}/openai/v1/responses, using a vision-capable deployment. Keep the MIME type aligned with the actual bytes, remove line breaks from the Base64 string, and send the complete data URL in image_url. A raw byte array still has to be converted into that supported request shape by the SDK.

    If it still returns invalid_payload, compare the exact body using raw REST, capture the request ID, endpoint and package versions, and open an Azure support case. A short-lived Blob SAS URL remains a practical workaround.