Structured Outputs fail with server_error when Bing Grounding is enabled in Azure AI Agents

Hi everyone,

I’m running into a reproducible issue when using Structured Outputs (response_format: json_schema) together with Azure AI Agents that have the Bing Grounding tool enabled.

The API always returns:

"last_error": {
    "code": "server_error",
    "message": "Sorry, something went wrong."
}

The call returns HTTP 200, but the run fails immediately before the model generates any tokens (prompt_tokens = 0).

Environment

• Azure AI Foundry (Sweden Central)
• Project: Azure AI Agents
• Model: gpt-4.1 (Standard DataZone)
• Agent with tool: bing_grounding (created from the UI)
• API version visible in logs: 2025-05-15-preview
• SDK:
  • azure-ai-projects 1.2.0b6
  • azure-ai-agents   1.2.0b6

What I am Trying to Do

I am attempting to enforce a JSON Schema output using:

response_format = ResponseFormatJsonSchemaType(
    json_schema=ResponseFormatJsonSchema(
        name="test_schema",
        description="Simple structured output test",
        schema={
            "type": "object",
            "properties": {
                "mensaje": {"type": "string"}
            },
            "required": ["mensaje"],
            "additionalProperties": False
        }
    )
)

Then calling:

run = client.agents.runs.create_and_process(
    thread_id=thread.id,
    agent_id=agent.id,
    response_format=response_format
)

This same schema works successfully when the agent does NOT have Bing grounding enabled or when using the model outside of Agents.

Observed Behavior

The API request succeeds (HTTP 200), but the run immediately fails:

Full run status:

{
  "id": "run_XXXX",
  "status": "failed",
  "last_error": {
    "code": "server_error",
    "message": "Sorry, something went wrong."
  },
  "model": "gpt-4.1-EU-LDZ",
  "tools": [
    {
      "type": "bing_grounding",
      "bing_grounding": {
        "search_configurations": [
          {
            "connection_id": "...",
            "market": "es-es",
            "set_lang": "es",
            "count": 5
          }
        ]
      }
    }
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "test_schema",
      "schema": {
        "type": "object",
        "properties": {"mensaje": {"type": "string"}},
        "required": ["mensaje"],
        "additionalProperties": false
      }
    }
  },
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}

Key points:

• prompt_tokens = 0 → The failure happens before the model receives the prompt.
• The same code works if:
  • The agent has no tools
  • Or I remove response_format
• The error is always the same: server_error.

How to Reproduce

• Create an Azure AI Agent in AI Foundry.
• Add Bing Grounding to the agent.
• Set the model to gpt-4.1.

Run the following minimal Python script:

from azure.ai.projects import AIProjectClient
from azure.ai.agents.models import ResponseFormatJsonSchema, ResponseFormatJsonSchemaType
from azure.identity import AzureCliCredential

client = AIProjectClient(
    endpoint="YOUR_ENDPOINT",
    credential=AzureCliCredential()
)

agent_id = "YOUR_AGENT_ID"

schema = {
    "type": "object",
    "properties": {"mensaje": {"type": "string"}},
    "required": ["mensaje"]
}

response_format = ResponseFormatJsonSchemaType(
    json_schema=ResponseFormatJsonSchema(
        name="test_schema",
        description="Test schema",
        schema=schema
    )
)

thread = client.agents.threads.create()

client.agents.messages.create(
    thread_id=thread.id,
    role="user",
    content="Say hello"
)

run = client.agents.runs.create_and_process(
    thread_id=thread.id,
    agent_id=agent_id,
    response_format=response_format
)

print(run.status, run.last_error)

Result:
status = failed, last_error = server_error.

Expected Behavior

Structured Outputs should work when the agent has tools enabled (including Bing grounding), or at least return a meaningful validation error instead of server_error.

Question

• Is the combination Agents + Bing Grounding + Structured Outputs (json_schema) + gpt-4.1 currently supported?
• Is this a known limitation or bug?
• Is there a recommended workaround?

I am happy to provide full request IDs (X-Request-ID and apim-request-id) privately via support channels if needed.

Thanks!