Structured Outputs fail with server_error when Bing Grounding is enabled in Azure AI Agents
Hi everyone, I’m running into a reproducible issue when using Structured Outputs (response_format: json_schema) together with Azure AI Agents that have the Bing Grounding tool enabled. The API always returns: "last_error": { "code": "server_error", "message": "Sorry, something went wrong." } The call returns HTTP 200, but the run fails immediately before the model generates any tokens (prompt_tokens = 0). Environment Azure AI Foundry (Sweden Central) Project: Azure AI Agents Model: gpt-4.1-EU-LDZ Agent with tool: bing_grounding (created from the UI) API version visible in logs: 2025-05-15-preview SDK: azure-ai-projects 1.2.0b6 azure-ai-agents 1.2.0b6 What I am Trying to Do I am attempting to enforce a JSON Schema output using: response_format = ResponseFormatJsonSchemaType( json_schema=ResponseFormatJsonSchema( name="test_schema", description="Simple structured output test", schema={ "type": "object", "properties": { "mensaje": {"type": "string"} }, "required": ["mensaje"], "additionalProperties": False } ) ) Then calling: run = client.agents.runs.create_and_process( thread_id=thread.id, agent_id=agent.id, response_format=response_format ) This same schema works successfully when the agent does NOT have Bing grounding enabled or when using the model outside of Agents. Observed Behavior The API request succeeds (HTTP 200), but the run immediately fails: Full run status: { "id": "run_XXXX", "status": "failed", "last_error": { "code": "server_error", "message": "Sorry, something went wrong." }, "model": "gpt-4.1-EU-LDZ", "tools": [ { "type": "bing_grounding", "bing_grounding": { "search_configurations": [ { "connection_id": "...", "market": "es-es", "set_lang": "es", "count": 5 } ] } } ], "response_format": { "type": "json_schema", "json_schema": { "name": "test_schema", "schema": { "type": "object", "properties": {"mensaje": {"type": "string"}}, "required": ["mensaje"], "additionalProperties": false } } }, "usage": { "prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0 } } Key points: prompt_tokens = 0 → The failure happens before the model receives the prompt. The same code works if: The agent has no tools Or I remove response_format The error is always the same: server_error. How to Reproduce Create an Azure AI Agent in AI Foundry. Add Bing Grounding to the agent. Set the model to gpt-4.1-EU-LDZ. Run the following minimal Python script: from azure.ai.projects import AIProjectClient from azure.ai.agents.models import ResponseFormatJsonSchema, ResponseFormatJsonSchemaType from azure.identity import AzureCliCredential client = AIProjectClient( endpoint="YOUR_ENDPOINT", credential=AzureCliCredential() ) agent_id = "YOUR_AGENT_ID" schema = { "type": "object", "properties": {"mensaje": {"type": "string"}}, "required": ["mensaje"] } response_format = ResponseFormatJsonSchemaType( json_schema=ResponseFormatJsonSchema( name="test_schema", description="Test schema", schema=schema ) ) thread = client.agents.threads.create() client.agents.messages.create( thread_id=thread.id, role="user", content="Say hello" ) run = client.agents.runs.create_and_process( thread_id=thread.id, agent_id=agent_id, response_format=response_format ) print(run.status, run.last_error) Result: status = failed, last_error = server_error. Expected Behavior Structured Outputs should work when the agent has tools enabled (including Bing grounding), or at least return a meaningful validation error instead of server_error. Question Is the combination Agents + Bing Grounding + Structured Outputs (json_schema) + gpt-4.1-EU-LDZ currently supported? Is this a known limitation or bug? Is there a recommended workaround? I am happy to provide full request IDs (X-Request-ID and apim-request-id) privately via support channels if needed. Thanks!
