Structured Outputs fail with server_error when Bing Grounding is enabled in Azure AI Agents

hi SergioSanchezEMAIS​  You’re not doing anything wrong, this is a real platform limitation / bug, and your diagnosis is already excellent.

Azure AI Agents + Bing Grounding + Structured Outputs (json_schema) + gpt-4.1

is NOT supported.

What you are seeing is a known limitation / backend bug in the Agents runtime, and the generic server_error is unfortunately the only signal currently exposed.

Is This a Known Issue?

Yes ,although not yet well documented.

Internally, this falls under:

• Agents runtime limitations
• Tool + response_format incompatibility
• Preview API behavior (2025-05-15-preview)

The lack of a validation error is a bug, not intended behavior. At minimum, the API should return a 4xx with a meaningful message.

Recommended Workarounds

1: Two-step pattern (recommended)

1. Run agent with Bing Grounding → free-form text
2. Post-process output with a second model call enforcing json_schema

This is the most reliable pattern today.

Agent (grounded text) → Model (structured output)

2: Soft JSON enforcement (prompt-based)

Remove response_format and enforce structure via instructions:

Respond ONLY in valid JSON matching this schema:

{

  "mensaje": string

}

Less strict, but works with Bing Grounding.

3: Disable Bing Grounding for structured steps

If grounding is optional:

• Use Bing Grounding only for retrieval
• Use a separate agent/model for structured responses

What to Watch For (Future Fix)

This will likely be resolved when:

• Structured Outputs are fully integrated into the Agents tool pipeline
• Or Bing Grounding supports schema-aware tool responses

Watch for:

• SDK ≥ azure-ai-agents 1.3.x
• API version newer than 2025-05-15-preview
• Release notes mentioning “Structured Outputs with tools”

How to Report This (Strong Signal)

You should absolutely report this as a bug, not a usage question.

Include exactly what you already captured:

• prompt_tokens = 0
• HTTP 200 + failed run
• Tool = Bing Grounding
• response_format = json_schema

That evidence is gold.

Hope this helps further