o3-deep-research is failed with the status incomplete with the reason as content filter
I working on an to do an deep research on internal data. I'm using currently the Azure OpenAI Responses API with MCP Tool. The underlying MCP server deployed into ACA with search and fetch tool with signatures in complaint with the specification (https://developers.openai.com/apps-sdk/build/mcp-server#company-knowledge-compatibility). OpenAI client created with 03-deep-research model with MCP tool, in a loop response status being checked. (https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/deep-research#remote-mcp-server-with-deep-research) Deep Research is being carried out for sometime, I could see in the log that handshake has been made, ListTools invoked, search tool is called post that fetch is called for the queries framed by the model.. But intermittently, the response status is becoming "incomplete" with incomplete reason as "content_filter". Otherwise the deep research is working fine. Not able identify the root cause as there is seems to be no way to identify what caused the content filtration whether its the prompt or completion. How to debug and check the root cause and rectify this ? Or is there known issue with the o3-deep-research model's intermediate reasoning completions Or search and fetch tool results are causing this ? I had uploaded a file made it available to MCP server, the search and fetch tool uses an Azure OpenAI agent to search the data using File Search and fetch tool gets the content of the file based on the id passed. For same file and same research topic the issue is not occurring always but intermittently.
Azure OpenAI Content Filter Result is always content_filter_error
I'm exploring blocklists as a solution for OpenAI not detecting sensitive words (specifically "wrist-cutting" in my local language (Cantonese) (to be fair not even Chinese AIs know the word) I have created a Blocklist with 1 entry: Term: [鎅𰾛𠝹]手 Type: Regex It can block inputs with ease: { "error": { "message": "The response was filtered due to the prompt triggering Azure OpenAI's content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766", "type": null, "param": "prompt", "code": "content_filter", "status": 400, "innererror": { "code": "ResponsibleAIPolicyViolation", "content_filter_result": { "custom_blocklists": { "details": [ { "filtered": true, "id": "ChineseBlockList" } ], "filtered": true }, "hate": { "filtered": false, "severity": "safe" }, "profanity": { "filtered": false, "detected": false }, "self_harm": { "filtered": false, "severity": "safe" }, "sexual": { "filtered": false, "severity": "safe" }, "violence": { "filtered": false, "severity": "safe" } } } } } However, it cannot block outputs. { "choices": [ { "content_filter_result": { "error": { "code": "content_filter_error", "message": "The contents are not filtered" } }, "content_filter_results": {}, "finish_reason": "stop", "index": 0, "logprobs": null, "message": { "content": "𠝹手（也寫作“拍手”）是一種手部動作，通常是將雙手合攏並用力拍打在一起，發出聲音。這個動作常用於表達讚賞、鼓勵或慶祝，像是在演出結束後觀眾的掌聲，或是在某些活動中用來引起注意。𠝹手也可以用於節奏感的表達，像是在音樂中隨著節拍拍手。這個動作在許多文化中都有其獨特的意義和用途。", "refusal": null, "role": "assistant" } } ], "created": 1737702254, "id": "chatcmpl-At81eUTIzDkZPCKznSKr19YMJU1ud", "model": "gpt-4o-mini-2024-07-18", "object": "chat.completion", "prompt_filter_results": [ { "prompt_index": 0, "content_filter_results": { "custom_blocklists": { "filtered": false }, "hate": { "filtered": false, "severity": "safe" }, "profanity": { "filtered": false, "detected": false }, "self_harm": { "filtered": false, "severity": "safe" }, "sexual": { "filtered": false, "severity": "safe" }, "violence": { "filtered": false, "severity": "safe" } } } ], "system_fingerprint": "fp_5154047bf2", "usage": { "completion_tokens": 138, "completion_tokens_details": { "accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0 }, "prompt_tokens": 34, "prompt_tokens_details": { "audio_tokens": 0, "cached_tokens": 0 }, "total_tokens": 172 } }Introduction to Content filtering and Embeddings in Azure Open AI Service
Content filtering and Embeddings in Azure AI Open Service: Abuse Monitoring Content Classification: Classifier models detect harmful language and/or images in user prompts (inputs) and completions (outputs). The system looks for categories of harms as defined in the Content Requirements, and assigns severity levels as described in more detail on the Content Filtering page. Abuse Pattern Capture: Azure OpenAI Service’s abuse monitoring looks at customer usage patterns and employs algorithms and heuristics to detect indicators of potential abuse. Detected patterns consider, for example, the frequency and severity at which harmful content is detected in a customer’s prompts and completions. Human Review and Decision: When prompts and/or completions are flagged through content classification and abuse pattern capture as described above, authorized Microsoft employees may assess the flagged content, and either confirm or correct the classification or determination based on predefined guidelines and policies. Data can be accessed for human review only by authorized Microsoft employees via Secure Access Workstations (SAWs) with Just-In-Time (JIT) request approval granted by team managers. For Azure OpenAI Service resources deployed in the European Economic Area, the authorized Microsoft employees are located in the European Economic Area. Notification and Action: When a threshold of abusive behavior has been confirmed based on the preceding three steps, the customer is informed of the determination by email. Except in cases of severe or recurring abuse, customers typically are given an opportunity to explain or remediate—and implement mechanisms to prevent recurrence of—the abusive behavior. Failure to address the behavior—or recurring or severe abuse—may result in suspension or termination of the customer’s access to Azure OpenAI resources and/or capabilities. Content filtering Azure OpenAI Service includes a content management system that works alongside core models to filter content. If the system identifies harmful content, you'll receive either An error on the API call Content_filter as the finish_reason on the response Mitigate Mitigating harms presented by large language models such as the Azure OpenAI models requires an iterative, layered approach that includes experimentation and continual measurement. Best practices – Content filtering Consider the following best practices Check the finish_reason to see if the generation is filtered Check that there's no error object in the content_filter_result Applications serving multiple end-users should pass the user parameter with each API call. More details: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/content-filter?tabs=warning%2Cpython#scenario-details Embedding: Cosine similarity Cosine similarity measures the cosine of the angle between two vectors projected in a multi-dimensional space. If two documents are far apart by Euclidean distance because of size, they could still have a smaller angle between them and therefore higher cosine similarity. Azure OpenAI embeddings rely on cosine similarity to compute similarity between documents and a query. More details: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/understand-embeddings#embedding-models
