Open-Source SDK for Evaluating AI Model Outputs (Sharing Resource)
Hi everyone, I wanted to share a helpful open-source resource for developers working with LLMs, AI agents, or prompt-based applications. One common challenge in AI development is evaluating model outputs in a consistent and structured way. Manual evaluation can be subjective and time-consuming. The project below provides a framework to help with that: AI-Evaluation SDK https://github.com/future-agi/ai-evaluation Key Features: - Ready-to-use evaluation metrics - Supports text, image, and audio evaluation - Pre-defined prompt templates - Quickstart examples available in Python and TypeScript - Can integrate with workflows using toolkits like LangChain Use Case: If you are comparing different models or experimenting with prompt variations, this SDK helps standardize the evaluation process and reduces manual scoring effort. If anyone has experience with other evaluation tools or best practices, I’d be interested to hear what approaches you use.
Azure OpenAI: gpt-5-mini chat/completions streaming returns empty response.
Summary When calling gpt-5-mini via Chat Completions with "stream": true, the server opens the stream but no assistant tokens are emitted and the final JSON is empty (choices: [], created: 0, empty id/model). The same code path streams correctly for gpt-5 and gpt-4o deployments. Also, non-streaming ("stream": false) with gpt-5-mini returns valid content as expected. Environment API: POST /openai/deployments/{deployment}/chat/completions?api-version=2025-01-01-preview Model / Deployment: gpt-5-mini (Azure OpenAI deployment) Date/Time observed: 26 Aug 2025, ~13:00 IST (UTC+05:30) Region: useast2 Note: Same client, headers, and network path work for gpt-5 and gpt-4o streaming. Request Endpoint /openai/deployments/gpt-5/chat/completions?api-version=2025-01-01-preview Body { "messages": [ { "role": "system", "content": "give the best result you can" }, { "role": "user", "content": "Hello" } ], "stream": true } Actual Response (final aggregated JSON after stream ends) { "choices": [], "created": 0, "id": "", "model": "", "object": "", "prompt_filter_results": [ { "prompt_index": 0, "content_filter_results": { "hate": { "filtered": false, "severity": "safe" }, "jailbreak": { "filtered": false, "detected": false }, "self_harm": { "filtered": false, "severity": "safe" }, "sexual": { "filtered": false, "severity": "safe" }, "violence": { "filtered": false, "severity": "safe" } } } ] } Notes: No delta tokens arrive on the SSE stream. No assistant message content is ever emitted. Content filter result is safe across categories. Expected Behavior With "stream": true, server should emit SSE chunks with assistant delta tokens and finish with a populated final message in choices[0].message.content. Azure OpenAI: gpt-5-mini chat/completions streaming returns empty response (choices: [], created: 0) while other models stream fine
Azure AI services building a chatbot
hey there, I was experimenting with chatgpt api and found out that it cannot analyze the context of data. For eg. I trained it on 10-k of Amazon and asked which sector made the most profit, it was unable to answer but if I asked the information with a targeted question like profit earned in the automation sector, It gave an accurate answer. Any idea how can I work it out
Step by step explanation of Microsoft bot framework C# SDK
Hi. Can someone tell me where I can find step by step explanation of Microsoft bot framework C# SDK? I've gone through following link but I'm unable to understand how bot is moving from main function to other .cs files, to OnMessageActivityAsync and many other confusions. https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-basics?view=azure-bot-service-4.0 Is there any documentation/video/course which explains about all files and code in the framework in a step by step and detailed way?
