Open-Source SDK for Evaluating AI Model Outputs (Sharing Resource)

hi vihargadhesariya​  Thanks for sharing this - evaluation is one of those areas everyone struggles with, especially once you move beyond simple demos.

An SDK that standardizes evaluation across text, image, and audio is really useful, particularly when you're comparing prompts, models, or agent behaviors over time. I like that this focuses on repeatable metrics and templates, which helps reduce the "gut feel" aspect of manual reviews.

For teams building with Azure OpenAI / agents, this kind of framework can also fit nicely into CI/CD or experimentation workflows, where you want consistent signals rather than ad-hoc human scoring.

Curious to see how others here are approaching evaluation as well - especially around:

• automated vs human-in-the-loop evaluation
• confidence / hallucination detection
• regression testing for prompts and agents

Appreciate you sharing the resource with the community!