Forum Discussion
parulpaul01
Feb 13, 2026Copper Contributor
Title: Synthetic Dataset Format from AI Foundry Not Compatible with Evaluation Schema
Current Situation The synthetic dataset created from AI Foundry Data Synthetic Data is generated in the following messages format { "messages": [ { "role": "system", "content": "You are a helpful...
AnjaliSadhukhan
Microsoft
Mar 24, 2026Based on my understanding, AI Foundry currently doesn’t provide a built-in way to convert messages format into the structured evaluation schema.
A practical approach would be to preprocess the dataset by mapping:
- user → question
- assistant → ground_truth
- system → optional (metadata or ignore)
For multi-turn conversations, this would need to be split into multiple Q&A pairs. For RAG evaluations, reference_context may need to be added separately.
Alternatively, generating synthetic data directly in the required structured format can help avoid this extra step.