Forum Discussion

ChrisOezbek's avatar
ChrisOezbek
Copper Contributor
Oct 25, 2026

Microsoft-Decision-1 scores changed on the same deployment / version 1 – anyone else?

We use Microsoft-Decision-1 in Foundry (DataZoneStandard, model version 1, Sweden Central) as a yes/no gate with `noul` questions and a fixed probability threshold.

Between Oct 23 (~15:00 UTC) and Oct 24 (~22:00 UTC) the scores changed for our calibration requests:

  • Same deployment, same API key, no changes in the activity log.
  • We had this running for pretty much since the announcement and probabilities for the same requests where near identical.
  • Now: Repeated calls of the same query can vary a lot (0.915 vs 0.932 seconds apart).
  • Re-scoring 10,000 stored requests: Results are all over the place.
  • Most low scores barely move, but clear positives dropped (0.99 -> 0.78) and some negatives rose (0.00 -> 0.90).
  • Pinning the deployment (versionUpgradeOption = NoAutoUpgrade) changed nothing.

The model now seems to read the instructions much more literally: removing one qualifying word from our context note moved a clear positive from 0.92 to 0.996. Did anyone else see Decision-1 probabilities shift around Oct 23/24? Has Microsoft said whether updates (e.g. the announced rebase onto MAI/OpenAI base models) can happen behind "version 1"?

No RepliesBe the first to reply