Hypercharging Work IQ with Decision APIs

The pattern in this post: a Decision API picks the Work IQ tool before it runs.

In short

• A Decision API picks the Work IQ tool (fetch, retrieve or ask) before anything runs, then checks the result.
• "Who is my manager?" took 20 to 25 seconds with ask and 1.3 seconds with fetch, with the same answer.
• Routed by Jev, answers came first on 12 of 13 real questions: a median of 5.6s, against 22.3s for Work IQ ask alone.
• Three hosted Decision APIs route almost equally well. On my 134 routing questions, Jev scored 131, and Microsoft-Decision-1 and OpenAI's Decisions API both 128.
• Microsoft-Decision-1 launched in Foundry on October 9, 2026. On a second, current-app set of 102 questions it scored 92, where Jev scored 91.
• I also tested eight open-weight models and five general LLMs. A 421M model I fine-tuned in 17 minutes went from 53 to 110.
• Code: workiq-decision-api and laya-workiq-routing.

This post is my own experiment, not a Work IQ feature or official guidance. I put a Decision API in front of Work IQ's tool calls to test one thing: do answers come back sooner with the same quality? All tests and numbers are my own, from my own tenant.

The first part of this post is the pattern and the result. The second part is the benchmark, for readers choosing a model.

Microsoft 365 Work IQ gives an agent a set of MCP tools over your mail, meetings, chats, files and org chart. At one end, ask hands the question to Microsoft 365 Copilot, which plans, searches and writes a full answer. At the other end, fetch reads one record, such as your manager, in about a second. Between them sits retrieve, which searches the sources you name.

Each tool is the best one for some questions. The speed-up comes from choosing well, every time, before any tool runs. A Decision API is built for exactly that: it reads the question, answers a few typed questions about it ("which tool?", "which record or source?", "did that answer it?") with probabilities.

I first put Jev in front of Work IQ in a desktop app and a voice app. Across 13 of my own questions, each asked three ways at the same moment, the routed path answered first on 12, with a median of 5.6 seconds. This post covers the tools, the three decisions, how the routing was tested, and how several Decision APIs and general LLMs compared at making the choice.

The voice app. The top bar chooses what picks the tool (a decision model, a general LLM, or Work IQ ask on its own) and the lane.

Work IQ's MCP tools, one by one

Work IQ's MCP server has 14 tools. Six of them write, act or support: they create, update or delete items, run actions, accept the terms of use and return a debug link. These are the other eight, with descriptions from the tool definitions. The times are what I measured from Singapore on my own tenant, calling each tool directly over MCP.

The spread in time follows from how much work each tool does. fetch reads one known record. retrieve searches and ranks across sources. ask does both of those and then writes an answer, which is the most complete result and takes the longest.

The discovery tools matter too. An agent that does not know a record's path can find it with search_paths and then read it with fetch in under a second. When the path is known in advance, as it is for the routes in this post, the app can go straight to fetch.

In a text window, half a minute for a full answer is fine. In a voice conversation, the first words need to come within a few seconds. So the question became: for each workplace question, which is the lightest tool that fully answers it?

Matching the question to the tool

"Who is my manager?" is one fetch of /me/manager. So is "what's on my calendar today" or "show my unread mail". Questions like "find the mail about the budget approval" need retrieve, ideally limited to the source named. Open questions such as "what should I prioritise this week?" are what ask is for.

"Who is my manager?" went from 20 to 25 seconds with ask to 1.3 seconds with fetch, with the same answer. Naming the right source for a search took it from about 16 seconds to about 10.

Three decisions in front of Work IQ

The router asks the Decision API three things.

1. Reach: which tool? Record (fetch), Search (retrieve), Reasoned (ask), or Other, for messages that are not a work question at all.
2. Scope: where? For a record, which of 15 Graph-style paths, such as /me/manager, /me/events or /me/mailFolders/inbox. For a search, which of five sources: mail, meetings, files, Teams or people. The router also counts how many separate requests the question holds.
3. Check: did that answer it? After fetch or retrieve, the result goes back to the Decision API with one more question: does this answer what was asked? If yes, a small model writes it up. If ask can add more, the question goes on to ask, so a hard question still gets Copilot's full answer.

In the original Jev runs, reach and scope went out together in one call of about 0.3 seconds. The app then calls the chosen Work IQ tool directly, with no agent loop and no tool list to read.

Five real questions from my history. For "What unread mail is waiting for me?", reach picks Record at 99%, fetch reads the inbox in 1.25s, and the check confirms the result answers the question; the Decision API's two calls take 0.62 of the 6.4 seconds. The timeline at the bottom is real seconds, to scale. Interactive version.

"Find my Northwind meeting transcript summary." Search in meetings and files finds the meeting, the check sees the summary itself is not in the results, and ask composes it.

The app's existing integration uses Jev, from TypeSafe. I also tested Microsoft-Decision-1, OpenAI's Decisions API, several open models and general LLMs in the routing role. The screenshots and end-to-end races below show the original Jev integration.

Each answer shows its route. Here the router chose the record lane in 460 ms with 98% on Record, and Work IQ read the manager record directly. The answer is blurred.

The app explains its own routing, so you can see why a question went where it did.

Same question, three ways

The other natural way to pick a tool is the standard agent pattern: give a model Work IQ's read tools and let it decide. To compare, the desktop app can send one question down three paths at the same moment, with the same account, the same Work IQ tools and the same model (gpt-5.4-mini) writing the reply:

• Jev routed: the three decisions above, then the chosen tool.
• Model picks tools: gpt-5.4-mini gets Work IQ's read tools, told to prefer the fastest one that can answer.
• Work IQ ask alone: the question as asked, straight to ask.

I ran 13 questions from my own use this way.

Four questions, each raced down three paths on one to-scale clock: Jev routed, model picks tools, and Work IQ ask alone

Four of the runs, played at 4x speed on one to-scale clock. For "What's my latest emails about", Jev picks the inbox record and has a written answer in 2.7s. Names and answer text are changed. Interactive version.

The model picking its own tools usually chose well. Its time went to reading the tool list on each round (4,300 to 9,700 prompt tokens) and, on some questions, to a second search round. The Decision API never reads the tool list: the app already knows the tools, and only asks which one.

The one question where Jev was not first is the most useful to look at:

For "Find my Northwind meeting transcript summary", Jev chose Search in meetings and files. retrieve found the meeting in 7.7 seconds, and the check saw that the summary itself was not in the results, so the question went on to ask. Jev's answer, composed by ask, came 10 seconds after ask alone, which went straight to composing. Questions that need Copilot to read and summarise a long item are where ask is the right first step, and a case I plan to add to the reach descriptions.

Three side-by-side runs where my own test rig hit a one-off hiccup (a call it did not retry, or its own 20-second timeout for the writing model) are left out of these numbers.

Making the routing right

A fast choice only helps if it is the right one, so I wrote a test of 134 workplace questions across 26 topics, each with the expected route. Every change was measured against it.

What moved the score:

• Several requests in one question. "Who are my reports and what are their titles?" is two requests. The router now counts the requests and finds where each one starts and ends. This took one test set from 100 to 110 of 116.
• Meeting content and ownership. "What did the planning meeting cover?" and "Who owns the billing system?" sound like record lookups but need a search. Describing each option more precisely fixed both.
• Small record details. Teams reports only the newest message per chat. Once the option descriptions said so, the router sent "what did X tell me" to search instead of to a record that holds only the latest message.

With these, Jev routes 129 to 131 of the 134 as expected.

Three things stood out:

1. Confidence and correctness were independent here. Correct routings scored as low as 0.10, and incorrect ones as high as 0.93. So I don't use a confidence threshold. I fall back to a wider search when a lane returns nothing.
2. Plain option wording scored higher than structured wording. TypeSafe suggests describing what each option is for, what it is not for, and giving examples. I tried that, three runs per variant, and the structured versions scored 124 to 129 against 129 to 131 for plain wording, mostly by sending judgement questions to search. My options were already short and specific, which may be why the extra structure did not add information. I kept the plain wording.
3. Say what "none of these" means. Every choice I send has a decline option. Jev has learned to rarely pick it, but a model reading "insufficient information or ambiguity" fresh takes it at its word, and a short question like "Am I available right now?" can sound ambiguous. Spelling out that declining means the message is not a work question at all, such as small talk, kept every model from declining real questions.

Which Decision API?

The races above used Jev, because that is what the app is wired to. I then ran the same 134 questions through other routers. These are the best of each kind, timed from a US region.

A plan is the set of routing calls for one question. Microsoft's time is per call, because its quota did not allow a full-plan replay.

The three hosted Decision APIs are within three questions of each other, on a test whose wording grew up around Jev. Any of them would make a workable router. The open model matched Jev's score but is too slow to route live on one GPU. The general LLM scores well with nothing new to host, and takes over a second longer.

I have not yet rerun the end-to-end races with Microsoft-Decision-1.

The rest of this post is the benchmark behind that table: how fair the test is, each router in detail, and where the time goes. If you only want the conclusions, skip to "What I took away" at the end.

Is the test tilted toward Jev?

Somewhat. The option wording was refined against Jev's misses, and checked on the same 134 questions. So I made two changes before comparing models.

• More realistic questions. I replaced 43 of the 134 with questions that name a concrete thing, such as "Find the design doc for the Contoso single sign-on rollout". Topics, expected routes and the prompt stayed the same, and every name is fictional.
• Clearer prompt wording. OpenAI's Decisions API is designed to read plain text, so I sent it the question that way, and it read my wording literally. "Runbook" was not on my list of documents, so it gave 0.01 to files on "Which document has the runbook for the Northwind cutover?". Jev, trained on wording like mine, filled in the gap. I fixed three such issues in the shared prompt and reran every model.

These are the scores used throughout this post:

pplx ties Jev on accuracy, and gpt-5.4-mini, the Decisions API and Microsoft-Decision-1 are within a few questions.

The new set is not a true held-out test, since I wrote it knowing the old one, and the added words came from looking at misses. But they are words any reader would expect on those lists, every model got them, and none was tuned on them.

The 134-question benchmark, timed from Singapore. Microsoft-Decision-1 is shown with a hollow marker at its time per API call, because its quota prevented a comparable plan time.

Microsoft's Decision API

Microsoft-Decision-1 launched in Microsoft Foundry on October 9, 2026. Its Decision API accepts text or JSON state and named questions: choice selects an option, noul returns a yes/no probability, and score grades an ordered scale. It adds a first-party System 1 Decision API to the hosted options for this routing pattern.

I deployed version 1 in East US 2 using DataZoneStandard, which Microsoft recommends for locality-sensitive workloads. The endpoint is /providers/microsoft/v1/systemone, with Microsoft Entra ID authentication. No GPU deployment was needed.

I ran it two ways: on the 134-question test used throughout this post, through the same Swift route checker and shared prompt as every other model, and on the 102-question corpus in the current app, next to Jev 1.13 with the app's own prompts and route validation.

134 questions. Microsoft-Decision-1 scored 128, level with OpenAI's Decisions API. Jev, rerun seven times the same day in the same checker (six of them at once, finishing in under three minutes with no errors), scored 129 to 130, a little under the 131 to 132 of its earlier runs. Against today's Jev, Microsoft is one or two questions behind. All 283 Microsoft calls returned HTTP 200. The median call took 401ms (p95 509ms) from Singapore. The deployment's quota is 17 requests a minute, so the checker had to wait between calls, and I can't give a full-plan time.

The six misses were two searches that named no source, one multi-source question split in two, one search narrowed to email when files were also expected, one meeting-or-email question sent to reasoned, and one four-request question treated as a single email search.

102 questions.

Same prompts and route validation. Successful API calls only, not complete Work IQ answer timings. The East US 2 strip is a separate measurement.

Latencies in the top panels were measured from Singapore and exclude deliberate quota waits and HTTP errors. The East US 2 panel is the container replay, with its full table just below. Five backend HTTP 500 responses occurred in the initial run; one question needed a rerun, which succeeded. The accuracy gap was one question, too small to establish a winner.

From Singapore, Jev answered faster in both tests, about 0.27s against 0.40 to 0.44s per call. Most of that gap is distance. Jev's connection ends at a Cloudflare edge in Singapore (41ms to connect), while Microsoft's endpoint is reached directly in East US 2 (232ms to connect, plus 464ms for the TLS handshake on a new connection).

So I replayed 56 of the recorded routing requests, the same ones for each service, from Azure Container Instances in two regions. These are median milliseconds per call, with p95 in brackets, and every call returned HTTP 200.

Inside Azure Jev is still faster per call, by about 65ms from East US 2 and 140ms from West US 3. Microsoft's round trip from East US 2 is 1ms, so its roughly 195ms is the service itself. The global deployment has the worst tail because it balances across regions. From West US 3, 36 of its 56 calls were served in Central US (245ms median), 11 in Australia East (413ms) and 9 in Sweden Central (430ms). The data zone deployment served all 56 from Central US. This is two regions, one afternoon and 56 calls each. Microsoft's announcement reports its own speed comparisons, and I did not reproduce them.

Throughput differed far more. Replaying the routing requests at rising concurrency, Jev handled 64 at once, about 8,000 requests a minute, with no errors, and its median stayed near 0.29s (287ms alone, 304ms at 64). My Microsoft deployment allowed 17 requests a minute in the data zone and 50 global, the quota my subscription had. I did not raise it or test Microsoft's own limits, so that figure says what my setup allowed, not what the service can do.

Two caveats. The prompts and option wording grew around Jev, as described under "Is the test tilted toward Jev?" above. And I sent Microsoft the same JSON state Jev receives. I tried two changes that could be fair to any model. Plain text state, which helped OpenAI, scored lower for Microsoft, 126 of 134. A single lower cut-off for every source left it at 128. Fitting a separate cut-off per source (email 0.2, files 0.2, people 0.05, teams 0.07, meetings 0.5) fixed two questions and reached 130, but those cut-offs were chosen on the same 134 questions. In five-fold cross-validation, fitting on four fifths and scoring the rest, it scored 128, no better than the default. The sources it fixed had probabilities of 0.05 to 0.09, and two such cases are too few to learn a cut-off that carries over. The four misses that remain are not cut-off problems. The model counted the requests right in two of them (four at 0.96, two at 0.77) but cut them in the wrong places, one is close to a coin flip between reasoned (0.58) and search (0.40), and one needs files, which scored 0.05. I have not tuned wording to them, since that would tilt the test the way it already tilts toward Jev.

Microsoft-Decision-1's announced price is $0.042 per million input tokens, with output free. The 134-question run used about 444,000 input tokens, roughly $0.019, and the 102-question test and its rerun about 315,000, roughly $0.013. I haven't rerun the end-to-end Work IQ races with Microsoft-Decision-1.

OpenAI's Decisions API

In October 2026 OpenAI opened a Decisions API in public beta. It is built for this kind of job: you send a set of typed questions and get back a choice with probabilities. Azure has the model behind it, gpt-6-luna, but not the Decisions endpoint yet, so I tested it on OpenAI directly with the same questions Jev gets.

It scored 128 of 134, three questions from Jev's 131. From Singapore its median for a full routing plan was 0.59 seconds against Jev's 0.41. Pricing is input tokens only, and all my runs together cost well under a dollar.

Neither service runs in Singapore, where I tested from. Both answer through a nearby network edge, but an unauthenticated request, which the server rejects without running a model, still takes about 0.25 seconds to come back from either one. That fits servers in the US. OpenAI reports its own processing time: about 0.15 seconds for a small decision, out of 0.4 seconds end to end. So most of the time for both services is distance. Replayed from the US, both answer a full routing plan in under 0.2 seconds, and the gap between them nearly disappears (see "Measured from the US" below).

Open decision models

Jev is a hosted service. For resilience, cost and data control, I wanted to see how open decision models I host myself would do. Several were released recently and accept the same request format, so each one dropped into the test unchanged.

I ran them on a single rented A100 GPU in Azure Container Apps, and the smallest on a T4. Container Apps made this the easy part. Serverless GPUs need no VM, driver or CUDA setup: one container image serves the model, and the app gets an HTTPS endpoint with its own key. Trying the next model meant pointing the same app at different weights, and billing is per second, so the GPU cost only what the tests used. Scores use the realistic question set and the clearer prompt wording described under "Is the test tilted toward Jev?" above.

The first three rows are the hosted Decision APIs. The rest are models I ran myself, ordered by score. Jev, OpenAI and pplx-decider show the median time for a full routing plan from the replay in "Measured from the US", so they compare like for like. Microsoft is per API call, because its quota let me replay only some requests (see "Microsoft's Decision API"). The other self-hosted rows come from the original runs, where Jev took 0.3s a question, all from Singapore.

• Microsoft-Decision-1 scored 128, level with OpenAI's Decisions API. Its quota allows 17 requests a minute, so I could not time a full plan and give a per-call figure instead (see "Microsoft's Decision API").
• OpenAI's Decisions API scored 128 on each of three runs. From a US container it answers a full plan in 0.19s, close to Jev (see "OpenAI's Decisions API").
• pplx-decider ties Jev at 131, but at 3 to 3.5 seconds a question on one A100, from either place, it is too slow to route live as I ran it. The time reflects the single GPU more than the model itself (see "Where the time goes").
• Quyet is close behind at 127.
• Winnow 12B scores 122 at about half the time of pplx.
• Clef-Flash is the fastest of the larger self-hosted models.
• Laya is far smaller than the rest, and its authors describe it as a base to fine-tune rather than a ready router. As released it scored 53, so I fine-tuned it.
• d1-3B came out as I was finishing this post and scored 109 with no training, level with my fine-tuned Laya. I only ran it on the T4, which is why it is slow here: the model itself took about a second for each routing request. In bf16 it scored 111, but a T4 runs bf16 slowly, so the time rose to 6.5s; the table shows the fp16 run.

Fine-tuning Laya on my own routing questions

I asked gpt-5.4-mini for 1,175 new workplace questions across the 26 topics in my test, with fictional names only. I dropped the 77 that were close to any of my 134 test questions, ran the remaining 1,098 through my router, and labelled every routing request with Jev's answer probabilities. That gave 2,322 training requests.

Training used Laya's own fine-tuning code and default settings, for three epochs. It took 17 minutes on one A100.

• Accuracy: 110 of 134, up from 53. On held-out training-style questions, accuracy went from 0.37 to 0.93.
• Speed: the model itself answers a routing request in about 33 milliseconds on the A100. Most of the 0.63 seconds per plan is the network from Singapore to West US 3.
• The gap: it routed single questions well across every topic, but none of the seven questions that join several requests in one sentence.

Only Laya was fine-tuned. Jev is TypeSafe's hosted model and I used it as served, so its 131 is untouched. Because Laya learned from Jev's answers, it is copying Jev, and it would be hard for it to beat Jev this way.

The data and scripts are at github.com/hieunhums/laya-workiq-routing.

General LLMs as the router

I also gave five Azure OpenAI models exactly the questions Jev gets, through a small adapter that makes one call per request and limits the answer to the offered options. Reasoning was switched off.

gpt-5.4-mini is within a few questions of the best decision models, with nothing to host. The smaller models agreed with fewer of the expected routes. My option descriptions are short and rely on fine distinctions, such as "a record" versus "a search of one source", and smaller general models seem to need those spelled out more.

Run from the same Azure region as the models, each of them is about a second faster (see "Measured from the US" below), and gpt-5.4-mini answers a full routing plan in 1.4 seconds. That remaining time comes from what each call does. A general LLM reads the prompt and writes its answer as text tokens inside a structured output. A decision model is built to read the prompt once and return probabilities for the offered options directly, so there is less to generate per request.

Leaderboards and this test

Public leaderboards for these models place Jev below several open models, while here it scored at or near the top. A few reasons:

• Different tests. Leaderboards average general benchmarks like reasoning puzzles, sentiment and fact checking. This test measures one narrow skill.
• Home advantage. The prompt wording was refined over many rounds against Jev's results, so the test suits it.
• Small gaps. On the main leaderboard index, Jev and the top open models are within about one and a half points.

Leaderboards are a good guide to which models to test. Your own test shows which one fits your task.

Where the time goes

Jev and the Decisions API answer in 0.3 to 0.5 seconds. The open models I hosted took 0.7 to 4. Measured per request, their time went to three places:

1. Reading the prompt. Each routing plan sends one block of text (the user's question and the option descriptions) with 8 questions about it, such as "should this search email?". Some servers treat those as 8 separate jobs and process the same text 8 times. Clef processes it once and answers all 8 from that, which is why it was the fastest of the larger models.
2. Waiting in line. The app sends two requests at once, and one GPU served them one after the other.
3. Distance. The GPU was in the US and I am in Singapore, which adds about half a second.

Prompt caching, more GPUs and a nearer region each address one of these.

Measured from the US

Distance changes speed, not which route a model picks, so I reran timing only. I recorded the 293 requests behind one full run (146 routing plans) and replayed exactly those requests against every router: once from my machine in Singapore, and once from a small container in the Azure region nearest each service. Requests that went out together in the app went out together in the replay. The table shows the median time for a full routing plan.

A few things stand out:

• Distance is a large share for the fastest routers. From Singapore, Jev and the Decisions API spend more time on the network than on the decision. From the US, both answer a full plan in under 0.2 seconds. Jev was 0.15 seconds from West US 3 and 0.18 from East US 2; the Decisions API was 0.19 and 0.20.
• The order nearly holds. Every router got faster, by 0.26 to 1 second. Only gpt-4o-mini and gpt-5.4-nano swap places. Accuracy does not depend on location.
• Same-region general LLMs land at 1.4 to 2.3 seconds. That is the time to read the prompt and write the structured answer as tokens.
• pplx-decider drops 0.55 seconds, matching the distance bar in the chart above. The remaining 3 seconds is the single-GPU serving setup.

A few calls to the Azure models timed out in the US run (two plans per model out of 146), and those plans are left out of the medians.

What I took away

• Every Work IQ tool has its place. A 1-second record read, a 10-second search, a full composed answer from ask. Matching the question to the lightest tool that fully answers it is the largest single latency gain.
• Decide in one step, then check. A Decision API picks the tool and the scope in a fraction of a second, and a second short call checks the result before replying, so ask is still there whenever it adds more.
• Build your own test early. 134 questions with expected routes caught every regression and made every comparison concrete.
• Measure on your own task. Prompt style guides, leaderboards and the model's own confidence each gave a different picture from my test.
• Gate on outcomes, not confidence. Move on to a wider tool when a lane's result does not answer the question, not when the model sounds unsure.

Where I landed

The top routers in the original test are close: Jev at 131, then Microsoft-Decision-1 and OpenAI's Decisions API at 128, on a test whose wording grew up around Jev, and a few hundredths of a second between Jev and OpenAI's Decisions API when both are called from the US. pplx-decider also scored 131 but took about 3s a question on my GPU. Any of the hosted three would make a workable router.

The published app still uses Jev, so its screenshots and end-to-end timings remain evidence for that integration. The next step is wiring Microsoft-Decision-1 into the app and repeating those races, rather than assuming the same timings carry over. Microsoft-Decision-1 adds a first-party Foundry option alongside Jev and OpenAI's Decisions API. pplx-decider is the strongest self-hosted option in my earlier tests. Fine-tuning a 421M Laya for 17 minutes took it from 53 to 110, so a small model trained on my own routing questions gets a long way. d1-3B matched that with no training at all, so I will try it on a faster GPU and fine-tune it too. Next, I'll work on questions that join several requests in one sentence, and on sending summary requests straight to ask.

With the tool chosen in under half a second, most questions answered by a 1-second fetch or a focused search, and ask ready for the rest, the voice app now answers "Who is my manager?" about as fast as a person would.