How to identify the most impactful GenAI use case

Microsoft

Jun 12, 2024

Generative AI solutions are very transformational for businesses nowadays. Organisations have access to tools and AI capabilities that will enable them to generate immense value for the business.
At the same time though, some businesses are struggling to identify the areas they should invest in AI. Most of the AI engagements fail because there isn’t really a business value that the AI solution is delivering. At times we find ourselves in a situation whereby we have an AI solution looking for a problem.
This is what leads to failure. In order to make sure a Generative AI or AI solution is successful, we need to make sure it’s clear the business value it will deliver before any technical work starts. It’s very important that we work with business and technical decision makers to make sure they are aligned on where to invest in AI, why they think it’s a good idea for their organization to invest and what is the expected business outcome the solution is supposed to deliver. AI experts need to work alongside the customer’s decision makers to engage in a Value Creation exercise. The objective of the Value Creation exercise is to identify those High Value use cases that will impact key business processes of an organization, enterprise-wide and deliver the highest business impact; consequently making clear the Value that the AI solution will deliver. The Value can be anything: cost savings, increase profit, put a new service on the market, employee productivity, etc.

When we work in a GenAI project, we should work alongside our customers and help uncover the following points:

What would we like to achieve in the AI space and by when?
To whom is this AI initiative important? Who is sponsoring it and why?
What are the core business processes within our organisation that deliver the highest business impact?
What are the core business processes within our organisation that would benefit from AI?
How are we measuring the impact of the core business processes?
Where do we think there is the highest ROI for our business if we were to invest in AI/Gen AI?
Where is the data at the moment? (i.e: Cloud, on-premise waiting for a DC migration?)

The above will help focus the conversation on what really matters for a given organisation that is planning to invest in AI.

Ultimately this conversation should be leading the business and technical decision makers to a list of use cases organised by: priority, complexity, business impact and ROI. Similar to the table below:

Use Case	Business Area	Priority	Complexity	ROI
M&A	Legal	H	M	£xM
Document drafting	Legal	H	H	£xM

To derive the ROI, it is important for the organisation to work on a Strawman business case. The Strawman business case is an estimate of the value that the AI solution in scope is expected to deliver to the business.

Below is an example template of Strawman business case in the legal space:

The Strawman business case will help the business and technical stakeholders where they should be focusing their investment in AI.

Once a use case or list of top X use cases have been identified, it’s important to derive the metrics and KPIs of the use case/s. These metrics, will be the same that the PoV will need to measure so that the business can validate the expected ROI which is delivered by the PoV and be able to extract the expected ROI once the solution is in production.

After the KPIs and metrics have been identified, it’s important for the business and technical stakeholder to understand if the ROI is compelling enough to invest in AI. If the business decides to press ahead, it’s important to create an evaluation team which is normally made by domain experts or end users of the AI solution.

The evaluation team is tasked to review/validate the output of the solution, collect the KPIs and at the end of the PoV to derive and validate the actual ROI the AI solution is delivering.

Once we have identified the right use case to focus on alongside the business case/ROI the GenAI solution will deliver, it’s necessary to understand which type of Copilot to use. That is, an out of the box GenAI capability like M365 Copilot or build a custom Copilot.

M365 Copilot is a GenAI SaaS offering from Microsoft that enables organisation to quickly adopt GenAI to extract insight not only within the M365 ecosystem but also within external systems too(i.e: SQL DB, etc).

The experience an end user gets with M365 Copilot is directly linked to how “good/precise” the prompt is and the quality of the data that particular end user has access to.

The M365 Copilot indexes all the data the end user has access to, so it's important to remember that a given user has access to working progress data and final data. Ultimately the quality of the output of a prompt is also linked to the quality of the data besides the quality of the prompt itself.

What if we need to be in control of:

The knowledge base(data) the GenAI model is grounded to/has access to
Prompt structure and how the prompts are executed
Evaluate the quality of the output of the prompts
Be able to get back to the user and ask for more information if the submitted prompt is “too generic”.

This is where a Custom Copilot will come handy.

When we talk about Custom Copilot, we have two options:

Low Code/No code option with Copilot Studio
Code first approach with Azure OpenAI

Copilot Studio or the ex PVA is a cloud service that enables organisation to build intelligent Bots powered by GenAI(if needed)

As the development team is designing the interaction between the end user and the Intelligent Bot, we can cater for all those scenario whereby we need to ask the end user for more information if the prompt is not too accurate or if we need to augment the data from an external system.

Copilot Studio comes with integration with Azure OpenAI or the wider Azure AI ecosystem and non - via API integration

Being a Low Code/No Code development tool, Copilot Studio works well for those use cases whereby it’s not required a huge data integration, aggregation, filtering etc before providing context to the GenAI model to execute a prompt.

Also, if a given organisation needs to be in control of the amount of compute the GenAI application needs alongside a pro code development approach, then this is where the Custom Copilot built with Azure OpenAI will help.

Via the Azure AI Studio developers have access to best in class data integration, LLM models, model evaluation, LLM Ops etc to build GenAI application that can be exposed within an organisation or externally to its own customers.

Moreover with Copilot Extensions available within Copilot Studio, we can build a custom plugin around our custom Copilot (built either with Azure AI Studio or Copilot Studio) and publish it to M365 Copilot. This allows it to leverage more specialised Copilots as and when needed from within a single UI.

Below is a decision tree that helps navigate among the different Copilots and when to use which Copilot: