The Twelve Days of Blog-mas: No.12 - Copilot(s) - Your AI Assistant(s)

MichaelHildebrand

Microsoft

Dec 15, 2023

Tally ho!

Now, you didn't really think I would go for 12 without one about Copilot, did you?

Our AI/ML efforts have been on-going for a long time, but very recently, they've gone mainstream -and SUCH a cool logo/icon. Be aware, though, for now, this space changes frequently, varies by region/market and software version (Windows, Office apps, Edge, etc.). Docs, product names, major and minor functionality are all moving very fast. Do your brain a favor and make some peace with that - but then, jump into the pool!

So, what IS Copilot?

After working with these technologies a lot recently, I like the simplest explanation: Copilot is your AI assistant.

Basically, Copilot is a system that allows you to offload a little or a lot to a processing system that can help save you time and make you more effective, as well as generate creative ideas for you to run with.
- Copilot for M365 can help you from within your Office 'world,' to create/summarize information across services
  - For example, it can create a concise bullet point list or draft a summary, based on information that it has access to from emails and a meeting last week - while you were out of the office.
  - Your Copilot for M365 system has access to your M365 content/data (i.e. Word docs, Outlook emails, Teams meetings and messages).
  - The prompts/results can be audited/retained as part of its integration with your org's Purview system - compliance settings, e-discovery and retention policies, as well as sensitivity labels and protections
- Copilot w/ Commercial Data Protection is a ‘back and forth’ ad hoc 'chat bot' way to send data/ideas/questions off into the AI/LLM processing system to get useful results or ‘getting started’ content
  - Unlike Copilot for M365, this Copilot experience doesn't have standing access to your M365 content/data - it isn't aware of it.
  - The prompts/results are not audited nor retained beyond that session – think of this as a transient use the AI/ML/LLM ‘system’ to help you at a point in time.
- Security Copilot can sift through a massive array of various ‘security’ data/signals. It can offload the effort and help find root-cause quicker than sifting through the data yourself. It could be an endpoint analyst 'asking questions' (via prompts) for details about a device or its recent activities. It could be used to generate a complete Conditional Access Policy, based on a 'prompt' describing what is needed.
- Copilot in Windows provides you with an AI assistant for using/configuring Windows settings, as well as a ‘canvas’ (i.e. the side-bar Copilot UI fly-out) to interact w/ the Copilot chat experience
- Copilot in Edge provides a browser-based side-bar experience to interact with the Copilot chat experience (similar to the Copilot in Windows experience)
- There are others – Dynamics, Azure, etc. - and there will be more.
  - Heck, you can even make your own!
    - Microsoft Copilot Studio | Extend Copilots or Create Your Own
    - Azure AI Studio - Generative AI Development Hub | Microsoft Azure

As mentioned, the Copilot for M365 system has the added context of your O365 data such as emails, meetings, docs, etc. and takes the input from you (the "prompt") and shoves all of that into one of our private AI/LLM Copilot orchestrator to mash it all up, reason over the results and then return enriched insights and responses to you - in ‘natural’ language.

But it can be iterative - it's not a 'one and done' - it can ‘take turns’ with you
1. “Summarize this file” ...
  - <Copilot processes the file and returns a summary of it to you>
2. “Make that summary 3 bullet points” ...
  - <Copilot processes the summary and returns 3 bullet points to you>
3. “Draft an email with those 3 bullet points” ...
  - <Copilot creates an email draft in Outlook with the 3 bullet point summary>
4. "Create a PowerPoint slide with those 3 bullet points" ...
  - <Copilot creates a new PowerPoint slide with those 3 bullet points>

The Copilot for M365 Train Ride

I think of the Copilot for M365 processing along the lines of a round-trip train ride that begins with your prompt.

The prompt-train leaves the station and travels to several other stations along the journey, picking up details and context, stopping here and there, where other activities take place
- Stopping of in your O365 Graph to see if there is anything related/of interest to your prompt there (emails, docs, Teams chats)
- Here and there, numerous AI checks and balances are performed against the prompt to help ensure safe AI uses
- Then it heads into the Copilot Grand Central Station, where the AI/ML/LLM orchestration model does its thing
- Then, the output may stop off at the Purview station, on the way back for any auditing, logging, or labeling.
- Then the train returns where it left, your Copilot UI, where it shows you some souvenirs and you see how it has been transformed from how it left.
The user doesn’t see it, but that initial prompt has been on quite a journey.

AI Image generation

This was fun – https://www.bing.com/images/create (you’ll need to sign in with a personal Microsoft ID vs an Entra ID)
This seems sorta like working with an AI police sketch artist?
- You type in words, phrases and descriptions (a 'prompt') and the AI/ML models use them to generate an image.
- I started with something somewhat related to this blog series:

Prompt: holiday themed cartoon santa technology microsoft laptop mobile phone

Dude. Seriously Santa? XP? Really?

But then, I sorta fell off the wagon...

For this one, my kids typed in some of my own physical traits - and although it doesn't really look like "me," there is a resemblance:

Prompt: cartoon style middle aged man with a smile, wearing a blue hoodie that says Hilde, hazel eyes, gray face stubble shaved head bald on top. Holding a laptop. text 3D rendering, typography, illustration, painting, photo, poster