Copilot Dashboard update – Features and data interpretation guide
Published Jun 12 2024 09:00 AM 5,660 Views
Microsoft

Since the Microsoft Copilot Dashboard became generally available on March 27th we’ve seen a huge amount of interest and usage from customers seeking to understand the impact of their Copilot for Microsoft 365 investments.


We’ve listened to your feedback, and today we’re pleased to share a set of feature announcements and best practices to help you maximize your Copilot Dashboard usage.


As a reminder, starting in 2024 Q3, the Copilot Dashboard will no longer require a Viva license to access, and will be included in your Copilot for Microsoft 365 subscription.


To access the Copilot Dashboard, simply navigate to aka.ms/CopilotDashboard from your browser or open the Viva Insights app in Microsoft Teams.


Copilot Dashboard Data Interpretation Guidance


Before we dive into features and roadmap, let’s begin with a set of 7 best practices gathered from customers and our internal data analytics team at Microsoft.


1. Understand the dashboard metrics and how they are calculated


It’s well worth your time to read the Copilot Dashboard online documentation so you can articulate the inputs that feed into dashboard metrics. This resource is continually updated and will help you tell the Copilot ROI story with deep dives on metrics like Assisted hours. See also our recent Worklab article on how we measure the value of AI at work.


2. Pair Copilot usage data with sentiment analysis


If your surveyed users are reporting high satisfaction with Copilot and telling you how it enables them to be more agile and synthesize more information, that's a powerful signal. Keep in mind that surveys are a leading indicator of benefits. Habits take time to develop and reflect in the data as behavioral changes. Utilize the sentiment analysis capabilities of Copilot Dashboard to get an early pulse on benefits while waiting for developing behavioral adaptations.


3. Understand that Copilot usage metrics and behavioral metrics are not causally related


The behavioral metrics included in Copilot Dashboard are influenced by a wide range of work and personal factors for each measured employee. Especially at the beginning of this transformation when volume of usage is still picking up, it is important to remember that Copilot is very likely not solely responsible for any behavioral metric (i.e. meetings hours, emails sent, etc) differences shown in the dashboard.


4. Ensure measured Copilot users have received skilling and adoption resources


We should not expect Copilot to drastically change user behavior or output without the proper skilling and adoption enablement. Has your organization clearly communicated, with leadership support, Copilot availability and expectations to users? Delivered skilling content to new Copilot users around prompt engineering and scenario guidance? Set up early adopter communities to share best practices?


As with any new tool, users require resources to accelerate adoption. The Viva Suite contains a set of purpose-built tools to do exactly this:


Copilot Academy in Viva Learning to drive AI and Copilot user upskilling
Copilot community templates in Viva Engage to enable best practice sharing and answer questions
Copilot communication templates in Viva Amplify to quickly get the word out on Copilot availability and expected actions
Copilot Dashboard in Viva Insights to measure Copilot adoption and impact


5. CSV upload & classifying organizations


To get the most out of the Copilot Dashboard, consider making an organizational data upload to customize the attribute filters visible in the dashboard. An organizational data upload is required to access the Job function filter within the dashboard.


6. Ensure licenses have been assigned for sufficient time before conducting analyses


Like all new technology, employees will utilize features more effectively as usage and familiarity increases. Ideally, we recommend allowing 2-3 months after adoption, per our research, before conducting analysis of Copilot impact. We recommend refraining from transferring licenses between users during the time of study as this makes it challenging to compare groups to determine impact.


7. Leverage custom reports from Viva Insights analyst workbench to go deeper


Custom reports available through Viva Insights analyst workbench provide an opportunity to deep dive into areas of Copilot reporting that may not be available from the Copilot Dashboard.


Note: you must be assigned the Viva Insights Analyst Role to access analyst workbench.


Some advantages of utilizing analyst workbench include:

  • Ability to pull usage data older than 28 days
  • Ability to slice usage data for groups smaller than 10
  • Ability to pull in custom organizational data – for example sales performance or geography – to identify business impact correlations
  • Get row level data for weekly Copilot usage so you can build custom views and dashboards
  • Ability to customize the definition of active Copilot users and explore usage intensity

For a walkthrough on how to utilize analyst workbench for Copilot analysis, see this video.

 

Hopefully the 7 best practices above are a helpful guide as you leverage the Copilot Dashboard. Now, let’s take a look at some recently released and upcoming features.


Recently released and upcoming features


Copilot assisted hours with calculation breakdowns - [generally available now]


Copilot assisted hours is an estimate of the total time employees were assisted by Copilot over the past 28 days. The metric is computed based on your employees’ actions in Copilot and multipliers derived from our research on Copilot users. The metric should be viewed as a general estimate based on the most relevant Copilot usage data and research currently available. The underlying calculation methodology will evolve over time as new information becomes available. Select ‘How do we estimate this’ to see a breakdown of how the total figure is calculated.

 

Copilot assisted hours 1.png

 

Copilot assisted hours 2.png

 

Date ranges - [generally available now]


In the Copilot Dashboard you will now see a specific date range identified in the upper right corner to clarify the measurement timeframe. This date range reflects a trailing 28 day period and is not configurable. For a custom date range analysis, use analyst workbench.

 

date range.png

 

Minimum aggregation group size threshold - [generally available now]


On the Adoption and Impact pages, you can view aggregated user-level metrics for groups that meet or exceed the minimum aggregation size set by your Viva Insights admin. By default, this is set to 10 employees. The minimum it can be set to is 5.


For tenant level metrics in the dashboard, minimum group size restrictions have been removed.

 

Minimum aggregation group size threshold.png

 

Copilot for Microsoft 365 metrics for custom reports - [generally available now]

 

Available from analyst workbench in Viva Insights, you can now add Copilot for Microsoft 365 metrics to custom reports. Analysts can now find these metrics in the metric library and in the “add metrics” step in a person query under the “Microsoft 365 Copilot” section. As part of this update, the Copilot-related metric names have also been updated to be more intuitive.

 

Note: if you have a custom Power BI report built based on a query including Copilot metrics, you may have to rerun your query (in the case of a non-recurring query) and update the logic in your Power BI report to reference the updated Copilot metric names. All Copilot metric names can be found in the documentation: Person query | Microsoft Learn.

 

AWB Copilot metrics.png

 

Trendlines – [available by end of July 2024]


6-month lookback trendline views will now be available from the Adoption and Impact pages. The Adoption page trendline shows Copilot adoption trends across the trailing 6 months, filterable by either # of Copilot active users, % of active Copilot users, # of Copilot licensed employees, or % of Copilot licensed employees.


Note: Active Copilot users are defined as the number of users who actively used Copilot at least once in any of the following Microsoft 365 apps during the last 28 days: Microsoft Teams, Outlook, Word, Excel, PowerPoint, OneNote, Loop, and Copilot chat (formerly Microsoft 365 Chat). More apps will be added over time.

 

Adoption Trendline.png

 

The Impact page trendline allows you to deep dive into a trailing 6-month view for Copilot impact metrics (e.g. Meeting hours summarized by Copilot) with filters available associated with the impact metric selected.

 

Imact Trendline.png

 

Delegate access - [available by end of June 2024]


Users with access to the Copilot Dashboard will soon be able to delegate access to other people within their company. By doing so, someone else such as a chief of staff or one of your direct reports, would have the same access you do to Copilot Dashboard.

 

Delegate access.png

 

Adoption PBI template - [available by end of June 2024]


Available from analyst workbench in Viva Insights, the new Adoption PBI template enables a deeper look into adoption trends, including a dynamic date slicer, trendlines, and highlight section calling out top actions.

 

Adoption_Summary_Page.png

 

Impact PBI template - [available by end of June 2024]


Available from analyst workbench in Viva Insights, the new Impact PBI template enables a deeper look into impact areas, including a dynamic date slicer, Copilot assisted hours and value calculator, group comparisons, and the ability to customize active usage definitions.

 

Impact_Summary_page.png

 

We hope you find the guidance and updates above useful. As always, we value your feedback! Please leave your comments below.


For additional information to get started, please read our technical documentation at

aka.ms/CopilotDashboardDocs

 

Not a Viva Insights customer? Check out our latest features, demos, and pricing. Please reach out to your Microsoft representative for more information.

10 Comments
Co-Authors
Version history
Last update:
‎Jun 11 2024 04:29 PM
Updated by: