The Workplace Analytics team is excited to announce our feature updates for April 2021. (You can see past blog articles here). This month’s update describes a coming attraction: Collaboration and Manager metrics with Teams IM and call data.
Coming soon: Collaboration and manager metrics with Teams IM and call data
Overview
We are pleased to announce a new feature release for April 19, 2021. The new release includes a few exciting updates:
- Integration of Microsoft Teams chats and Teams calls into the Collaboration hours metric and refined Collaboration hours logic to better accommodate overlapping activities
- New “is person’s manager” and “is person’s direct report” attributes available in Person-query participant filters. (Please note that these became available on March 29, 2021.)
Additionally, we’ve implemented a handful of improvements to other metrics:
- Outlier improvements to Email hours and Call hours metrics
- Better alignment of After-hours collaboration hours and Working hours collaboration hours to total Collaboration hours, and of After-hours email hours and Working hours email hours to total Email hours
These updates reflect customer feedback and help leaders better understand how collaboration in Microsoft Teams impacts wellbeing and productivity.
How do these changes impact you?
For Workplace Analytics analysts
- Adjusted results - If you are accessing insights in Microsoft Viva Insights app in Teams or the Workplace Analytics web application, running custom queries, or using Workplace Analytics Power BI templates, some of the aggregated results might show different numbers than previously seen.
These changes will not impact any queries that have already been run and saved, and starting April 19, 2021, new queries and calculated insights will use the new logic over the entire historical period of collaboration data. If you are in the middle of an active project that uses these metrics, we recommend re-running your queries to update the results with the new versions of the impacted metrics.
- “Unstacking” the Collaboration hours metric - You might be used to seeing or using visualizations that “stack” the components of collaboration time (like Meeting hours and Email hours) to get to the total collaboration time. But since emails and Teams chats and calls can occur during meetings and Teams calls, we’ve refined the logic so that these “overlapping activities” only count toward total collaboration time once.
As a result, expect to see Collaboration hours that is no longer just the sum of its parts – Email hours, Call hours, Teams chat hours, and Meeting hours. If you have reports and visuals that compare collaboration hours with its parts, you might want to adjust the report to show these components side by side instead. For example, the Workplace Analytics Ways of working assessment and Business continuity templates for Power BI both previously included examples of this “stacked” view, which will reflect revised visuals when you download the newest versions of the templates.
- New manager measures - Want to know how many emails the average manager sends to their directs? Or whether managers on a team tend to use unscheduled calls instead of scheduled 1:1s? You are no longer limited to just the “built-in” manager meeting metrics in Workplace Analytics.
If you are interested in understanding how employees communicate with their direct managers, you can create new custom measures in a Person query to measure the meeting, email, call, and chat activity where any or all participants are the measured employee’s direct manager or direct report.
- Impact on Plans - If you are currently running a Wellbeing plan to reduce after-hours collaboration, you might observe a shift in the baseline After-hours collaboration hours metric, which might cause the goal that was selected for the plan to no longer make sense. If this is the case, we recommend requesting a deferral of this feature so that ongoing plans can finish running undisturbed.
For manager or leader insights
Shifting points of reference - If you are used to seeing a specific result for some metrics (for example, “I know that our average email hours are usually around 8 hours per week, and that’s something we’d like to reduce.”), the new changes will likely change the results. However, that baseline number might shift as a result of the improved methodology. If you are working directly with a Workplace Analytics practitioner from Microsoft, a partner, or your own organization, they can help you evaluate whether this raises any new considerations for ongoing projects.
What if you want to get these features sooner than April 19, 2021?
To sign up for early access, please complete this online form.
Can you defer this release?
Expect to see some shifts in the results for metrics impacted by these changes (full list below). If you are in the middle of an active project that uses these metrics, we recommend re-running your queries to update the results with the new version of collaboration hours.
If this shift would be disruptive to your project, you can optionally request a one-time deferral of this feature release for up to three months. Please complete the online form by April 15, 2021, if you would like to request a one-time, three-month deferral.
Additional details about the changes
Integrates Microsoft Teams chats and calls into Collaboration hours and related metrics
The legacy Collaboration hours metric simply added email hours and meeting hours. However, in reality, these activities can overlap. Collaboration hours now reflect the total impact of different types of collaboration activity, including emails, meetings, Teams chats, and Teams calls. With this release, Collaboration hours capture more time and activity, and adjusts the results so that any overlapping activities are counted only once.
The following queries and metrics will reflect this new logic:
Person query and Peer analysis query
- Collaboration hours
- Working hours collaboration hours
- After hours collaboration hours
- Collaboration hours external
Person-to-group query
- Collaboration hours
Group-to-group query
- Collaboration hours
The following join the other metrics that already include Teams activity:
Person query and Peer analysis query
- Workweek span
- Internal network size
- External network size
- Networking outside organization
- Networking outside company
Network: Person query
- Influence
Network: Person-to-person query
- Strong and diverse ties
Adds new “Is person’s manager” and “Is person’s direct report” metric filter options
We're adding new participant filter options to our email, meeting, chat, and call metrics for Person queries. These new options enable you to filter activity where all, none, or at least one participant includes the measured employee's direct manager or their direct report.
You can use the new filters to customize any base metric that measures meeting, email, instant message, or call activity (such as Email hours, Emails sent, Working hours email hours, After hours email hours, Meeting hours, and Meetings).
Selecting the "Is person's direct report" filter to customize a metric
The following are examples of some custom metrics you can create in a Person query with these new filters.
|
Analysis question |
Definition |
Base metric |
Customized filter |
|
How much time do employees spend chatting with their manager? |
The number of hours the person’s manager spent talking to the person through IMs |
Instant message hours |
(Participant: At least one participant’s: Is person’s manager = True) |
|
How often do managers use unscheduled calls for 1:1s with their direct reports? |
Total number of hours that a manager spent in 1:1 calls with their direct reports |
Call hours |
(Call: Participant Count =2) AND (Participant: At least one participant’s: Is person’s direct report = True) AND |
|
How much discussion between employees and their manager occurs via email? |
Total number of hours that a person spent in emails with their manager |
Email hours |
(Participant: At least one participant’s: Is person’s manager = True) |
Improves outlier handling for Email hours and Call hours
When actual received email data is not available, Workplace Analytics uses logic to impute an approximation of the volume of received mail. We are adjusting this logic to reflect the results of more recent data science efforts to refine these assumptions. Further, we have received reports about measured employees with extremely high measured call hours. This was a result of “runaway calls” where the employee joined a call and forgot to hang up. We have capped call hours to a maximum of three hours to avoid attributing excessive time for these scenarios.
The following queries and metrics will use the new logic:
Person query and Peer analysis query
- Collaboration hours
- Working hours collaboration hours
- After hours collaboration hours
- Collaboration hours external
- Email hours
- Working hours email hours
- After hours email hours
- Call hours
Person-to-group query
- Collaboration hours
- Email hours
Group-to-group query
- Collaboration hours
- Email hours
Better aligns working hours and after-hours metrics with their respective overall metrics
Previously, After-hours email hours plus Working hours email hours and After-hours collaboration hours plus Working hours collaboration hours did not add up to total Email hours or Collaboration hours, because of limitations attributing certain types of measured activity to a specific time of day. We improved the algorithm to better attribute time for these metrics, resulting in better alignment between working hours and after-hours metrics.
The following queries and metrics will reflect the new logic:
Person query and Peer analysis query
- Working hours collaboration hours
- After hours collaboration hours
- Working hours email hours
- After hours email hours
Impacted metrics by query type
|
Person and Peer analysis queries |
|
|
Collaboration hours |
The number of hours the person spent in meetings, emails, IMs, and calls with at least one other person, either internal or external, after deduplication of time due to overlapping activities (for example, calls during a meeting).
|
|
Working hours collaboration hours |
The number of hours the person spent in meetings, emails, IMs, and calls with at least one other person, either internal or external, after deduplication of time due to overlapping activities (for example, calls during a meeting), during working hours.
|
|
After hours collaboration hours |
The number of hours the person spent in meetings, emails, IMs, and calls with at least one other person, either internal or external, after deduplication of time due to overlapping activities (for example, calls during a meeting), outside of working hours.
|
|
Collaboration hours external |
The number of hours the person spent in meetings, emails, IMs, and calls with at least one other person outside the company, after deduplication of time due to overlapping activities (for example, calls during a meeting).
|
|
Email hours |
The number of hours the person spent sending and receiving emails.
|
|
After hours email hours |
The number of hours the person spent sending and receiving emails outside of working hours.
|
|
Working hours email hours |
The number of hours the person spent sending and receiving emails during working hours.
|
|
Generated workload email hours |
The number of email hours the person created for internal recipients by sending emails.
|
|
Call hours |
The number of hours the person spent in scheduled and unscheduled calls through Teams with at least one other person, during and outside of working hours.
|
|
After hours in calls |
The number of hours a person spent in scheduled and unscheduled calls through Teams, outside of working hours.
|
|
Working hours in calls |
The total number of hours a person spent in scheduled and unscheduled calls through Teams, during working hours.
|
|
Person-to-group queries |
|
|
Collaboration hours |
The number of hours that the time investor spent in meetings, emails, IMs, and calls with one or more people in the collaborator group, after deduplication of time due to overlapping activities (for example, calls during a meeting). This metric uses time-allocation logic.
|
|
Email hours |
Total number of hours that the time investor spent sending and receiving emails with one or more people in the collaborator group. This metric uses time-allocation logic.
|
|
Group-to-group queries |
|
|
Collaboration hours |
The number of hours that the time investor spent in meetings, emails, IMs, and calls with one or more people in the collaborator group, after deduplication of time due to overlapping activities (for example, calls during a meeting). This metric uses time-allocation logic.
|
|
Email hours |
Total number of hours that the time investor spent sending and receiving emails with one or more people in the collaborator group. This metric uses time-allocation logic.
|