Microsoft Teams Release Processes - Why do I not see a feature but my colleague does?
Published Feb 02 2021 08:00 AM 60.5K Views
Microsoft

At Microsoft Teams, we frequently hear the question, “I am running the same version as my coworker, but they have a feature I don’t. Why don’t I have that feature now? And how can I get it now?” Before we answer that question, we want to shed some light on our overall release processes so that the answer makes sense.


As a productivity tool connecting hundreds of millions of people around the world and enabling remote work, remote learning, and connections with family and friends, we take a very orchestrated approach to how we roll out and enable new features. We have multiple updates that get rolled out: web, desktop (Windows, Mac, Linux), mobile (iOS, Android), packages for conference room devices, and our backend services. Each of these packages are backed by feature flag configurations that let us ship a new Teams version and enable features separately.


When we roll out features, it consists of two activities:

  1. Shipping the version (Teams application version with code for new features included)
  2. Enabling the feature through the feature flag

Both activities happen progressively, but at different times. We first ship the build with the feature flags turned off. We progressively roll out the build to users, wait for the build to be picked up and used by users, and reach certain penetration rates. We’ll run scorecards for key performance and usage metrics between the prior build and the new build to ensure we are not introducing any form of regression with our latest version.


Once we have scorecards and confidence in the version, we can then begin to progressively enable our feature flags – making the new features available for users. For our larger, external customer facing rings, this is a multiple step process that usually happens over a few days.


Below is a high-level view of our audience segmentation. Each ring represents an audience with specific gating criteria to allow us to exit the build and the feature flag, and to progress them to the next ring.

Jessie_Hwang_0-1612282675618.png

When we begin rolling out feature flags within a ring is where you will usually see differences in features within the same version. We roll features and versions out on in increasing % tranches at user level (considering the worldwide user pool and not at an organization/tenant level) as it gives us the best cross section of use cases, hardware configurations, software configurations, network topology, bandwidth availability, etc., to validate our changes and the user experience they provide – but this does mean that co-workers on the same build can see differences in their features. Rolling out feature flags by organization introduces the potential to bias our results with similar hardware, bandwidth, and usage patterns, so we focus on getting a cross section of users and usage patterns with our rollouts.


In December 2020 we announced the availability of Teams Public Preview. Public Preview is a great mechanism to expose a subset of your user base to features a little ahead of everyone else. This allows you to get familiar with new features before they are available to all users.


Launch of Microsoft Teams preview experience and alignment with Microsoft 365 deployment channels | ...


Public preview in Microsoft Teams - Microsoft Teams | Microsoft Docs


Teams-Updates - Microsoft Teams | Microsoft Docs

 

40 Comments
Copper Contributor

We receive this question on a regular basis. Thank you for helping clarify the process that your team takes. Keep up the good work!

Silver Contributor

This doesn't help in explaining users why they don't have a feature that another user has. This article will not help. They will still think they IT is dumb or doesn't care. Even personally, i just don't care about news about new features, because i know it can take months (not days) until you see it. Like with tasks in Teams, native notifications (where?) and so on. You are killing the excitement about new features with very long deployments and delays.

Bronze Contributor

Thanks to you @Martin Rinas to sharing this. But I believe the biggest problem on the organizational level is the transparency and predictability. I could use the client version report on CQD to see which version users are using and if new version is coming, I could see how widely it has been distributed. But as you told also, that is not the full picture.

 

I cannot see those feature flags from anywhere (?), I cannot see on which rings we are, and I have no possibilities to see when certain feature rollout is starting to our end users. Are you planning to share any visibility to this, or is this just a cost we need to pay when using the cloud services?

 

And about the public preview, if we allow users to choose to use that option, how I could see who have accepted the preview, and of course in centralized? It could be a challenge for the support organization if people who have forgotten that they are part of the preview features, and start reporting odd user experiences.

Copper Contributor

This doesn't really answer the question of how to ensure all those in the same ring and in the same tenant get the updates as quickly as they can *within* that tenant.  That is the main question we (IT) face every time a new set of features is rolled out.

 

Please solve this by pushing to everyone in the same tenant for whichever release ring they are in.  This just creates more work for IT groups everywhere like it is now and always makes users question if something is working correctly or not when their colleague on the same team gets new features they do not yet have themselves, etc.

Microsoft

Thanks Martin. This is something we deal with customers on frequent basis. having a public blog definitely helps.

Iron Contributor

Hi , when we push Teams client via package using version 1.3 vs 1.4, where we can check which features are added in 1.4

Based on this blog, once the new version is deployed to the tenant and the feature flag is turned on, everyone in the tenant should see the new feature. That is not my experience. I have users that are on the latest version, but still can't see features (like Together Mode) that have been out for months. Other users in the same tenant have all of the latest features.

 

Is there something else we need to do on a device or user level to ensure new features are available?

Bronze Contributor

@Don Kirkham 

As much I have learnt, the flags and versions are not going hand by hand. The features seems to be written into the earlier versions of the clients already, but are disabled until the flags on tenant enables it. Because of that, you could have older version from client (not too old) and still get the same features than newer clients.

Silver Contributor

You are right. I was downgrading my Teams client some time ago and was surprised to see Presence scheduling option working in a much older version, when this option just appeared as available a few days ago at that point..

 

I can only guess, but maybe some users don't get Together Mode because of hardware restrictions. But i would rather guess features are not actually trickling down in a few days as MS brags. Not a first or second time i am waiting for months to get some new features.

Iron Contributor

This was not all of the information I was looking for, but it is a start. While I understand that MS needs the time to roll out features, having some users have disparate experiences tends to degrade the user experience. For instance, the Teams Together Mode. One user can have it, and others, wanting to use it will not. That creates a disparate experience where if someone wants to kick off that new experience in their next meeting, because they're excited to share what they've learned, they don't have the feature. - Bad user experience and also creating issues with level 1 support seeking further clarification as to why they don't have that feature. 

 

What I would suggest Microsoft do is create talking points that would be available through their Microsoft adoption guides. It's important to note this and not just stumble upon this information. We need to be able to speak to it and temper the expectations of our own user base. I'll be able to take this information and craft it into a watered-down version for my company. 

Silver Contributor

How about giving visibility to enabled/dormant flags? Maybe there can be a dashboard in Teams admin center showing which part of tenant users is on which version and which flags (features) are currently enabled and which are not yet for some users with an approximate ETA on enablement (which should change dynamically based on roll out progress, issues found, etc.). And maybe even in the client in some section about what's new, coming features, show a list of major features with a checkmark, if it is enabled on this client and some sort of "coming in x weeks" for the not enabled yet features.

Brass Contributor

I do support on the MS Answers forum/community.  We constantly get this question.

.

Your excuse is WORTHLESS! 

 

All we users can see is:

  • Version number,
  • build number and
  • Channel

In the rest of the development world that would be enough information to uniquely identify what features we should be able to see in our applications. Different features, different build number!

.

If you want to play silly b**gers with "feature flags", fine. But YOU MUST EXPOSE/DISPLAY those flags (with publicly available explanation documentation) TO USERS! At the very least give us documentation telling us what registry keys to look at!

.

"You" (generically MS (mis)management) are obviously oblivious to the waste of user and user corporate time (and money) and the angst you are inflicting on your users!

(have you gotten the idea that I feel strongly about this issue?)

.

Fix this problem in ALL Office/"Microsoft 365" (stupid name change, a rant for another time) applications!  Display this/these "flag(s)" in the File menu > Account command > About information. 

PLEASE!

Brass Contributor

PS: I have submitted a "feedback" in Word UserVoice "Display "Feature Flags" to users" (I intend it as a global "Office" feedback, not just Word specific):

https://word.uservoice.com/forums/304924-word-for-windows-desktop-application/suggestions/42618259-d...

Everyone who has commented, please vote and comment on this

.

Don K:

I believe your understanding is not quite on.  As I understand it, the "feature flags" are implemented at the user level, not the corporate level. I suspect they are "secret" registry keys on the computer.  Maybe flags hidden in the "MS Account" information.  Either way, users have to see this information!

Silver Contributor

Well, you should have posted it on https://microsoftteams.uservoice.com/ as this blog post is only about Teams.

Copper Contributor

How does this map to the different types of Tenants (e.g. gcc, edu, etc...). Is this process repeated for each type of tenant, or are they brought in a different points as the feature is made public?

Copper Contributor

Agree with ron S. Why it can't be just build number that would explain what version of the program I people have (=what features are available)? The updates are quire frequent, so it's quite impossible to know if everyone in my organization has the same features enable. In addition, on corporate level some features might be turned off so looking at Microsoft help pages won't help since the feature might not be turned on by my corporation. And one more annoyance is the really long period of rollouts.. New features are announced big, but the roll out to all users might take months. That's why I don't even look at the news about new features since it might take looooong time before I'm able to use it.

Silver Contributor

The traditional Office products do a good job of notifying the users about new features, it would be helpful if Teams provided a What’s New panel like they do 

Brass Contributor

This sort of makes sense, but then you find machines which are hopelessly out of date running 1.2 releases which refuse to update. Surely downloading the msi from the web and running it on a machine should be all that is required? Maybe it won't get you onto the latest and greatest version, but at it should at least bring you up to a recent version, no?

 

The same goes for all these profile installed versions. I'd prefer to have only a machine-wide copy installed. Why is this not possible? Installing the machine wide installer should eliminate the copy installed in the users folder, but it never does.

 

Things are ok at the moment, but they day someone discovers a major security bug in the client and we all have to urgently update to version 1.x this is going to become a major issue.

Brass Contributor

@IanMurphy48  -- on the part you mentioned about  --refuse to update---,    perhaps you have seen this discussion:
--Force Teams desktop client update --

https://techcommunity.microsoft.com/t5/microsoft-teams/force-teams-desktop-client-update/m-p/2056566

 

---

Copper Contributor

Respectfully, while this information is helpful for those of us who are IT admins, our end users just don't care. They are only frustrated that their coworkers have access to features that they do not. This makes change management for larger organizations very difficult - especially since now many of us are completely wedded to Teams for UC. 

Silver Contributor

Don't get me started on appdata installation :) I am now in the process of remediation RCE vulnerability in Teams and having fun going through machines and deleting ancient Teams versions from users profiles. Sometimes 5, sometimes 12 (multi user desktops). At least don't make Teams automatically start and auto install when user just logs in into system and maybe has no interest in using Teams. Also fun in VDI when working on updating images, etc. and having to close that annoying login window jumping at you every time. MS is giving you tools fighting Shadow IT and yet on another hand gladly lending it to users. "What? Administrative permissions to install apps. Please! You don't need that for Teams.".

Copper Contributor

This is an excellent blog post explaining how to ship features to users and separating deployments from releases. We run an agency and use feature flags extensively to enable features directly on our clients' websites and only for their internal team. This allow us to get the best possible feedback. We use a tool called Unlaunch due to its tiny size and overhead (had some concerns regarding other tools on page speed and performance.) 

Our server teams uses the Java SDK to separate deployments from releases. We also use kill switches for Op toggles. 

 

 

Brass Contributor

What I'm seeing is totally inconsistent with Teams mysteriously working perfectly and being up to date on some machines but on others I diligently push out the machine wide installer each few days but the user version just never updates. I have quite a few where the machine wide is 1.4.00.xxx and the user is still on 1.3.0.xxx

Worse, the msi installer will occasionally remove teams completely - with no errors logged as far as I can tell. It just disappears. I always run the msi with logging to a file and can't find any indication of why it would have just removed everything when it should have updated.

I have all of this automated, so whenever a new version comes out I push out the msi to hundreds of machines. I do this with many many applications, Teams is the worst to deal with. Acrobat comes a close second.

 

Silver Contributor

How come you are not just letting Teams update itself? It sounds like you may have created extra work 

Silver Contributor

It IS updating itself once it gets to user AppData and user is using it. Ian is doing vain work by pushing msi over and over again. If use is not using it, it will stay the same version forever and eventually will be marked by security scanners for having some RCE. You can push msi to this machine, it will not update the version in AppData. Which is the same as if the user installed it on his own via exe from MS page. We are going to have Teams rollout soon using msi and i kind of dread this. As this will only create many more vulnerabilities. Especially on multi user machines. Because system wide installer "conveniently" activates when a new user logs in and copies its version at that point to user's profile and opens a sign in window. But if user ignores that, yeah, another obsolete vulnerable in future version sitting in user profile. MS it pushing Shadow IT with this design and then sells MCAS to control Shadow IT :D

Brass Contributor

Being able to request that all users on smaller tenants (e.g. < 10,000 accounts) are synchronised on feature flags would be useful.

Brass Contributor

In other words, you're testing your code in production.  If it works, you continue rollout.  Got it.

Silver Contributor

I don't like the way you roll these features out as it is very hard to support users when the IT staff doesn't have the features before a large portion of users and even with public preview enabled for IT staff not all of those features become available in advance. Microsoft PLEASE FIX your horrible roll out system!

Copper Contributor

I totally agree with @Jeffrey Allen comments above and I am having difficulty understanding Microsoft's roll out logic. How the hell are O365 Admins supposed to stay ahead of the curve with regards to testing, updating documentation, training and pre-informing users of upcoming updates. You get to work and find out that half the tenant suddenly has new features and the other half don't. All users have the same client version and I'm tired of checking forums every single day for some sort of solution. I thought (with public preview enabled), that our admins would have the latest features weeks before the rest of the tenant - not to be. Very embarrassing for us in trying to explain this to management and users alike. How can you not roll out features at the tenant level??? And while I'm here complaining about the roll-out, thank you also for f**king with the speed dial groups as well. While half my tenant is p*ssed about not having the new features... the other half don't like 'Call History, missing overview of groups' and want to roll back!

 

I give up - have a nice weekend and..... Microsoft PLEASE FIX your horrible roll out system!

Silver Contributor

@Martin Rinas, what is the logic behind the release process if it isn't beneficial to businesses IT offices if we have public preview and can't get the features in advance to test and prepare for training? This is the end of June and I am under the impression, I'm the only member of my organization that doesn't have the new calls tab features and keep getting asked for assistance and I can't assist as I don't have the new call tabs feature and so I look like the idiot.  Fix your release cycle or drop it! Highly disappointed in the way it exists now.

Brass Contributor

I agree with @Jeffrey Allen. I have been in public preview since the beginning. I have set myself as Targetted Release user. But till today I do not have some preview features even now they have been rolling out for quite some time like the Dynamic View and Presenter Mode. 

 

I have made service request from admin center but they cannot help also, as they said those are inside  Microsoft matter that they also do not have access to them.

 

I wonder if there are some config file in AppData that we can modify to enable those new features.

 

Hope that Microsoft can provide a setting in Admin Center or user level (of Public Previewer) to enable those new feature flag as soon as they are released.

 

Thank you

Silver Contributor

@RikkyCitra, I finally got the new calling experience this morning, however, I still don't have the dynamic view yet. It would be nice to be able to make an adjustment in the Teams admin center.

Copper Contributor

Hello Martin

It is a nice article, it is clear how this work but honestly it might work for Microsoft but it does not work for corporate customers.

There is not accepted answer "You know, read this article and then you would understand why you have to wait". Even if the users are "happy" to wait then they will logically ask "OK, and how long do I have to wait? Hours? Days? Weeks?" Telling them that we cannot answer this question because we do not know the answer, this will not be accepted.

Absolutely agree with @Petri X that transparency is gone. How we can trace it? We (Company Teams/O365 admins) are responsible towards our users (customers).  

 

Furthermore, how will you distribute critical application patches? These have to be pushed as soon as possible. 

(Example VDI (limited) user experience on physical NBs)

 

You said: We first ship the build with the feature flags turned off. We progressively roll out the build to users, wait for the build to be picked up and used by users, and reach certain penetration rates.

Does it mean that users will get the new patched version (build) within hours/days/weeks? 

Could we somehow distribute it with internal tools (SCCM for instance)?

 

Thank you  
Michal

 

Brass Contributor

Btw @Jeffrey Allen , have you ever check in AppData\Roaming\Microsoft\Teams there is a file named setting.json?

When I am not in Public Preview, the first line shows that I am in ring "general". If I switch to Public Preview, I am in "ring3_6". May be you can check yours to see which ring you are in when in Public Preview. The setting.jason file can be opened with notepad.

 

Till today I still don't understand why Public Preview doesn't get some preview features activated when they are released. Yesterday I just got Dynamic View and Presenter Mode activated along with may be the whole users in my organisation, while those features supposed to be available for Public Preview months ago ahead form general users. My guess that these features are tenant level features that should be activated for the whole users at once to be working. If so, then what's the point that they released them for Public Preview. I really hope that @Martin Rinas can give clear explanation.

Thank you

Silver Contributor

@RikkyCitra, thanks and it does show I'm in ring3_6 and still don't get all of the features early. I have presenter mode but not dynamic view. I finally got the new calls tab recently after almost all of the staff did and most weren't on public preview.

Copper Contributor

I no longer need to tell our handful of users (who were always patiently waiting for the same Ms Teams features as their colleagues have had for ages) “I’m sorry, but we are at the complete and utter mercy of Microsoft; our IT department have zero say and control as to what happens with MS-Teams and some of the other office products”. 


Why?

 

… the users now say… “we’ve given up on MS Teams, because it was far too unpredictable. -  We can’t work with a system that we have to guess if a feature works or not between our colleagues”. . 

Instead, our users have chosen the predictability and stability of Zoom and Slack. Yes, I understand that these come with a higher financial price tag, but at the end of the day when you’re running a business you want some level of predictability and stability.

 

It’s a real crying shame because personally I liked that MS Teams could offer the full UC experience, but who am I to argue with all of our user-base? After all, I’m a user too and I agree with them!

 

Silver Contributor

Unfortunately we are only a drop in the ocean of customers. I always run into a problem, when i see some new feature in my client and post in our Teams news channel and then many users complain that they don't see it and i tell them wait a week or two. I think i am just going to stop posting about news as i don't like to seem like a jerk bragging about having something that others don't. And i always skip the towel posts about what's new in Teams on this site. I don't care for most of the features and i don't know when or whether they will appear.

Copper Contributor

This idea of rolling out the features works well in a public space , but fails in the enterprise space. It wouldn't surprise me because of this decision you will lose customers who will switch from Teams to an alternative just because the CEO or other Senior Manager did not get a feature all their other colleagues had.  If you think I'm kidding - I've seen this in multiple ORG's in the past. You need to allow Tenants to be able to enable/disable features within Teams across their entire tenancy or groups of people within a tenancy. If you don't want to do this - then perhaps Teams will end up being used outside of an enterprise environment and blocked from usage within.  Microsoft might like to eat its own dog food but sometimes when you feed it to someone else you might find they just don't like the taste.

Brass Contributor

It's been 3 weeks since someone reported that others have a few new features and they don't. 

I tried manually updating MS Teams without success. So why some have it and others don't 3 weeks after? Same version, same tenant. Mind boggling! 

Copper Contributor

This is garbage. My coworkers can do things I cannot. What gives Microsoft the right to segregate indiscriminately? They can't even figure out how to add a copy button to the code snippets. Who writes this stuff that everything takes so long? Is it being written by the former twitter devs?

Co-Authors
Version history
Last update:
‎Feb 02 2021 08:19 AM
Updated by: