Controlled Feature Roll-outs in Microsoft Edge

Microsoft

Feature Rollout
Hello Edge Insiders! We have so many cool new features in the works for you, and we’d like to share with you the process behind how these new features will become available.
To honor our commitment to building a great browser, we test our feature changes for any regressions by gradually rolling out the feature to a small randomly selected subset of Canary & Dev channel, and then increase this subset as we are more confident about the feature. These controlled feature rollouts begin after completing standard quality, reliability, and compliance tests and determining that the feature is at ship quality. As we progressively ramp up the percentage of users, we are validating that each new feature aligns with our promise to be a safe, fast, and delightful browser in a scientific manner.
But I’m an Insider, so I must be getting these updates first?
Every time we run a controlled feature rollout, we will randomly select a percentage of the Canary and Dev channels to get the treatment, and an equally sized group to remain in the control group. A set of guardrail metrics, including performance, reliability, browser usage, and feedback reports, are measured for each group. Statistical tests to compare the metrics between treatment and control allow us to estimate the impact of each individual change. This also requires that we not tamper with the audience selection to the degree that it would influence behaviors or participation. As a result, we will generally announce new features available in dev at the conclusion of the controlled feature rollout.
What if I really can’t wait?
If a feature has not yet been enabled by default for you, and you’d like to try it out, you can enable it on edge://flags and the new feature will be applied the next time you launch your browser.Controlled_Rollout_1.png

edge://flags

Thanks for being an Insider!
Exposing some of you to new features as the final stage of validation not only ensures the feature works better when you all get it; but safeguards later channels as well. Your input (both explicit and aggregated from browser telemetry) is helping us build a better browser.
Continue calling out what’s working, what’s not, what you like, and what could be better. In some cases, you might be ‘one of the few’ seeing a new experience. In those cases, your usage and feedback become even more critical. It helps us decide whether a feature should be enabled or not.

16 Replies

@Elliot Kirk wow! So many edge://flags

Cool idea. I like how much power you are putting the hands of all testers!

what is a good idea

PLEASE Instead of showing us "Default" value for the flags, tell us whether they are currently enabled or disabled. when you say Default we Don't know what it means, we don't know what the default is, is it enabled? is it disabled? and if we don't know then we can't know what new effects to expect after enabling or disabling them.

@HotCakeX Agreed. This is one of my biggest annoyances with Chrome. Even though there have been many complaints reported already in Chromium Bug Tracker, such as this one, yet it is disappointing to see it is not being taken care of.

@Elliot Kirk Is there any way to know which flags are unique to Edge and which are ones included due to them being part of Chromium at large?

@Elliot Kirk  A/B testing is very popular but I find it difficult as I never know if I have a new feature enabled or not or later get it enabled but am unaware I have it now.

I've tried the Collections flag shortcut, but there doesn't seem to be anything obvious to turn on the Collections feature. Suggestions? (BTW, Yes, I am using the latest Beta build from today)

@Elliot Kirk 

Hi,

in the section where you mentioned "What if I really can’t wait?"

you suggest that we can use flags to enable those features. but there are many features that are unavailable and they don't have any flags to enable them.

Features such as new privacy dashboard, share button, tab grouping, turning off news feed, turning off auto play etc.

@HotCakeX 

@Elliot Kirk 

Would be nicer if all features was on by default for everyone in Canary.

Sure it may break and it may be unstable - but isn't that what canary is suppised to be - bleeding edge ?

Hello @Elliot Kirk 

 

I see that there are many controlled features that are not activated even after activating the flag.

I see that the team will continue in this method where the flag activation will not make any difference

So I have to say that I don't like the controlled feature

The wording in this OP really needs to change as there are many roll-out features that are "NOT" selectable via flags. So NO if a person really can't wait, they are still out of luck as there are no flags to turn "on" to get to try the roll-out feature they want to try.

Dennis5mile

@Elliot Kirk 

 

I can understand controlled features in Stable version, and even Dev. But why made it for Canary?

I want to test Vertical Tabs and there is NO WAY to activate it if you are not in the test group. Activating flags that was mentioned in several how-tos (Shy UI) do nothing...

 

For the test purposes it will be cool to have flags for every testing features.

" if a person really can't wait" because there is an urgency?
This is not a playground, so everybody stays in line until it's their turn.
@HotCakeX True, if it is "Default" it actually means "Disabled." Both mean the same thing.
Yes, the flags need to be specific whether they are enabled or disabled