Is there A/B Testing going on in Office 365?

[Ivan Unger]

So this happened today. One of my users tells me he's missing a lot of tiles on portal.office.com.

I, of course, haven't removed any licenses (E5), so I'm sure that this is not an issue.

I take look at his portal page and immediately notice, that his tiles are "collapsed". I check my portal and that of other users, and we don't even have the option.

My tenant setting is set to "full tenant first release". So am I right to assume, that apart from the tenant release setting, additional A/B testing is going on?

[#2]

I am not aware of any A/B Testing outside of Yammer. More like a partial rollout of new features or premature release of a feature (or just a glitch), that will go back to normal after a few days perhaps, especially in a FR tenant.  

What does that highlight text say by the way?  Something like click here to see more apps?  Are missing items like People, Tasks, and Delve shown afterwards?

Incidentally, we know the app launcher is getting a makeover and this also may affect the homepage with a new App Gallery option for example.

[#3]

I wasn't aware of any A/B testing either. But this marks the second time in the last year, that this specific user a different UI item on the Portal than the rest, even though we're set to full tenant FR.

Yes, the text does say "show more apps" and then the regular tiles are back.

About the "app launcher" note, I don't think it's that. Isn't the AppLauncher just the top left menu that is activated upon click?

[#4]

I'm sure we'll here all about it, in a blog post on a obscure team blog in about 2 weeks :)

[#5]

@Ivan Unger wrote:
I wasn't aware of any A/B testing either. But this marks the second time in the last year, that this specific user a different UI item on the Portal than the rest, even though we're set to full tenant FR.

That must be disconcerting, I'd chalk that up to oddities within First Release but I agree it should explainable.

About the "app launcher" note, I don't think it's that. Isn't the AppLauncher just the top left menu that is activated upon click?

---

True, though this new App Gallery will be accessible via the app launcher and Office 365 Homepage.

[#6]

I dont think it's A/B testing, more of a gradual rollout to all servers/datacenters. But they definitely do some targetting also - I've seen many cases where some new feature first light up in NA tenants, then some other region, etc. Even though all my test tenants are on "full" FR.