Yammer Explained: How data and testing shapes Yammer

Have you ever been curious about how we build Yammer?  We want to take you behind the scenes of our product team to show what goes into building the features that you see every day in your Yammer communities.

The best way to ask tens of millions of Yammer users their opinion is to run tests at scale.  Using data to create deeper engagement and adoption is a cornerstone of the product development process at Yammer and at Microsoft in general. To do this, we try different product variations in order to understand the experiences of many users without sitting next to them. These variations could relate to how and where we surface a product feature, the impacts of a new launch, or optimization of the recommendations we show in the Home Feed, among others.

Not everything works... taking a hypothesis driven approach

When we build Yammer, our product team makes hypotheses about how users will react and respond to different changes and feature sets. However, we know based on past experiences that our hypothesis may not always be correct, and there may be unexpected impact from a change. We can’t assume we will always know how you will react and respond to these changes. To bridge the gap between our expectations and reality, we run experiments using variations of Yammer.

We collect anonymized data on user behavior from these experiments to inform decisions about where we take the product. We run most experiments for a few weeks in order to gather a statistically significant sample, but some may go longer in order to determine lasting vs. novelty effects. For our testing to be valid, we choose a random set of users from across millions of people . The answers we gain from experiments let us learn from you and help to inform decisions. These interactions and learnings help us evolve the features we deliver for Yammer.

For example, our data shows that conversations with images or videos attached get twice as many reactions/replies. So, we recently launched an experiment to add an image icon to the publisher Based on some qualitative insights we collected from feedback forms, we learned that users don’t have a clear entry point to add images. The paperclip icon we have indicates 'attach a file' to users, but users told us they didn’t know to click on it to add an image. Our hypothesis was that if we make the entry point for adding images/videos clearer, then more people will add images/video to their posts.

We tend to be cautious about big changes like this, but in this case, we saw positive results and our hypothesis was correct. This change resulted in an increase in attachment clicks, growth in the total number of employees creating conversations, and growth in total conversations with an image created. Interestingly, we saw more users choose the photo icon to add an image than the paper clip icon.

New designs are not always tested with the explicit purpose of increasing engagement.   Often times, we are brainstorming ways to enhance the experience for our customers, and want to ensure that a new design does so without introducing a new set of issues or complexity.   Below, you can see two such examples.

Enhanced Feed Explanations

In this variant of the explanations that appear at the top of content cards in the home feed, we wanted to be provide users with an annotation that let them know whether or not suggested content was from a community they followed.  In the case where it was not, like the example above, we added a button to enable them to join communities.  This change both increased transparency on why content was being suggested and helped us ensure our recommendations are working as intended.  The change did not result in significantly increased engagement (Home Feed Views decreased by ~1%), but we shipped it anyway because it was a better user experience and helped us to identify and fix errors more easily.

Communities App with no Left Navigation pane

Sometimes, our hypothesis is proved incorrect.  In this case, we experimented with removing the left-side navigation that shows Communities you are a member of from our Communities in Teams app, to better align with the Teams user interface styling.  The results indicated that users were confused and had a difficult time navigating to their communities--messages views, community visits, and active engagement in communities were all down by 10%+, with high statistical significance.  In this case, the cleaner interface and alignment with Teams was not enough to offset the significant drop in engagement, which told us users were having a hard time navigating to their communities.  We did not ship the treated version, and our Communities experience today looks very similar to what you see on Yammer.com.

Educate your employees

We are always exploring variations of Yammer to help you and your employees be more productive and efficient. Since changes to our product are inevitable, it might be useful to help your end users understand that their Yammer interface might not always look the same as their co-workers'. And depending on the changes we release, you may need to update your documentation.

Here is some sample copy that you can use to share with your Yammer 101 community to help your employees understand why Yammer instances may look slightly different over time.   

The Yammer team at Microsoft explores different variations of Yammer in order to determine what’s effective and productive for you. For example, you may see different placements or colors of buttons within the Yammer interface. Yammer explores variations of the product to put their own assumptions to the test and to see which features create the best experience. Your interactions and use of Yammer help to improve the future product for everyone.

Stay connected and know what’s coming next

Learn about the new features coming and when by following the M365 Roadmap. You can filter by Yammer and by product stage.  Additionally, as features come to production environment of Yammer, are sent to admins and the Yammer blog is updated with new features as they are announced and released. If you have technical issues, please contact Microsoft Support .

Learn more about our Microsoft 365 Change management and communication policies here.

-The Yammer Team