In this blog, we share the new capabilities that make it even easier to validate your applications against Windows client and server operating system updates in a Microsoft-managed test environment.
Since we announced general availability in November 2021, many organizations spanning different industry segments and regions have been using Test Base for Microsoft 365 to simplify validating their applications against the latest Windows updates. As an Azure service that facilitates intelligent data-driven testing of applications, Test Base reduces the hassle, time commitment, and expenditure of setting up and maintaining complex test environments for software vendors who want to validate their apps. It also enables IT professionals to test their line-of-business (LOB) applications through integration with Microsoft Intune.
Test Base for Microsoft 365 and Windows 11
Microsoft is committed to ensuring your applications work on Windows 11. Our compatibility promise states that apps that worked on Windows 7, 8, and 10 will work on Windows 11. That said, IT pros continue to use processes such as ring-based testing and flighting to assess the compatibility of the devices and apps they manage against upcoming builds of Windows and Office before deploying those updates to their broader base of end users. Test Base is a flagship offering that strengthens our compatibility promise by enabling the validation of business-critical apps against Windows updates.
What's new
Here's a list of the latest Test Base for Microsoft 365 features and capabilities based on the feedback we've received from early adopters and the community.
New package onboarding experience
Many Test Base users are not developers and found writing install and launch scripts for apps to be a challenge. To make the experience smooth and inclusive for everyone, we developed a simplified experience so that all organizations can easily create a package from scratch, configure and edit auto-generated scripts online, and get their applications onboarded with five easy steps. Learn more.
Test Base GitHub community
In talking to the Test Base community, we discovered that many have very little in the way of test automation for their organizations. To help you take advantage of Test Base quickly, we are now providing open-source test samples and tools on GitHub. This makes it even easier to build Test Base packages, integrate them into your CI/CD process, and query your test results by API. Learn more.
Pre-release access to Windows security updates
When we announced the general availability of Test Base last year, only a few select organizations had access to pre-release Windows update content. Since then, we've made improvements to the vetting process so that organizations can gain access to pre-release builds more quickly. Once your request is approved, your uploaded packages will automatically get scheduled to be tested against the pre-release Windows updates for the OS versions selected during onboarding. To request access, simply select Access level change request in the left navigation bar and fill out all the details for your organization. To learn more, read Request to change access level.
Enhanced package management experience
We heard feedback that you wanted an easier way to manage your packages. We revamped the Manage packages experience in Test Base to provide you with more informational messages that guide and help you manage your packages more efficiently. We've added messages to help you understand the lifecycle of your package for each verification status. The newly added Quick actions column provides recommended operations for your packages according to package status to help you take the proper actions. To learn more, watch this short video:
Enhanced test summary page
Many of you said that you would like a visual summary of your application tests to better inform you when and where to take action. We heard you loud and clear! The enhanced Test summary page provides two summary charts to provide visibility into the test run status and OS failures. You can see the package level test summary and export that data to a CSV file. On the single package detail page, you will find a security update/feature update tab if you have the specific test execution. You can also select the OS version to access the corresponding test results page. To learn more, watch this video:
Testing on demand
One of our most popular feature requests is the ability to run tests at your convenience, in addition to running them when new Windows builds are available. Test Base now provides the option to manually run a test. For each completed test run, you can select the package and all associated tests you'd like to run. You can also drill down for a specific package and select all tests grouped by the operating system.
Note: This feature is only available for packages that have passed validation and have a test run status of either Completed or Test failure.
Billing hub
To help your organization understand its usage and management costs, we now offer a Billing hub. The hub provides granular tracking of your trial balance and billable hours. By accessing the Usage console under the Billing hub section of the Test Base portal, you can review your usage, balance of free hours, and billable cost based on usage. You can also check usage by choosing a relative time range (last 24 hours/last 7 days/last 14 days) or by customizing the time range and specifying a start date and end date (up to three months billing history). Check out the video below to get more details:
Console log viewer
One popular request was to provide the ability to view a single log output without the need to download the entire results zip file.
When Test Base was first released, we only provided log output in a zip file for each process during a test execution. With the new console log viewer, you no longer have to download a large zip file to view the log output. You can simply select the script from the Test details page and a viewer will open, allowing you to view the log details and output directly in the Test Base web portal. You can easily view specific logs without navigating a large log folder and remediate issues faster.
Quicker validation runs
We heard your feedback that validation runs took too long, so we optimized the way these runs are scheduled to provide you with quicker feedback and results. Validation runs are now prioritized so they can be completed within a few hours after a request is received. This means you can get up and running more quickly and can begin setting up your test subscriptions sooner. At this time, we are able to complete the validation run in 2-3 hours (vs. 24 hours previously). Our goal in the next few months is to reduce this to less than an hour.
Out of box tests are now optional
As many of you are looking for more flexibility when selecting which tests to run, we have made the out of box tests optional. This helps those of you who just want to run functional tests optimize your testing accordingly.
New Windows 11 feature update available
A preview build of the latest feature update of Windows 11, version 22H2 (OS build: 22621.1), is now available on Test Base. To receive this new package, activate "Feature Update" with "Insider Beta Channel" selected. If you've already registered, your applications will be automatically validated against all upcoming feature updates. For more information about the Windows Insider Program, please see Flight Hub.
Join the Test Base community
We continue to rapidly innovate and invest in application validation. We look forward to sharing more with you in the coming months. In the meantime, visit our Test Base documentation to learn more. There, you'll find quick start guides to help you create your Test Base account, upload applications to test, and interpret the data.
We continue to engage developers and IT pros like you to add more value to the service and address additional use cases. Sign up for Test Base today or get in touch with us directly. We would also like to invite you to come join us at the Test Base for Microsoft 365 community on Tech Community to share your experiences and connect with others using the service.