Test against Windows new features

Important

Test Base for Microsoft 365 will transition to end-of-life (EOL) on May 31, 2024. We're committed to working closely with each customer to provide support and guidance to make the transition as smooth as possible. If you have any questions, concerns, or need assistance, submit a support request.

If you’re seeking further validation of your eco-system solution(s) against Windows new features, or hesitating whether to deploy the latest Windows feature or the next version of security solution to your organization, Test Base provides you with a way to test your applications against preview builds in Windows Insider program automatically (More about Windows Insider).

This section shows you how to set up your own scheduled tests against preview features in Windows insider channel.

Prepare your package

1. Define content: Depending on your package type, you may select below instructions:

• Creating and Testing Binary Files on Test Base
• Test your Intune application on Test Base
• Uploading a pre-built zip package

2. Configure test: Both Out-of-Box test and Functional test are supported. Selecting Out-of-Box applies the officially suggested test flow and automatically generate install / uninstall / launch / close test scripts for you; Functional tests allow you more flexibility to set up your own test flow. You may also select both.

3. Edit package: Edit test scripts and test flow as you need.

   Note

   All your tests would be directly launched against the latest Windows OS with new features. No need to add any additional test script for system feature enablement. Thus, feature update testing would be more economical for you if you only want to validate potential compatibility issues against Windows new features since you won’t spend time in update/upgrade validation.

Sign-up for scheduled tests against preview Windows builds

Scheduled tests against preview Windows builds are set in the Test matrix step. By selecting Feature update, your package would be tested against preview builds flight to your indicated insider channel.

To set up for feature updates, you must specify the target product and its preview channel from “Insider Channel” dropdown list.

Your selection will register your application for automatic test runs against the latest feature updates of your selected product channel and all future new updates in the latest Windows Insider Preview Builds of your selection.

Note

• Check in Flight Hub for more details on the Windows Insider Preview builds.
• Windows 11, version 23H2 shares the same servicing branch as Windows 11, version 22H2. New features are firstly flight to insider audience via a simple update with enablement package(eKB) (See in Windows client roadmap update).

You may also set your current OS in “OS baseline for Insight”. We would provide you with more test insights by regression analysis of your as-is OS environment and the latest target OS.

• If the OS you’re looking for didn’t show up in our current supported product list, feel free to Let us know your request, or contact us via testbasepreview@microsoft.com.

Check Feature update test results

A test run will be executed after the package passes the validation. For all new feature updates in your selected insider channel, an automated run is scheduled.

You can view the results of the test run under the Test summary page by clicking the link on the package name.

You may use the release number / release version / KB number to map with the update version pushed to your organization. You will not only get the detailed script execution results with test logs but also compare regressively with previous month’s execution result to deep dive into any further performance risks. In case you might need to reproduce the failure and see in detail the execution process in video, you may click on Re-run test.