First published on MSDN on Jun 05, 2018
Authored by Philip Froese [MSFT]
[Last updated October 26, 2018]
In the next release of the HLK, the UCSI tests have been updated to run against the new Type-C SuperMUTT device instead of a partner system. This means the test setup will be simpler, the tests will run more quickly, and test content will be more thorough than in Windows 10 April 2018 Update HLK and earlier. The new preview tests are only in pre-release HLK builds 17676 and higher.
New Test Setup
The new test setup is simple: it requires a single Windows PC with Type-C, designated the System Under Test (SUT), and the Type-C SuperMUTT connected to one of the Type-C connectors on the system. See the image below.
Type-C SuperMUTT
The Type-C SuperMUTT is the newest edition to the Microsoft USB Test Tool (MUTT) family of devices. It incorporates the functionality of the legacy SuperMUTT device as well as Type-C and PD functionality.
The software and firmware utilities are available in the MUTT Package download: https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/mutt-software-package
And the tool is available for purchase through MCCI: http://www.mcci.com/mcci-v5/devtools/type-c-supermutt.html
Firmware version 45 or later, available in MUTT Package v2.6 and later, is required to run the UCSI HLK tests in the next release of the HLK. See the TypeCSuperMUTT.pdf documentation in the MUTT Package download for more information.
New Test Names
The UCSI compliance tests have been replaced with a new version of the tests to run against the Type-C SuperMUTT. The new version of the test has the tag “[Type-C MUTT]” added to the test name.
What is Different?
Beyond a new name and a simpler test setup, what else is new in the tests? For the most part, the tests cover the same features in the same manner as the original tests did. However, there are some cases where the new tests will be more rigorous.
Take USB Operation Role tests for example. Many desktop UCSI systems do not support being set into the UFP role, though they may resolve to that role initially when connected to a pure DFP (such as a charger). This behavior makes it difficult to test every permutation of DFP/UFP/DRP connections that a platform may have to support if it is tested against a functionally equivalent peer device. However, with the Type-C SuperMUTT the test can deterministically place the Type-C SuperMUTT into a specific, known port direction or configuration before connecting it to the SUT and is thus able to provide more complete test coverage of USB Operation Role features.
Thus, there may be some test cases in next release of the HLK that are more thorough and thus uncover new platform quirks that Windows 10 April 2018 Update HLK and earlier test content did not.
You will also notice that some tests have been removed. Some tests were determined to be redundant when run against the Type-C SuperMUTT and so were removed. Others targeted UCSI commands that have been removed in UCSI v1.1, and which Windows never supported, so were removed as obsolete.
USB Type-C UCSI Data and Power Role Swap Tests
Applicable Tests
- USB Type-C UCSI Data Role Swap
- USB Type-C UCSI Power Role Swap
Hardware Requirements
- One UCSI compliant Windows system.
- One Type-C MUTT device.
Test Setup
- Install HLK client on the System Under Test (SUT).
- Connect the Type-C MUTT to any USB Type-C port on the SUT.
- Locate the device node in Device Manager (devmgmt.msc) named "UCM-UCSI ACPI Device". The node is under the "USB Connector Managers" category.
- Right-click on the device, and select "Properties" and open the "Details" tab.
- Select "Device Instance Path" from the drop-down and note the property value.
- Open Registry Editor (regedit.exe).
- Navigate to the device instance path under this key: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Enum\<device-instance-path from step 6>\Device Parameters
- Create a DWORD value named "TestInterfaceEnabled" and set the value to 0x1.
- Restart the device by selecting the "Disable" option on the device node in Device Manager, and then selecting "Enable". Alternatively, you can simply restart the PC.
Test Parameters
Parameter NameParameter Description
| SwapsToPerform | Number of data role swaps to perform. The minimum is 2 so that both host and function mode are tested. |
| ValidateUsbFn | If ValidateUsbFn = true, the test will validate function stack behavior. |
Troubleshooting
- Some UCSI commands are marked "optional" in the UCSI specification. However, Microsoft requires some optional commands to be implemented in order for the UCSI driver to function. The tests may fail if you've forgotten to implement one of these commands in your BIOS. You can find that list of commands here .
-
- If the test is not detecting the Type-C MUTT:
- Check your cable connection to the Type-C MUTT.
- Check in Device Manager. Has the Type-C MUTT enumerated?
- Look for the "SuperMUTT" device with hardware ID USB\VID_045E&PID_078F in the Device Manager.
- Does the "Device Status" of the UCSI device report any errors? If so:
- Right-click on the UCSI device and disable it.Check in Device Manager. Does the "Device Status" of the UCSI device report any errors? If so:
- Start UCSI logging (see https://aka.ms/usbtrace )
- Enable the UCSI device. Do whatever it takes to get the yellow bang to reproduce.
- Stop logging.
- View the resulting WPP logs and check for errors.
- If the test is not detecting the Type-C MUTT:
USB Type-C UCM Data and Power Role Swap Tests
Applicable Tests
- USB Type-C UCM Data Role Swap
- USB Type-C UCM Power Role Swap
Hardware Requirements
- One Windows system with a USB Type-C connector.
- One Type-C MUTT device.
Test Setup
- Install the HLK client on the System Under Test (SUT).
- Connect the Type-C MUTT to any USB Type-C port on the SUT.
- Ensure all other USB Type-C ports are disconnected.
Test Parameters
Parameter NameParameter Description
| SwapsToPerform | Number of data role swaps to perform. The minimum is 2 so that both host and function mode are tested. |
| ValidateUsbFn | If ValidateUsbFn = true, the test will validate function stack behavior. |
Troubleshooting
- "No USB Type-C Connectors found with partners attached".
- Check your cable connection to the Type-C MUTT.
- Check in Device Manager. Has the Type-C MUTT enumerated?
- Look for the "SuperMUTT" device with hardware ID USB\VID_045E&PID_078F in the Device Manager.
- If your system is using UCSI, you can take UCSI logs during attach to investigate why the attach has not been reported to the OS.
UCSI Compliance tests
Applicable tests
This category of tests refers to all tests in the HLK with a name that begins with "UCSI" and is post-fixed with “[Type-C MUTT]”. The early pre-release HLK builds will include the original UCSI Compliance Tests as well, these tests have the same names, but without the “[Type-C MUTT]” suffix. Unless you are specifically running the old test for comparison, you should ignore the old versions of the tests.
UCSI Compliance Tests are meant to test the UCSI-capable Type-C system’s compliance to UCSI Specification.
Broad Categories of UCSI Compliance Tests
- UCSI Command Interface tests.
- Tests all of the UCSI commands that are claimed to be supported by the SUT.
- USB Operation Mode tests.
- Tests all of the USB Operation Modes that are claimed to be supported by the SUT on the given Connector.
- USB Operation Role tests.
- Tests all of the USB Operation Roles and role swaps that are claimed to be supported by the SUT on the given Connector.
- Power Direction Role Tests.
- Tests all of the Power Direction Roles that are claimed to be supported by the SUT on the given Connector. Performs role swaps.
- UCSI Notification tests.
- Tests all of the UCSI notifications that are claimed to be supported by the System under Test on the given Connector.
Hardware Requirements
- One UCSI compliant Windows system.
- One USB Type-C MUTT device.
How to identify Connector 1
In the next release of the HLK, the tests can be run against any available connector on the SUT, but you will need to specify the connector that the Type-C MUTT is connected to via the UcsiConnectorNumber parameter. If you do not know the connector numbers on a multi-port system, the UCSI Connector One Identification [Type-C MUTT] test can help you identify which is connector 1. If the SUT has 3 or more connectors, and you don’t know their mapping, you will simply need to experiment by connecting the Type-C MUTT to an unknown port, setting UcsiConnectorNumber to a new value, and running a test to see if the device is found on that connector; repeat with a new connector number until successful. This should just take a few attempts to establish the connector number layout for any new system.
Test Setup
- Install the HLK client on the System Under Test (SUT).
- Connect the Type-C MUTT to any USB Type-C port on the SUT.
- Record the connector number to which the Type-C MUTT is attached. You will supply this number when scheduling the tests.
- Install TestUcsi.sys driver on the SUT. On a URS system, it is important that the Type-C MUTT is connected before the test driver is installed, otherwise the USB host stack may not get loaded.
- In Device Manager, find the device "UCM-UCSI ACPI Device". This device will have the driver UcmUcsiAcpiClient.sys installed.
- Right-click the device and select "Update Driver Software"
- Enter the path to TestUcsi.inf
- TestUcsi.inf and TestUcsi.sys are located in \TestUcsi\<ARCHITECTURE>\
- Follow the prompts to complete the driver installation. This will replace UcmUcsiAcpiClient.sys with TestUcsi.sys.Right-click the device and select "Update Driver Software"
- Enter the path to TestUcsi.inf
Test Parameters
Parameter NameParameter Description
| UcsiConnectorNumber | The Type-C connector number on which the Type-C MUTT device is attached. |
| WaitTimeInMinutes | For tests that require manual intervention, this is the amount of time in minutes the test will wait to see the event that is expected to occur in response to the requested manual action. |
| WaitTimeMultiplier | An integral multiplier of wait times required in the test when interacting with the Type-C MUTT. Some systems may take somewhat longer than expected to enumerate the device when it reconnects. |
Test Execution
The Type-C MUTT allows the HLK tests to emulate and automate all scenarios in next release of the HLK that previously may have required manual intervention. You can select all of them at once, or only run a subset of them if you wish.
Test Cleanup
The UCSI Data and Power role swap tests require UcmUcsiAcpiClient.sys rather than TestUcsi.sys. If you plan to run the UCSI Data Role Swap or UCSI Power Role Swap tests in the HLK after this test, be sure to clean up your test environment after running the UCSI compliance tests. You can do this by replacing TestUcsi.sys with the original UcmUcsiAcpiClient.sys or by reinstalling Windows on both systems.
Troubleshooting
- Please review the HLK test log to determine why the test failed. In many cases, the test log will state common known causes of a given failure and how it may be resolved.
- Examine the driver logs per the guidance provided in the following blog post:
- If the test failure is unclear from both the test and driver logs or you believe it to be incorrect, and you are going to reach out to Microsoft for additional guidance, please prepare the following when reporting the issue to us. This will help us get to the bottom of your bug report as quickly as possible!
- The HLKX package containing the failed test result. This will contain the failed test log as well as driver and Type-C MUTT firmware logs we may need to review.
- An explanation of what diagnostic efforts you have already applied and why they were inconclusive:
- Was the test log inconclusive or confusing? (We'd love to hear feedback in order to make them better!)
- Do you believe the test result to be incorrect, and if so, why?
- If you believe a test failure to be in error, do you have passing logs from the same test case in Windows 10 April 2018 Update HLK? If so, please provide them. Or do you have other evidence (e.g. driver traces, PD trace, etc.) that would demonstrate the PPM behaving properly in the scenario under test?