Setting up an environment to run USB Type-C system HLK tests

[Shyamal Varma]

First published on MSDN on Jul 06, 2016
Authored by Michelle Bergeron [MSFT] and Makarand Sonare [MSFT]

[Last updated 10/3/2017]

There are new tests in the Hardware Lab Kit (HLK) that target systems with USB Type-C. These tests are available for the Windows Anniversary Update version of HLK and Windows 10. Some of the tests require extra configuration – here is a guide to help you set up your test environment for them.

USB Type-C UCSI Data and Power Role Swap tests

Applicable tests

• USB Type-C UCSI Data Role Swap

• USB Type-C UCSI Power Role Swap

Hardware Requirements

1. Two UCSI 1.0 compliant Windows desktop systems.

1. UcmUcsi.sys must be loaded as the UCSI controller driver.

1. USB Type-C cable.

Note: If your Type-C system does not use UCSI, the test will detect this and allow your system to pass.
Test Setup

1. Designate one system as the System Under Test (SUT) and the other as the "Partner" system.

1. Install HLK client on the SUT.

1. Connect the UCSI Type-C connector on the SUT to the UCSI Type-C connector on the Partner via the Type-C cable.

1. Locate the device node in Device Manager (devmgmt.msc) named "UCSI USB Connector Manager". The node is under the "Universal Serial Bus controllers" category.

1. Right-click on the device, and select "Properties" and open the "Details" tab.

1. Select "Device Instance Path" from the drop-down and note the property value.

1. Open Registry Editor (regedit.exe).

1. Navigate to the device instance path under this key: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Enum\<device-instance-path from step 6>\Device Parameters

1. Create a DWORD value named "TestInterfaceEnabled" and set the value to 0x1.

1. Restart the device by selecting the "Disable" option on the device node in Device Manager, and then selecting "Enable". Alternatively, you can simply restart the PC.

Tracing Instructions
Please note that for RS3 UCSI HLK tests, tracing is automatically captured as noted in this blog post . For Windows 10 Anniversary Update and Creators Update, tracing still needs to be started manually using the instructions below.

Test failures for the UCSI data and power role swap tests can be investigated by collecting logs as per the below instructions.

1. Start WPP tracing before executing tests. Run the following from an elevated command prompt:
   logman start -ets ucmucsi -ct perf -p {EAD1EE75-4BFE-4E28-8AFA-E94B0A1BAF37} 0xffffffff 0xff -o c:\ucmucsi\ucmucsilogs.etl
   

1. Stop WPP tracing after running the tests. Add the following commands to a .cmd file and run from an elevated command prompt once the test completes.
   logman stop ucmucsi -ets
   

1. You will be able to view the resulting logs in UcmUcsiLogs.etl.

Test Parameters
Parameter NameParameter Description

SwapsToPerform	Number of data role swaps to perform. The minimum is 2 so that both host and function mode are tested.
ValidateUsbFn	If ValidateUsbFn = true, the test will validate function stack behavior.

Troubleshooting

• Some UCSI commands are marked "optional" in the UCSI specification. However, Microsoft requires some optional commands to be implemented in order for the UCSI driver to function. The tests may fail if you've forgotten to implement one of these commands in your BIOS. You can find that list of commands here .

• If the test is not detecting an attached partner
  
  
  • Check your cable connection to the attached partner
  
  
  • Check in Device Manager. Has the attached partner actually enumerated? If not...
    
    
    • Detach the partner
    
    
    • Start UCSI logging (instructions above)
    
    
    • Attach the partner.
    
    
    • Stop logging.
    
    
    • View the resulting WPP logs and check for errors.
    
    
    
  
  
  • Check in Device Manager. Is the UCSI device a yellow bang? If it is...
    
    
    • Right click on the UCSI device and disable it.
    
    
    • Start UCSI logging (instructions above)
    
    
    • Enable the UCSI device. Do whatever it takes to get the yellow bang to reproduce.
    
    
    • Stop logging.
    
    
    • View the resulting WPP logs and check for errors.
    
    
    
  
  
  

USB Type-C UCM Data and Power Role Swap tests

Applicable tests

• USB Type-C UCM Data Role Swap

• USB Type-C UCM Power Role Swap

Hardware Requirements

1. Two Windows systems, each with a USB Type-C connector.

1. The Type-C connector on the SUT must have a UcmCx client driver. For information about developing a UcmCx client driver, visit USB Type-C connector driver programming reference .

1. USB Type-C cable.

Note: If your Type-C system implements UCSI, this test is still applicable because UcmUcsi.sys is a UcmCx client driver.
Test Setup

1. Designate one system as the System Under Test (SUT) and the other as the "Partner" system.

1. Install HLK client on the SUT.

1. Connect the Type-C connector on the SUT to the Type-C connector on the Partner via the Type-C cable.

Test Parameters
Parameter NameParameter Description

SwapsToPerform	Number of data role swaps to perform. The minimum is 2 so that both host and function mode are tested.
ValidateUsbFn	If ValidateUsbFn = true, the test will validate function stack behavior.

Troubleshooting

• "No USB Type-C Connectors found with partners attached".
  
  
  • Check your cable connection to the partner.
  
  
  • Check in Device Manager. Has the partner device enumerated?
    
    
    • If your system is using UCSI, you can take UCSI logs during attach to investigate why the attach has not been reported to the OS. UCSI logging instructions are found above.
    
    
    
  
  
  

UCSI Compliance tests

Applicable tests
This category of tests refers to all tests in the HLK with a name that begins with "UCSI". UCSI Compliance Tests are meant to test the UCSI-capable Type C system’s compliance to UCSI Specification V1.0 . These tests are marked as "manual" in the HLK. The test binaries are included in the HLK; however, you will need to run them yourself using the instructions below.
Broad Categories of UCSI Compliance Tests

• UCSI Command Interface tests.
  
  
  • Tests all of the UCSI commands that are claimed to be supported by the SUT.
  
  
  

• USB Operation Mode tests.
  
  
  • Tests all of the USB Operation Modes that are claimed to be supported by the SUT on the given Connector.
  
  
  

• USB Operation Role tests.
  
  
  • Tests all of the USB Operation Roles and role swaps that are claimed to be supported by the SUT on the given Connector.
  
  
  

• Power Direction Mode tests.
  
  
  • Tests all of the Power Direction Modes that are claimed to be supported by the SUT on the given Connector.
  
  
  

• Power Direction Role tests.
  
  
  • Tests all of the Power Direction Roles that are claimed to be supported by the SUT on the given Connector. Performs role swaps.
  
  
  

• UCSI Notification tests.
  
  
  • Tests all of the UCSI notifications that are claimed to be supported by the System Under Test on the given Connector.
  
  
  

Hardware Requirements

1. Two UCSI 1.0 compliant Windows desktop systems.

1. Connection Exerciser hardware (optional - unless you are running the test from HLK Studio. If you are running the test in HLK Studio, the Connection Exerciser is required.)

1. USB Type-C PD capable cable.

1. USB 2.0 and USB 3.0 device

1. Debug accessory (if supported)

1. Analog audio accessory (if supported)

How to Identify Connector 1
All of the UCSI compliance tests assume Partner system or Devices are connected on Connector 1. To identify Connector 1, perform the following steps:

1. On the SUT, open an elevated command prompt.

1. From the elevated command prompt on the SUT, run the following test to identify Connector 1:
   Te.exe UcsiComplianceTest.dll /name:TestUcsi::UcsiCommandInterfaceTests::TestIdentifyConnectorOne /p:WaitTimeInMinutes=5
   You must specify the WaitTimeInMinutes parameter in order for the test to pass.

1. Repeat Step 2 on the Partner system to identify Connector 1.

Test Setup

1. Designate one system as the System Under Test (SUT) and the other as the "Partner" system.

1. Install HLK client on both systems. You will only schedule tests on the SUT.

1. Connect the Type-B port on the Connection Exerciser Arduino to the SUT on any port other than Connector 1

1. Connect Connector 1 of the SUT to J1 connector on the Connection Exerciser directly.

1. Connect Connector 1 of the Partner to J2 or J3 connector on Connection Exerciser via Type-C cable.

1. Install TAEF on both systems
   
   
   1. For information about installing TAEF, refer to the TAEF Getting Started page.
   
   
   1. Set up Te.Service on the Partner system. Instructions to do that are here .
   
   
   

1. Install the TestUcsi.sys driver on both systems.
   
   
   1. In Device Manager, find the device "UCSI USB Connector Manager". This device will have the driver UcmUcsi.sys installed.
   
   
   1. Right click the device and select "Update Driver Software"
      
      
      1. Enter the path to TestUcsi.inf
         
         
         1. TestUcsi.inf and TestUcsi.sys are located in \TestUcsi\<ARCHITECTURE>\
         
         
         
      
      
      1. Follow the prompts to complete the driver installation. This will replace UcmUcsi.sys with TestUcsi.sys.
      
      
      
   
   
   

Test Parameters
Parameter NameParameter Description

IsRunningOnSystemUnderTest	Identifies whether the test is running on the System Under Test or Partner machine. Values: true - If test is executed on SUT. false - If test is executed on Partner.
PartnerMachineName	Machine name of the partner machine that is connected to the SUT using a USB C-to-C cable. The test performs TAEF network remote execution to execute code on this partner machine.
WaitTimeInMinutes	For tests that require manual intervention, this is the amount of time in minutes the test will wait to see the event that is expected to occur in response to the requested manual action.

Test Execution with Connection Exerciser

1. Connect the SUT and Partner system via a Connection Exerciser and Type-C Cable on Connector 1.

1. Refer to the Test Setup section for more information.

1. On the SUT, run UcsiCommandInterfaceTests.
   te.exe UcsiComplianceTest.dll /name:TestUcsi::UcsiCommandInterfaceTests::*

1. On the SUT, run UcsiTests.
   te.exe UcsiComplianceTest.dll /name:TestUcsi::UcsiTest::* /p: IsRunningOnSystemUnderTest=true /p:waittimeinminutes=1 /p:partnerMachine=<MACHINE_NAME>

Test Execution without Connection Exerciser

1. Connect the SUT and Partner system via a Type-C Cable on Connector 1.

1. On the SUT, run UcsiCommandInterfaceTests.
   te.exe UcsiComplianceTest.dll /name:TestUcsi::UcsiCommandInterfaceTests::*

1. On the SUT, run UcsiTestsManual.
   te.exe UcsiComplianceTest.dll /name:TestUcsi::UcsiTestsManual::* /p:IsSUT=true /p:waittimeinminutes=1 /p:partnerMachine=<MACHINE_NAME>

1. Each test requires executing some TAEF tests on the Partner system as per the instructions given on the command prompt.

Example: When running UcsiTestsManual::TestDFPModeToUFP on the System Under Test:
CMD> te.exe UcsiComplianceTest.dll /name:TestUcsi::UcsiTestsManual::TestDFPModeToUFP /p:IsSUT=true /p:waittimeinminutes=1 /p:partnermachine=<MACHINE_NAME>
Test Authoring and Execution Framework v5.6 for x64
Resetting the PPM
Enabling All Notifications

StartGroup: TestUcsi::UcsiTestsManual::TestDFPModeToUFP
Setting USB Operation Mode to Dfp on Connector 1
1. On Partner Machine
Run: CMD Prompt>Te.exe UcsicomplianceTest.dll /p:IsSUT=false /p:waitTimeInMinutes=<timeInMinutes> /name:TestUcsi::UcsiTestsManual::TestUFPModeOnPartner
2. Connect the Partner
3. Waiting for 1 minutes

Waiting for:                   ConnectChange

The above test sets Connector 1's USB Operation Mode to DFP on the SUT. It then requires you to run the following command on the Partner system:
Te.exe UcsicomplianceTest.dll /p:IsSUT=false /p:waitTimeInMinutes=<timeInMinutes> /name:TestUcsi::UcsiTestsManual::TestUFPModeOnPartner
Tracing Instructions
Test failures for the UCSI compliance tests can be investigated by collecting logs as per the below instructions.

1. Start WPP tracing before executing tests. Add the following commands to a .cmd file and run from an elevated command prompt:
   logman start -ets testucsi -ct perf -p {C9ED7F4E-286B-4054-B74F-7DE43EFBCA2A} 0xffffffff 0xff -o c:\testucsi\testucsi.etl
   logman start -ets testucsicontroller -ct perf -p {EBFD4AED-ED11-4F0C-8F37-9C93084D1530} 0xffffffff 0xff -o c:\testucsi\TestUcsiController.etl
   logman start -ets ucsitest -ct perf -p {F8BADD54-3B0F-4B79-8389-D0ABB7AE242B} 0xffffffff 0xff -o c:\testucsi\ucsitest.etl

1. Stop WPP tracing after running the tests. Add the following commands to a .cmd file and run from an elevated command prompt once the test completes.
   logman stop testucsi -ets
   logman stop ucsitest -ets
   logman stop testucsicontroller -ets

1. You will be able to view the resulting logs in UcsiTest.etl, TestUcsi.etl, and TestUcsiController.etl.

Test Cleanup
The UCSI Data and Power role swap tests require UcmUcsi.sys rather than TestUcsi.sys. If you plan to run the UCSI Data Role Swap or UCSI Power Role Swap tests in the HLK after this test, be sure to clean up your test environment after running the UCSI compliance tests. You can do this by replacing TestUcsi.sys with the original UcmUcsi.sys or by reinstalling Windows on both systems.
Troubleshooting

• "Unable to connect to TE.Service"
  
  
  • Verify that you've used the above steps to configure TAEF remote execution
  
  
  • Verify that the machines can ping each other
  
  
  • Instead of using the machine name, try using its IP address. You can get an IP address of a machine using the following command in Command Prompt:
    ping -4 <machine name>
  
  
  

• No USB error notifications while using the UCSI test driver
  
  
  • This is by design - the UCSI test driver exists only for the purposes of testing your PPM and does not interact with the OS in the same way as the inbox UcmUcsi.sys. To verify user notifications, please reinstall UcmUcsi.sys.
  
  
  

• One of my role swap tests failed because "UcsiConnectorPartnerTypePoweredCableWithUfp".
  
  
  • We're on it! There will be an errata for RS2 (we don't have a date for the errata release yet - sorry!) and we've already fixed it for RS3. In  the meantime, you may try to work around it by connecting a different partner and/or using a different USB cable.
  
  
  

Selecting the tests in HLK

The following steps illustrate where to find the tests in the HLK.

1. Add the SUT as your test target
   

1. Right click on the machine, click "Add/Modify features". Verify that System.Fundamentals.SystemUSB.USBC is selected (and System.Fundamentals.SystemUSB.USBTypeCUCSI if your system is UCSI).
   

1. Select the "Tests" tab. Scroll down and see that the tests are available. In the example, the SUT uses UCSI.