The Device IOCTL/WMI Record and Playback platform (DRPP) records the Device I/O Control Codes (IOCTLs, kernel model IRP_MJ_DEVICE_CONTROL) and WMI I/O Request Packets (IRPs, kernel mode IRP_MJ_SYSTEM_CONTROL) that are successfully processed by the devnode for one or more Device(s) Under Test (DUTs). The purpose of this security extension to the WDTF platform is to discover which Device I/O Control Codes and WMI IRPs cause the driver and/or device to perform a specific action as opposed to simply return an error. These IOCTLs, IRPs are recorded to a file which can be written to when executing any WDTF based test such as Devfund". A new test can be written to replay this IO back to the Device under test thus saving time and machine cycles (since only ‘meaningful’ IOCTLs and WMI IRPs will be executed rather than all IOCTLs and IRPs). This platform is shipped as part of WDK..
The DRP platform consists of the following components:
- An Upper Filter Device Object (FiDO) installed on the DUT that records Device I/O Control Codes and WMI IRPs to a data file with target binding information.
- An IoSpy Action to enable and disable Filter Device Object(s) installed on the DUT.
- A Fuzz test is used to generate the IOCTLs and WMI IRPs for DUT(s).
- An IoSpy Action to disable the Filter Device Object installed on one or more the DUTs.
- A Replay Action to playback any successfully processed IRPs or IOCTLs.
The features of the platform are executed in order as a system which consists of the following phases:
- Using WDK to prepare a system for driver development and testing which will consist of a developer box and a target machine and the target machine will host the hardware/device under test.
- Upper Filter Device Object (IoSpy) Enable Phase: enables/installs the IoSpy driver on the DUT.
- Recording (training) Phase: records various details about the DUT, Device I/O Control Codes, and WMI IRPs to a file.
- Playback Phase: uses the data file to intelligently test DUT(s).
These phases are described in more detail through the rest of this document.
Provision the target from Visual Studio (sets debugger + surfaces KD key/port) ( https://learn.microsoft.com/en-us/windows-hardware/drivers/gettingstarted/provision-a-target-computer)
- Install WDK using https://learn.microsoft.com/en-us/windows-hardware/drivers/download-the-wdk
- On the host, open Visual Studio → Extensions → Driver → Test → Configure Devices.
- Select Add new device → set Network host name to the target’s computer name or local IP.
- Check Provision device and choose debugger settings → Next.
- Choose KDNET (Network) and required settings will be prefilled, make sure the host IP is set correct. For KDNET, Visual Studio will configure the network transport and generate the KD key and port values used by WinDbg. Provisioning may reboot the target multiple times. When finished, select Finish, then restart the host if prompted.
After provisioning, the Configure Devices menu displays connection details for kernel debugging—KD key and debug port. Use these values to attach WinDbg (x64) from the host:
Kernel Debugger (recommended)
- Launch WinDbg (x64) as Administrator on the host.
- File → Kernel Debug → NET.
- Enter the Port, and Key exactly as shown on the Target Machine Ready screen of driver extension.
Recommended test/debug switches for DRPP runs
- Test signing: Keep testsigning on for unsigned prototypes during IoSpy, Fuzz, and Replay iterations.
- Enable Driver Verifier for all the drivers that are part of the stack e.g. Verifier /standard /driver usbvideos.sys, ksthunk.sys etc.
Upper Filter Device Object (IoSpy) the Enable Phase
IoSpy is a filter driver that creates an upper Filter Device Object (FiDO) that records data about IOCTLs and WMI requests made to the kernel-mode device or devnode/DUT.
The IoSpy Action is used to install, uninstall the IoSpy driver’s upper FiDO on one or more DUTs. The Device Query (DQ) parameter controls which DUT(s) the IoSpy filter driver’s FiDO is installed on. The diagram below shows how the IoSpy driver is enabled on a DUT and what the device stack looks like after the FiDO has been installed:
Below is sample code of how to install the IoSpy FiDO on a ‘media’ class DUT on a given system using powershell 7.5.0:
|
# Create the WDTF COM object $wdtf = New-Object -ComObject "WDTF2.WDTF" # Query for all started devices in the desired class # Use 'Camera' for cameras, or 'Media' if you want to keep the scope wider. See here for more # options for SDEL query https://learn.microsoft.com/en-us/windows-hardware/drivers/wdtf/simple-data-evaluation-language-overview $devices = $wdtf.DeviceDepot.Query("IsDevice AND Class='Media' AND IsStarted=True") # Loop through each device and enable IoSpy foreach ($device in $devices) { $ioSpy = $device.GetInterface("IoSpy") $ioSpy.Enable() } |
Below logging information is from this phase. This information is logged by an example test which uses the DRP platform, specifically the IoSpy driver:
|
WDTF_TARGETS : INFO : Target: NVIDIA High Definition Audio HDAUDIO\FUNC_01&VEN_10DE&DEV_0051&SUBSYS_10DE1087&REV_1001\5&2A94585&0&0001 WDTF_TARGETS : INFO : Target: Microsoft® LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&64B94B2&0&0002 WDTF_TARGETS : INFO : Target: Realtek High Definition Audio HDAUDIO\FUNC_01&VEN_10EC&DEV_0899&SUBSYS_102805B7&REV_1000\4&220B1BB C&0&0001 WDTF_TARGET : INFO : - GetInterface("IoSpy") WDTF_TARGET : INFO : Target: NVIDIA High Definition Audio HDAUDIO\FUNC_01&VEN_10DE&DEV_0051&SUBSYS_10DE1087&REV_1001\5&2A94585&0&0001 WDTF_IOSPY : INFO : - Enable() WDTF_IOSPY : INFO : Target: NVIDIA High Definition Audio HDAUDIO\FUNC_01&VEN_10DE&DEV_0051&SUBSYS_10DE1087&REV_1001\5&2A94585&0&0001 WDTF_IOSPY : INFO : Data file located at : \??\C:\iospy\IoSpyData_00000037 WDTF_PNP : INFO : - DisableDevice() WDTF_PNP : INFO : Target: NVIDIA High Definition Audio HDAUDIO\FUNC_01&VEN_10DE&DEV_0051&SUBSYS_10DE1087&REV_1001\5&2A94585&0&0001 WDTF_PNP : INFO : - EnableDevice() : ( get status count: 1 ) WDTF_PNP : INFO : Target: NVIDIA High Definition Audio HDAUDIO\FUNC_01&VEN_10DE&DEV_0051&SUBSYS_10DE1087&REV_1001\5&2A94585&0&0001 WDTF_TARGET : INFO : - GetInterface("IoSpy") WDTF_TARGET : INFO : Target: Microsoft® LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&64B94B2&0&0002 WDTF_IOSPY : INFO : - Enable() WDTF_IOSPY : INFO : Target: Microsoft® LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&64B94B2&0&0002 WDTF_IOSPY : INFO : Data file located at : \??\C:\iospy\IoSpyData_00000041 WDTF_PNP : INFO : - DisableDevice() WDTF_PNP : INFO : Target: Microsoft® LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&64B94B2&0&0002
|
Recording (training) Phase
The training phase is longer than the playback phase as we use the Fuzz Action to send many Device I/O Control Codes (IOCTLs) and WMI IRPs to the DUT. In this phase, we specifically identify the IOCTLs and IRPS that the DUT succeeded in and write this information to a data file specific to the DUT. This is how the DRP extension to the WDTF platform determines which IOCTLs and IRPs are ‘meaningful’ to the device- that is, the specific IOCTL or WMI IRP causes the device or driver to perform some action besides simply return an error.
The diagram below shows how the IoSpy driver records IRP and IOCTL processing for a DUT:
The logging information below is from this phase. This information is logged by a test which uses the DRP, specifically the IoSpy driver:
|
WDTF_TARGETS : INFO : - Query("IsDevice AND Class='Media' AND IsStarted=True") WDTF_TARGETS : INFO : Target: NVIDIA High Definition Audio HDAUDIO\FUNC_01&VEN_10DE&DEV_0051&SUBSYS_10DE1087&REV_1001\5&2A94585&0&0001 WDTF_TARGETS : INFO : Target: Microsoft® LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&64B94B2&0&0002 WDTF_TARGETS : INFO : Target: Realtek High Definition Audio HDAUDIO\FUNC_01&VEN_10EC&DEV_0899&SUBSYS_102805B7&REV_1000\4&220B1BBC&0&0001 WDTF_TEST : INFO : Device NVIDIA High Definition Audio HDAUDIO\FUNC_01&VEN_10DE&DEV_0051&SUBSYS_10DE1087&REV_1001\5&2A94585&0&0001 WDTF_TEST : INFO : IoSpy Info WDTF_TEST : INFO : Enabled : True WDTF_TEST : INFO : DataFileName : \??\C:\iospy\IoSpyData_00000037 WDTF_TEST : INFO : fsDataFilePresent : True WDTF_TEST : INFO : Device Microsoft® LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&64B94B2&0&0002 WDTF_TEST : INFO : IoSpy Info WDTF_TEST : INFO : Enabled : True WDTF_TEST : INFO : DataFileName : \??\C:\iospy\IoSpyData_00000041 WDTF_TEST : INFO : fsDataFilePresent : True WDTF_TEST : INFO : Device Realtek High Definition Audio HDAUDIO\FUNC_01&VEN_10EC&DEV_0899&SUBSYS_102805B7&REV_1000\4&220B1BBC&0&0001 WDTF_TEST : INFO : IoSpy Info WDTF_TEST : INFO : Enabled : True WDTF_TEST : INFO : DataFileName : \??\C:\iospy\IoSpyData_0000003a WDTF_TEST : INFO : fsDataFilePresent : True WDTF_TARGETS : INFO : - Query("IsDevice AND Class='Media' AND IsStarted=True") WDTF_TARGETS : INFO : Target: NVIDIA High Definition Audio HDAUDIO\FUNC_01&VEN_10DE&DEV_0051&SUBSYS_10DE1087&REV_1001\5&2A94585&0&0001 WDTF_TARGETS : INFO : Target: Microsoft® LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&64B94B2&0&0002 WDTF_TARGET : INFO : - HasInterface("FuzzTest") WDTF_TARGET : INFO : Target: Microsoft® LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&64B94B2&0&0002 WDTF_TARGET : INFO : - GetInterface("FuzzTest") WDTF_TARGET : INFO : Target: Microsoft® LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&64B94B2&0&0002 WDTF_TEST : INFO : Number of tests found:5 WDTF_TEST : INFO : Run tests WDTF_FUZZTESTS : INFO : - RunAll() WDTF_FUZZTESTS : INFO : Target: Microsoft® LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&64B94B2&0&0002 WDTF_FUZZTESTS : INFO : - RunTestUntilStopped(SubOpens Test) WDTF_FUZZTESTS : INFO : Target: Microsoft® LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&64B94B2&0&0002 WDTF_FUZZTEST : INFO : - Start() WDTF_FUZZTEST : INFO : Target: Microsoft® LifeCam VX-7000 WDTF_FUZZTEST : INFO : - Starting fuzz test: SubOpens Test WDTF_FUZZTEST : INFO : Target : Microsoft® LifeCam VX-7000 WDTF_FUZZTEST : INFO : Found 2 interfaces. WDTF_FUZZTEST : INFO : Attempting to open interface: \DosDevices\usb#vid_045e&pid_0723&mi_02#6&64b94b2&0&0002#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global WDTF_FUZZTEST : INFO : Opened \DosDevices\usb#vid_045e&pid_0723&mi_02#6&64b94b2&0&0002#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global. Access Mode=NO SYNC. Open Type=OTHER. Opened with access 1201bf [0] WDTF_FUZZTEST : INFO : Opened \DosDevices\usb#vid_045e&pid_0723&mi_02#6&64b94b2&0&0002#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global. Access Mode=DIRECT. Open Type=OTHER. Opened with access 120080 [0] WDTF_FUZZTEST : INFO : Opened \DosDevices\usb#vid_045e&pid_0723&mi_02#6&64b94b2&0&0002#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global. Access Mode=NO SYNC. Open Type=TREE_CONNECT. Opened with access 1201bf [0] WDTF_FUZZTEST : INFO : Opened \DosDevices\usb#vid_045e&pid_0723&mi_02#6&64b94b2&0&0002#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global\ . Access Mode=NO SYNC. Open Type=OTHER. Opened with access 1201bf [0] WDTF_FUZZTEST : INFO : Opened \DosDevices\usb#vid_045e&pid_0723&mi_02#6&64b94b2&0&0002#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global\ . Access Mode=DIRECT. Open Type=OTHER. Opened with access 120080 [0] WDTF_FUZZTEST : INFO : Opened \DosDevices\usb#vid_045e&pid_0723&mi_02#6&64b94b2&0&0002#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global\ . Access Mode=NO SYNC. Open Type=TREE_CONNECT. Opened with access 1201bf [0] WDTF_FUZZTEST : INFO : Opened \DosDevices\usb#vid_045e&pid_0723&mi_02#6&64b94b2&0&0002#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global\\. Access Mode=NO SYNC. Open Type=OTHER. Opened with access 1201bf [0] WDTF_FUZZTEST : INFO : Opened \DosDevices\usb#vid_045e&pid_0723&mi_02#6&64b94b2&0&0002#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global\\. Access Mode=DIRECT. Open Type=OTHER. Opened with access 120080 [0]
|
Run fuzzing from Visual Studio using the Driver Extension Menu
As the target machine is already provisioned using Visual Studio. Click on the Driver menu under Extensions and select Driver → Test → New Test Group from the context menu. Click add remove tests Choose Fuzz Tests from the list as they make sure to access all the code paths from the available test suites e.g. DF Fuzz Random IOCTL and Fuzz Zero Length buffer test.
- Configure and Run the Test
- Set any required test parameters, such as the device query string (e.g., IsDevice AND Class='Media' AND IsStarted=True for media devices).
- Click Run to start the Devfund test. Visual Studio will deploy the test binaries, execute the test on the target, and collect results.
- Review Results
- Test results, including logs and status, will appear in the Visual Studio Test Explorer or Output window.
When the devices were configured to be spied upon, IoSpy Data file is generated for all the devices which are under test. Following binding file and IoSpy data file information will be provided when IoSpy is enabled. These files are not expected to be modified and they are not in human readable format.
Binding File Information Produced
|
DisplayName |
DeviceID |
ClassGuid |
DriverBinaryNamesAndVersion |
IoSpy Data File |
|
NVIDIA High Definition Audio |
HDAUDIO\FUNC_01&VEN_10DE&DEV_0051&SUBSYS_10DE1087&REV_1001\5&2A94585&0&0001 |
{4d36e96c-e325-11ce-bfc1-08002be10318} |
ksthunk.sys 10.0.18218.1000 nvhda64v.sys 1.3.35.1 |
IoSpyData_00000043 |
|
Microsoft® LifeCam VX-7000 |
USB\VID_045E&PID_0723&MI_02\6&24DE6407&0&0002 |
{4d36e96c-e325-11ce-bfc1-08002be10318} |
ksthunk.sys 10.0.18218.1000 usbaudio.sys 10.0.18218.1000 |
IoSpyData_00000044 |
|
Realtek High Definition Audio |
HDAUDIO\FUNC_01&VEN_10EC&DEV_0899&SUBSYS_102805B7&REV_1000\4&220B1BBC&0&0001 |
{4d36e96c-e325-11ce-bfc1-08002be10318} |
ksthunk.sys 10.0.18218.1000 rtkvhd64.sys 6.0.1.7544 |
IoSpyData_00000042 |
|
Microsoft® LifeCam VX-7000 |
USB\VID_045E&PID_0723&MI_00\6&24DE6407&0&0000 |
{ca3e7ab9-b4c3-4ae6-8251-579ef933890f} |
wdmcompanionfilter.sys 10.0.18218.1000 ksthunk.sys 10.0.18218.1000 usbvideo.sys 10.0.18218.1000 |
IoSpyData_00000041 |
Playback Phase
Now that we have a data file and binding information, using the DRP extension to the WDTF platform we can playback these I/O (IOCTLS and IRPs) to any number of instance(s) of the same Device on any number of systems..
The diagram below shows how playback works for many DUTs on a single system:
Console Output
The user will see the following output on the screen when running a test that uses the Replay Action on a ‘media’ class device, for example:
|
WDTF_TARGETS : INFO : - Query("IsDevice AND Class='Media' AND IsStarted=True") WDTF_TARGETS : INFO : Target: NVIDIA High Definition Audio HDAUDIO\FUNC_01&VEN_10DE&DEV_0051&SUBSYS_10DE1087&REV_1001\5&2A94585&0&0001 WDTF_TARGETS : INFO : Target: Microsoft« LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&24DE6407&0&0002 WDTF_TARGET : INFO : - GetInterface("IoReplay") WDTF_TARGET : INFO : Target: Microsoft« LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&24DE6407&0&0002 WDTF_TEST : INFO : DataFile: \??\C:\iospy\IoSpyData_0000003d WDTF_IoReplay : INFO : - IoReplayIOCTL() WDTF_IoReplay : INFO : Target: Microsoft« LifeCam VX-7000 USB\VID_045E&PID_0723&MI_02\6&24DE6407&0&0002 WDTF_IoReplay : INFO : Data file located at : \??\C:\iospy\IoSpyData_0000003d WDTF_IoReplay : INFO : Result: Number of IOCTL entries: 2219286. Number of IOCTL to replay 69862 WDTF_TARGET : INFO : - GetInterface("IoReplay") WDTF_TARGET : INFO : Target: Realtek High Definition Audio HDAUDIO\FUNC_01&VEN_10EC&DEV_0899&SUBSYS_102805B7&REV_1000\4&220B1BBC&0&0001 WDTF_TEST : INFO : DataFile: \??\C:\iospy\IoSpyData_00000037 WDTF_IoReplay : INFO : - IoReplayIOCTL() WDTF_IoReplay : INFO : Target: Realtek High Definition Audio HDAUDIO\FUNC_01&VEN_10EC&DEV_0899&SUBSYS_102805B7&REV_1000\4&220B1BBC&0&0001 WDTF_IoReplay : INFO : Data file located at : \??\C:\iospy\IoSpyData_00000037 WDTF_IoReplay : INFO : Result: Number of IOCTL entries: 59936772. Number of IOCTL to replay 2285436 |
Below is sample powershell script that is used to call the Replay action for a Microsoft LifeCam Camera or Media DUT, for example:
|
# Create the WDTF COM object $wdtf = New-Object -ComObject "WDTF2.WDTF" # Query for all started Camera device or Media $devices = $wdtf.DeviceDepot.Query("IsDevice AND Class='Media’ AND IsStarted=True") # Loop through each device and replay all I/O foreach ($device in $devices) { $ioReplay = $device.GetInterface("IoReplay") $ioReplay.IoReplayAll() } |
Following powershell script can be used to disable IoSpy
|
# Create the WDTF COM object $wdtf = New-Object -ComObject "WDTF2.WDTF" # Query for all started Media devices $devices = $wdtf.DeviceDepot.Query("IsDevice AND Class='Media' AND IsStarted=True") # Loop through each device and disable IoSpy foreach ($device in $devices) { $ioSpy = $device.GetInterface("IoSpy") $ioSpy.Disable() }
|
Advantages of the Device IOCTL/WMI Record and Playback platform
There are several advantages to employing the DRPP during ‘fuzz’ testing, including:
- The system produces a data file and a binding file that can be used to reproduce a bug on a remote machine with the same hardware configuration as the system on which the files were generated (portable repro).
- This file can also be used to test a DUT without having the Fuzz Test or any Simple I/O plugins on the machine- that is, setup requirements are much smaller.
- The replay of the IOCTL or WMI IRP should run quicker and only present relevant IO to the DUT and thus test time can be reduced.
- The data file and binding file can be tailored to run on similar devices that are part of the same device class.
- The data file can be used to playback the I/O to any number of DUTs on single system.
- The data file can be used to playback the I/O to any number of DUTs on multiple systems.
- We can share the data files when we need to reproduce a security bug that causes a BSOD.
- Microsoft can reuse these data files in the WDK and HLK to test device classes for which we don’t have SimpleIO plug-ins (Window Driver Test Framework plug-ins).
- These files can be shared via the various hardware kits (HLK, WDK, etc.) with third party developers.
- Microsoft can learn about high-risk devices- those classes of devices which have a high number of security risks associated with them based on the results of running the DRP and associated Fuzz Tests.
Extensions
- We can install the recording driver (IoSpy) in a lab and run other tests from Microsoft and third parties, and share the data files with other teams.
- We can reuse the data files to do security testing (e.g. IOCTL buffer attacks) on any number DUTs.
- We may allow third parties to tailor the binding data so they can replay or do security testing on similar devices of the same device setup class.
- We can collect the training data from many machines at Microsoft and share it with third parties via Azure. This should help shorten the time required for security testing.
Future Opportunities
- Supply the data files to IHVs to reduce the test cycles.
- Ability to merge multiple data files for a given DUT or class of DUTs.
- Edit and update the binding files.
- Remove IOCTL entries from the data file.
References
U.S. Patent 10,922,249 B2. Available at: https://patents.google.com/patent/US10922249B2
Contributors:
Prashant Chahar, Con McGarvey