Microsoft Secure Tech Accelerator
Apr 03 2024, 07:00 AM - 11:00 AM (PDT)
Microsoft Tech Community
Converting ETL Files to PCAP Files
Published Jan 28 2020 09:19 AM 155K Views
Microsoft

Hello. Sean Greenbaum here with a tale from the field.  

 

As many of you have probably experienced, when working with Microsoft Premier support, you’ll often be asked to capture some data and upload it to Microsoft for analysis. Maybe you want to review that data yourself. Maybe you or your staff also has the technical expertise to review the data and make some preliminary observations while waiting for Microsoft Support to complete the investigation 

 

If your issue requires network traces to be captured, Microsoft Support will often ask you to capture them running a built-in utility called NETSH. 

 

Some Background 

NETSH is a great tool built into the Windows OS and can be used to configure many parts of the networking stack within your Windows OS. You can read all about what NETSH can be used for here. When using NETSH to capture a network trace, it generates a specialized file with an ETL file extension. 

 

For the last few years, Microsoft has used a variety of tools to decode and view the data in ETL files, mainly NetMon, Windows Performance Analyzer and Microsoft Message Analyzer. No improvements to Netmon have been made since 2010 but is still available for download from MicrosoftWindows Performance Analyzer is a great tool to view ETL files that contain system performance data, but not the best thing for network traces. This brings us to Microsoft Message Analyzer

 

Microsoft Message Analyzer was our tool to capture, display and analyze protocol messaging traffic. It can open ETL files and decode the networking data contained within. It also can export that data into a standard .CAP file which could then be used by lots of other networking applications like Wireshark. 

 

The issue 

Now that we have some background, let's talk about a recent support issue I ran into. One of my customers was having some issues which required us to take a network trace. Microsoft Support asked that they run the standard network trace capture command and switches: 

  • Netsh trace start capture=yes tracefile=c:\temp\%computername%.etl maxsize=1024 filemode=circular 

(Note: If working with Microsoft Support, the Support Engineer may give you a slightly modified version of this command to enable certain trace options specific to your reported issue. Use the switches they provide you if asked.) 

clipboard_image_0.png

 

With the trace now running, the issue now needs to be reproduced. 

Once reproduced, stop the trace to generate the ETL file. 

  • Netsh trace stop 

clipboard_image_1.png

 

Notice that NETSH trace generated an ETL file and saved it in the folder specified when starting the trace. It also captures some related diagnostic information and compresses that information into a CAB file. 

clipboard_image_2.png

 

At this point, Support will ask for either the ETL file, or both the ETL and CAB file depending on the information they are looking for, to be uploaded for analysisMicrosoft Support will analyze the data and will report back with any conclusions or next steps. 

 

But what if you want to review the captured data as well? Simply opening the CAB file you can see there are lots of TXT files with human readable System Information, Registry Keys, and Event Logs. But the ETL file has all the network trace data. How do you get into that? Well, as I mentioned above, Microsoft has the Microsoft Message Analyzer which can open these files and even convert them to a format other networking tools can read. 

 

I advised my customer to download this tool and use it to review the network traces while Support is doing the same. Except, we ran into a problem. Microsoft Message Analyzer has been discontinued. Even worse, Microsoft has pulled Microsoft Message Analyzer from all official download locations effective November 25th, 2019. And there is no replacement in development as of the time of this posting. 

 

Great.  

 

Surely, our customers will want to be able to generate and analyze their own network traces without needing to rely on Microsoft Support. Installing another tool on your systems to capture network traces isn’t always going to be an acceptable option in many companies either. How can we convert these ETL files that the built-in tooling generates? 

 

The solution 

Welcome to the world of Open Source software. In this case, it turns out one of our Microsoft Developers, Matt Olson, thought of this already. Using the official Microsoft GitHub repo, he wrote and published an open source tool that does exactly that, named ETL2PCAPNG

 

ETL2PCAPNG takes an ETL file that was generated using NETSH and converts the network frames to a new version of the CAP format, called PCAPNG. Standard network analysis tools like Wireshark can read this format. 

 

If you don’t feel like building the tool from source, check out the Releases section on the GitHub to find the latest prebuilt executable.  

 

This allows you to generate the ETL file on the server, copy to your local machine, or approved jump box/tools machine/etc and convert the ETL file there. To convert your ETL file the command is: 

Etl2pcapng.exe file.etl newfile.pcapng 

 

Using that to convert the ETL file I captured earlier: 

clipboard_image_3.png

 

 

Now I can open the PCAPNG file in my favorite networking tool. 

Happy troubleshooting. 

13 Comments
Copper Contributor
You sure? Every time MS support has wanted network traces, they've insisted that I install wireshark and don't seem to understand why anyone would be reluctant to do that on a production server that is in use. Most of them had never heard of message analyzer and have no idea how to use netsh. Also, who is the joker that decided to quickly and quietly get rid of message analyzer? It's was one of the increasingly fewer tools from MS that was still very useful, and I rely on it pretty often. I can't really stress enough how much it needs to be put back ASAP.
Copper Contributor

Thanks for the article. There are so many hidden gems on GitHub, but sometimes it takes an article like this to lead us to them. The tool works great.

Copper Contributor

@Sean Greenbaum , THANK YOU. Not later than today, I also noticed that message analyzer was out of support. I ignore during weeks the errors message telling blah blah. I will grab that from github.

@Jordan Mills , wireshark is nice, powerful etc, but since we can do "netsh trace start" , i no longer install wireshark. Netsh , you don't need to maintain it too. And you can use some filter very handy to remove noise and things not relevant. When have your trace, you can copy it and conduct the analysis with the shark. Give a try to netsh trace, it is very easy ;). 

Brass Contributor

ChristopheCLDC I think you misread my post.  Message analyzer uses exactly the same interface as netsh (and New-NetEventSession).  It's all built into the OS, no installation or configuration required.

Microsoft

Since all MS network analysis tool are deprecated (netmon and message analyzer), Hopefully some day in the future Microsoft will work with Wireshark team and help them to have built-in elt parsers directly in Wireshark.... 

Copper Contributor

Are you sure?  What about pktmon.exe?  It was added to windows 10 in October 2018, and seems to use the same ETL interface that netsh and powershell use (and I've seen nothing about either of those being sabotaged either).

 

Why would microsoft kill such a useful tool as message analyzer when so many people are actively using it? 

Copper Contributor

Sadly etl2pcapng doesn't work as well as the message analyser export.     Capturing VPN traffic I find the resulting pcap is missing that VPN traffic even though it is present in the etl file and is properly produced by the pcap export in Message analyser.

 

netsh trace capture used to get traffic before it goes into and after it comes out of the VPN

 

netsh trace start provider=Microsoft-Windows-Ras-NdisWanPacketCapture Ethernet.Type=(IPv4,IPv6,0) Wifi.Type=Data capture=yes correlation=disable overwrite=yes tracefile=vpn-traffic.etl

Bronze Contributor

Along the same lines as what ayshfm1 said above, etl2pcapng also doesn't seem to work with a capture using the Hyper-V VMSwitch provider.  After taking a capture using this command: 

Netsh trace start provider=Microsoft-Windows-Hyper-V-Vmswitch

 When I try to pass the resulting .etl file through etl2pcapng it says 'Input ETL file does not contain a packet capture.'

 

:(

Brass Contributor

Hi @Steve Whitcher,

 

The etl2pcapng only convert the "network trace"/network packets collected with the parameter capture=yes.

These network packets are the ones that can be read with Wireshark.

 

Other providers traces are not converted with etl2pcapng.

 

Copper Contributor

Hi 

Recently I tried converting acpi.etl file using above tool and it also failed with below message . 

 

"Input ETL file does not contain a packet capture." 

 

regards

Atul Vaish 

Copper Contributor

@avintel , basically the ETL file do not contains data when the filter, provider etc is not good or rights are missing.

Personally, I always do that
CMD as admin, and use netsh making firstly a short capture without any filter or provider ... once I am sure that I see traffic, I refine it adding provided, filter etc. IPs... whenever it is possible to target the specific flow.

Hope this help
    

Microsoft

The conversion works fine, as far as I can tell.

 

However, the message analyzer was an official software clients would agree to install in secure environments while they are hesitant with WireShark. This is also the reason why I knew it and why I miss it.

Copper Contributor

According to the source code, currently only Ndis GUID {2ED6006E-4729-4609-B423-3EE7BCD678EF} class decode should be supported
BOOLEAN IsNdisCapEvent = IsEqualGUID(&ev->EventHeader.ProviderId, &NdisCapId) &&
(ev->EventHeader.EventDescriptor.Id == tidPacketFragment ||
ev->EventHeader.EventDescriptor.Id == tidPacketMetadata ||
ev->EventHeader.EventDescriptor.Id == tidVMSwitchPacketFragment);
Will there be an updated version that supports more GUID classes?

Bes Regards

Version history
Last update:
‎Jan 28 2020 09:19 AM
Updated by: