How to capture underlying outbound traffic from Cloud Service Web Role to other servers

It’s common that a part of the data of a web page is saved in another service such as storage account or SQL server. When the website is hosted on Azure Cloud Service, when we visit the page, the w3wp process of IIS component will need to send out a request to the target remote server to read the needed data. 

But when the Cloud Service fails to read the data from the remote server and developer wants to troubleshoot this issue, it will be difficult as by default users are unable to track the outbound traffic from Cloud Service to these remote servers. If user can capture trace of inbound traffic from the remote server side, it’s good as user can somehow make sure that the request from Cloud Service reaches remote server successfully and get the response code by filtering with the source IP address. But if the remote server is also a Cloud platform service, such as Azure Storage Account which will not allow user to capture network trace, it will be a big problem. 

This blog will mainly talk about how to capture the outbound traffic initiated by the Cloud Service application and sent by application (w3wp process in IIS component). 

Pre-requisites: 

In order to follow this blog, user should have already done following points: 

• Have a Cloud Service with WebRole project deployed (About how to create Cloud Service, please check this official document. About the sample project, please find it from here.) 
• Enable RDP on the Cloud Service instance (Related official document) 
• Install classic Fiddler in Cloud Service instance (Download link of classic Fiddler) 

Background of the sample project in this blog:

To make this blog easier to understand, here is a short explanation of the sample project first. 

This sample project is a simple MVC based webpage server project. It contains in total 3 pages, index, about and contact page. According to the controller, every time when user visits the about page, it will build connection with a remote Azure Storage Account and query some data from the Table endpoint of the Storage Account. The IP address of the remote Storage Account is 20.150.90.38. 

Once the data is read from Azure Storage Account, it will insert it into the about page of the website. The text is “string to display”, and the about page will look like: 

Required change to capture outbound traffic from w3wp.exe 

The configuration needed is as following: 

<system.net> 
  <defaultProxy> 
    <proxy autoDetect="false" bypassonlocal="false" proxyaddress="http://127.0.0.1:8888" usesystemdefault="false" /> 
  </defaultProxy> 
</system.net> 

Since the .Net application of Cloud Service will definitely start before user can RDP into the instance(s), as explained in the official document of Fiddler, user can only do some configuration change in order to meet the purpose. There are two possible options here but the nature of them is the same: 

• Either user can find the web.config file in his project, add necessary configuration, then redeploy the project into Cloud Service. (The advantage is that this can make the project allow users to capture outbound traffic of the application anytime and the change only needs to be made once. The disadvantage is that this way needs redeployment which may be difficult for the production environment. )

• Or user can directly RDP into the instance, then find the application folder which is in drive E or F. The path will be E:\sitesroot\0\Web.config or F:\sitesroot\0\Web.config. User only needs to modify the Web.config file in the RDP window and it will take effect immediately. ( The advantage is this way doesn’t need new deployment so it’s easier to apply. The disadvantage are mainly two points:  1. If there are more than one instance, user needs to RDP into every instance one by one and manually do the change.  2. This configuration change is temporary. As explained in this document, when the Cloud Service is rebooted/reimaged from Azure Portal, experiences Host/Guest OS upgrade, Stop/Start Service, experiences in-place upgrade, node migration or rebuild API is called, the application disk E/F will be rebuilt which means the manual change will be recovered. )

The effect of this change: 

Before: 

The WireShark (Layer 4/TCP) and Fiddler (Layer 7/HTTP) are both running and capturing the traffic. When someone visits the specified webpage (About page in this example) hosted in this Cloud Service, from WireShark trace, it’s clear that there is outgoing traffic from local (10.0.1.4) to target storage account (20.150.90.38) and the remote storage account server returns the application data to this Cloud Service instance, but there is nothing captured in Fiddler. 

After: 

After the required configuration change is added/uncommented, this time, the traffic from this Cloud Service instance to target Storage Account sent by w3wp process can be successfully captured in Fiddler as well. If the decrypt HTTPS traffic feature is configured correctly (only needed when the underlying traffic from w3wp is using https protocol), when user double-clicks on the captured request, we can see the request details as normal requests sent out from browser in Fiddler.