Microsoft Secure Tech Accelerator
Apr 03 2024, 07:00 AM - 11:00 AM (PDT)
Microsoft Tech Community
Microsoft Defender ATP: Web Content Filtering
Published Aug 13 2020 09:00 PM 52.5K Views
Microsoft

Introduction 

 
This is John Barbare and I am a Sr Customer Engineer at Microsoft focusing on all things in the Cybersecurity space. In this blog I will focus on the newly released feature of web content filtering in Microsoft Defender Advanced Threat Protection (ATP)Before July 6th, 2020 you either had to purchase or use a trial license though Cyren or be in public preview to use and/or test the feature.  

 

With the overwhelming positive feedback during the public preview, it was highly recommended that Microsoft enable this feature inside Microsoft Defender ATP. With having a partner license with Cyren to be able to use it, customers did not want to implement it wide scale after the trial/preview and spend additional budget to activate and implement web content filteringMicrosoft is pleased to announce that customers can now activate and use web content filtering without spending any more budget, deploying additional hardware, or purchasing a third party license to use web content filtering through CyrenThe feature is still in public preview and anyone can turn the feature on by turning on preview features and then setting up web content filtering inside advanced features. With that said, lets see what web content filter does, configure the settings, test out in a lab, and then view the results in Microsoft Defender ATP. 

 

Prerequisites for Web Content Filtering 

 

  • Windows 10 Enterprise E5 license 
  • Access to Microsoft Defender Security Center portal 
  • Devices running Windows 10 Anniversary Update (version 1607) or later with the latest MoCAMP update. Note that if SmartScreen is not turned on, Network Protection will take over the blocking.  

Data Processing 
 

Data processing will be handled by the region you selected when you first onboarded Microsoft Defender ATP (US, UK, or Europe) and will not leave the selected data region or shared between any third party providers or data providers. Certain times Microsoft may send the aggregated data to other third parties to assist in their feeds. Aggregated data is the process of combing the results in web content filtering into totals or summary statistics based off the results from applying and using the filtering. These detailed statistics provide companies with answers to large analytical questions without having to sort through private user information and large amounts of customer data that Microsoft deems private. Data processing is kept safe and secure when you enable and use web content filtering. 

 

Web Content Filtering vs SmartSceen vs Network Protection 

 

To gain a better understanding of the differences in web content filtering if you are already utilizing SmartScreen and Network Protection, we will summarize each one to gain a better understanding of what each technology detects/blocks and the similarities and differences of each one.  Web content filtering in Microsoft Defender ATP allows you to secure your devices across the enterprise against web based threats and helps you regulate unwanted content based off multiple content categories and sub categories.  

 

John_Barbare_0-1595942352106.jpeg

 

Microsoft Defender SmartScreen protects against phishing or malware websites and applications, and the downloading of potentially malicious files. This protects users from sites that are reported to host phishing attacks or attempt to distribute malicious software. It can also help protect against deceptive advertisements, scam sites, and drive-by attacks. Drive-by attacks are web-based attacks that tend to start on a trusted site, targeting security vulnerabilities in commonly used software. Microsoft Defender SmartScreen evaluates a website's URLs to determine if they're known to distribute or host unsafe content. It also provides reputation checks for apps, checking downloaded programs and the digital signature used to sign a file. If a URL, a file, an app, or a certificate has an established reputation, users won't see any warnings. If, however, there's no reputation, the item is marked as a higher risk and presents a warning to the user. 

 

John_Barbare_1-1595942352165.jpeg

 

Network protection helps reduce the attack surface of your devices from Internet-based events. It prevents employees from using any application to access dangerous domains that may host phishing scams, exploits, and other malicious content on the Internet. IT expands the scope of Microsoft Defender SmartScreen by blocking all outbound requests to low reputation sources (based on the domain or hostname). When network protection blocks a connection, a notification will be displayed from the Action Center. You can customize the notification with your company details and contact information. You can also enable the rules individually to customize what techniques the feature monitors. Network Protection takes Microsoft Defender SmartScreen’s industry-leading protection and makes it available to all browsers and processes. 

 

John_Barbare_2-1595942352169.jpeg

 

Setting up Web Content Filtering 

 

Navigate to Microsoft Defender Security Center and login with your credentials at https://securitycenter.windows.com/ 

 

John_Barbare_3-1595942352172.jpeg

 

Navigate to Settings and then Advanced features. Make sure Web content filtering and Preview Features are turned to on. In later releases the preview feature will not need to be turned on but as of this article, it will need to be turned on.  

 

John_Barbare_4-1595942352173.jpeg

 

Creating a Web Content Filtering Policy 
 

To create a web content filtering policyclick on Web content filtering under Settings and then click on + Add Item at the top.  

 

John_Barbare_5-1595942352175.jpeg

 

This will bring you to the creation of the initial policy. Give the web content filtering policy a name of your choosing and click next. 

 

John_Barbare_6-1595942352114.jpeg

 

After you have selected next, it will take you to the most important part of the web content filtering policy where you will select which categories and subcategories to block against. The main five categories are adult content, high bandwidth, legal liability, leisure, and uncategorized as seen below. 

 

 

 John_Barbare_7-1595942352118.jpeg

 

Clicking on the arrow next to the categories will dropdown all the subcategories for each individual category. 

 

John_Barbare_8-1595942352121.jpeg

 

You can select the box next to the left of the category to select all the subcategories underneath (category box will turn blue with a check) or just select a few of the subcategories that you want to filter web traffic on (category box will be blue but subcategory will turn blue with a check) as seen below. Once you have decided on what you want to filter, go ahead, and click next. 

 

John_Barbare_9-1595942352125.jpeg

 

 This will bring you to the scope of where the policy will be applied to. You have two options to select 

  1. All devices. 
  2. Select devices. When selecting this, only select device groups will be prevented from accessing websites in the selected categories. All others will not be applied. 

 Also, for user access to devices in a group, you can add in Azure AD user groups and then pick the correct access. Go to SettingsDevice groupsselect the device group, and then select User access. 

 

John_Barbare_10-1595942352177.jpeg

 

In this policy, the Dev Group was selected to apply the multiple categories and subcategories to this web content filter policy and the other groups will not be affected by this policy. Go ahead and click next when done. 

 

John_Barbare_11-1595942352130.jpeg

 

Once you click next, you are able to review the final policy one last time before clicking save to apply to the new policy. 

 

John_Barbare_12-1595942352134.jpeg

 

I’ve tested on my home lab machines and it has taken anywhere from 1-3 minutes to apply on the select machines in my group. With other clients it has taken up to 10, 15, or at the most 30 minutes depending on bandwidth, size of machine group, and how spread apart the machines are in a region.  

 

Creating an Audit Policy and Testing 

 

To deploy an audit policy for web content filtering in Microsoft Defender ATP, do the exact same steps as above, except do not select any of the categories and/or subcategories. Once reviewing that none have been selected, apply the policy to the appropriate device group to audit. By deploying an audit only policy, this will help your enterprise understand user behavior and the categories of websites they are viewing. Then you can create a block policy for the categories and subcategories of your choosing and apply to the select groups. 

 

Before testing my new web content filtering policy on my lab machine, I will create an Audit policy to make sure everything is working.  

 

John_Barbare_13-1595942352137.jpeg

 

Once applied, I will wait the appropriate time to make sure the audit policy has synced with my test machineNext, I will go to a gambling site that is not malicious in nature, no known attack vectors, categorized as a gambling site, and a high URL/IP reputation. The reason for this is because I have SmartScreen and Network Protection enabled along with all the Microsoft Security Baselines for Microsoft Defender Antivirus to include Real Time Protection and direct access to the Microsoft Security Intelligence GraphThis way the site is a known good site and will not get blocked from SmartScreen, Network Protection, or the other security measures I have deployed.  

 

John_Barbare_14-1595942352187.jpeg

 

The gambling site was able to load and was not blocked by any of the security features I had enabled. Since it was in audit mode, this was the expected behavior.  

 

Heading back to Microsoft Defender ATP we can see the connection was made to the site successfully and all relevant information for the gambling site is shown below. Since it was not malicious or had a low URL/IP reputation, it did not get blocked. 

 

John_Barbare_15-1595942352180.jpeg

 

Testing the Web Content Filtering Policy 

 

Since we have the audit policy applied, we will switch the policy to the new policy we created at the start for the Dev Group in which we selected most of the categories and subcategories to include gambling sites. This way we can test the actual policy in block mode and see if the web content filter will block the gambling site we were able to successfully navigate to and also a social networking site. 

  

John_Barbare_16-1595942352184.jpeg

 

Once the policy is synched, I will refresh the browser and see the use of Microsoft Defender ATP web content filtering in action. 

 

John_Barbare_17-1595942352151.jpeg

 

As you can see the same gambling website was blocked using web content filtering. Next, I will test a social media site to see if it will get blocked since we have that checked in our policy. 

 

John_Barbare_18-1595942352156.jpeg

 

If you want to double check the classification of a website against the web filter, you can go here and see where the URL is classified into a category based on a variety of information. 

 

Microsoft Defender ATP Portal – Web Content Filtering Activity 

 

To view all the activity and reports for your web content filtering policies, click on Reports and then Web protection. You can change the timeframe for web activity by category from last 30 days to last 6 months and the other cards can be changed by clicking on the colored bar from the chart in the row.  

   

Web Activity by Category 

 

This card lists the parent web content categories with the largest percentage change in the number of access attempts, whether they have increased or decreased. You can use this card to understand drastic changes in web activity patterns in your organization from last 30 days, 3 months, or 6 months. Select a category name to view more information about that particular category. 

 

In the first 30 days of using this feature, your enterprise might not have sufficient data to display in this card. After the 30 days, the percentages will show as seen in the above screenshot. 

 

Web content filtering summary card 

 

This card displays the distribution of blocked access attempts across the different parent web content categories. Select one of the colored bars to view more information about a specific parent web category. 

 

Web activity summary card 

 

This card displays the total number of requests for web content in all URLs. 

 

View card details 

 

You can access the Report details for each card by selecting a table row or colored bar from the chart in the card. The report details page for each card contains extensive statistical data about web content categories, website domains, and device groups. Here I am selecting the Web content filtering summary colored bar to see all the activity from web categories, domains, and device groups and specifically the gambling website I tested. 

 

  • Web categories: Lists the web content categories that have had access attempts in your organization. Select a specific category to open a summary flyout. 

John_Barbare_19-1595942352158.jpeg

 

  • Domains: Lists the web domains that have been accessed or blocked in your organization. Select a specific domain to view detailed information about that domain.  

John_Barbare_20-1595942352186.jpeg

 

  •  Device groups: Lists all the device groups that have generated web activity in your organization 

John_Barbare_21-1595942352161.jpeg

 

 Use the time range filter at the top left of the page to select a time period. You can also filter the information or customize the columns. Select a row to open a flyout pane with even more information about the selected item. 

 

Errors and Known Issues 

 

As of the publication of this blog, several known issues have been identified and are currently in the process of being corrected. Once corrected, this section will be updated and/or deleted from this blog post. 

  • Only Edge is supported if your device's OS configuration is Server (cmd > Systeminfo > OS Configuration). This is because Network Protection is only supported in Inspect mode on Server devices which is responsible for securing traffic across Chrome/Firefox. 
  • Unassigned devices will have incorrect data shown within the report. In the Report details > Device groups pivot, you may see a row with a blank Device Group field. This group contains your unassigned devices in the interim before they get put into your specified group. The report for this row may not contain an accurate count of devices or access counts. 

Conclusion 


Thanks for taking the time to read this blog and I hope you had fun reading how to use the newly released feature of web content filtering in Microsoft Defender Advanced ATP that everyone can use now that has access to Microsoft Security Center. Some of my customers are currently paying/using a third party proxy, but now they can rest assured that using web content filtering in Microsoft Defender ATP can be used in its place. Before using a Microsoft security feature for the first time be sure to test in audit mode first before putting in block mode. 

 

Hope to see you in my next blog and always protect your endpoints!  

Thanks for reading and have a great Cybersecurity day!  

 

Follow my Microsoft Security Blogs: http://aka.ms/JohnBarbare  and also on LinkedIn.  

 

35 Comments

Thank you @John_Barbare for Sharing this Awesome Blogpost with the Community :cool:

Copper Contributor

Is there a way to exclude a site from the block list?  For instance if we blocked web-based email, could we add an exception for gmail?

Microsoft

@BrianMills you create the exclusion via the IOC section.

1. Advanced Settings

2. IOC --> Add

3. add url

4. Set to ALLOW

 

the IOC allow rule will take over vs the web filtering rule

Microsoft

@BrianMills Here is what you want to do in the portal. Also for "Action" set to allow. Thanks @Brian Tirch for responding quicker!Allow Gmail.jpg

Hi John - when is the web content filtering feature expected to be GA?

Copper Contributor

Does this meet CIPA compliance?

Copper Contributor

Hello 

 

I don't get the option to select the Device group. By default it says 'all devices' 

 

any pointers will be highly appreciative.

 

thank you team

vish

Copper Contributor

The problem with device groups is that they are used for other things as well. A device can only be in in one group at the time. So I can't see how I can deploy different policies to different computers. Is there any plan to have new device groups that only select web content filtering policy. It would also be nice if you could set the ranking on those groups.  

Iron Contributor

Similar to @Sverker Vinell comment is that it would be great to be able to apply not just to a device but also users for shared devices. In a computer lab or factory setting, it is common to have individuals with different levels of approval to use common machines.

 

This is a great start though! I love that it is tightly integrated with the OS and harder to bypass.

Iron Contributor

This is a good post but there isn't much information about what to do if it isn't working. I have found other resources talking about the policies that need to be enabled via either group policy or InTune but nothing that explains the configuration process end-to-end.

 

I have been trying to deploy this to a pilot group and so far have not figured out how to get it to block sites.

Copper Contributor

Hi there,

 

Thanks for this article really useful. I'd love to suggest that Microsoft allow the customisation of the block page. I think that my colleagues might be a bit shocked with a big red page. Love to be able to add corporate branding and be able to change the hyperlink to direct users to our ICT support page instead of Microsoft - after all we created the filtering policy not Microsoft

Iron Contributor

Any plans to support this on OSX or Linux? 

Copper Contributor

How to change the default Microsoft Web Content filtering and customize it to your corp?

Copper Contributor

@Marc Rohde  did you manage to find a solution to this not been applied? I have set it up as per the instructions, unfortunately it is not working for me either. I have applied it, gone through it all, the policy appears to be applied and yet no filtering. Any info kindly appreciated (preview is also enabled) and using E5 licensing.

Iron Contributor

@solegroup  it is working but I think it ultimately just took a really long time to start working. I would guess 48 or 72 hours but my memory is a bit foggy.

Copper Contributor

@Marc Rohde thanks so much. Had it for a week and wasn't working. I had to add some extra policies to get it working but it's a bit messy. Thanks for the reply 

Iron Contributor

it would be nice to add test URLs to https://demo.wd.microsoft.com/

Thanks!

Copper Contributor

Hi, 

 

I have configured our web content for auditing so far.  However, data does not seem to be coming through from our Mac and iOS devices, even though MDATP is set up on these devices and web protection enabled.  Is this expected behaviour?  Any tips on how to get these types of devices reporting?

 

Cheers.  

Iron Contributor

Nice feature.

 

We need more categories to block anonymization services like web proxies and Tor networks.

 

In the reports it would also be helpful if we could see the entire URL.

 

After changing the content list provider, there are some wrong categories, here there should be the possibility for the admin and the end-user to report a wrong category.

 

Is there a possibility of feedback?

Copper Contributor

We have been testing and its being very useful but for me the use case is slightly confusing.  Most companies push web traffic through a proxy which as well as routing traffic from a single point (or via cloud) also handles filtering/classifying.  As this does not handle routing how would you use it?  For example we would ideally like to replace our proxy but to do that all the clients would need full access to Internet - so how do you achieve that without also allowing non-managed devices access? 

 

If you already have a proxy - why would you use this as having two products doing same thing is confusing to support and confusing for end users?

Iron Contributor

@Andy_Bellperhaps MDATP WCF is not to displace existing sophisticated and feature rich solutions, but an added value product for smaller Microsoft customers without any content filtering product at all. I believe you can enable MDATP WCF without any blocking at all. Just assign the policy with all categories allowed and you will get an extra layer of monitoring.

Iron Contributor

Hello @John_Barbare  is there a way to perform tenant restriction for managed Windows 10 devices with Web Content Filtering?

P.S. This is cross post from MCAS > Block Access to Unsanctioned Apps with Microsoft Defender ATP & Cloud App Security post. MS employee confirmed tenent restriction is not part of MCAS and suggested checking with WCF team.

Copper Contributor

We have a couple internal domains being tagged as pornography, I have submitted the inaccuracy, multiple times. Nothing? I am curious how the calculation is made to make domain porn? Either, some how we have had porn served up off an internal domain and we do not know about it? Or this is not an accurate tagging. The process to question the tagging is pretty obscure. Anywhere I can find out more without opening a case with Microsoft?

Iron Contributor

@denting24by7I suggest to go and open a ticket, but i'm not sure if public-preview is supported. There is indeed very little information on inner functioning of the product.  Try your URLs in https://incompass.netstar-inc.com/urlsearch Since this post link there is throw away comment - "We recently completed the transition from Cyren to NetStar on 12/7. This was a staged rollout, and all Web Content Filtering users are now receiving NetStar categorization."

 

Copper Contributor

Does this feature applied on Chrome/FireFox? As I tried it but only worked with EDGE.

Iron Contributor

@khourani 

 

yep. Verify that network protection is enabled.

https://docs.microsoft.com/en-us/microsoft-365/security/defender-endpoint/enable-network-protection?...

you will get a toast notification if request was blocked. The red alert page is only supported in edge. 

 

 

Copper Contributor

is there a way to block the sites based on AD Group membership?  We have a mobile workforce and the hostname can change on a daily basis. That is why we need a way where the user's can be blocked based on ROLE/AD Group ?

 

Any help is appreciated.

 

 

 

Copper Contributor

If you do a lookup on the URL/IP Lookup | NetSTAR (netstar-inc.com) - there seems to be a lot more categories to choose from. Is adding more categories on the MS roadmap at all, if so is there any expected timeframe? Also, as others have said the ability to use Azure AD Groups would be great and make things a lot easier.

Microsoft

@John_Barbare  Fantastic Article ! Thank you so much for sharing this extremely valuable and very needed details.

Copper Contributor

How do we run a report showing a user activity to include date/time, username, IP address, site URL, Action (Blocked/Access), etc?

Copper Contributor

Is it possible to define the policies to be effective between specific hours?
Thinking in terms of blocking social media but then opening for 2hrs at lunchtime for example.

Iron Contributor

No, there are not time settings for the rules.

Copper Contributor

Thanks for confirming @Marc Rohde.. Would be a good feature tp add to the list. 

Copper Contributor

OECM_Support_0-1648582507541.png

Is there a reason why it's greyed out for me when I try to apply to a group of devices?

Copper Contributor

Every other web filtering service I have used has a way to look up URLs and see what category they are. I can't seem to find anything like that for Defender. Does anything like that exist?

Thanks,
Jason

Version history
Last update:
‎Oct 12 2020 01:45 PM
Updated by: