Office 365 URL based filtering is just better and easier to sustain
Published Dec 02 2013 12:07 PM 67.1K Views

One of the requirements for properly integrating with Microsoft Office 365 is to ensure that your clients and (in some instances such as hybrid Exchange) servers have access to all of the proper endpoints. To achieve this, most customers simply allow their clients internet access and there is no outbound restrictions put in place that would prevent access to the services.

However, there are some customers that want to only allow access to a minimal amount of endpoints on the Internet and have an outbound proxy device that is in place to ensure that they can control this closely. This control can be done in one of two ways:

  • A customer could use IP address filtering which will only allow their internal client machines access to the specific endpoint they specify.
  • They can also use URL based filtering which allows customers to control access by only allowing access to specific URL’s.

Many customers ask, “What are the challenges involved with IP vs. URL filtering? Which option is best for me? Is there a recommended option?”

Keeping up with changes is the first challenge customers may face. The IP addresses and URL’s that are used in the service are mentioned here and if configured today, could change at any time. We do have an RSS feed for this page to try to alert customers of changes and we do try to prevent IP address changes by using larger IP ranges, but in the end there are still times when we have additional datacenters come online or other factors that lead to more IP addresses than is on the list.

Some features will simply not work is the second challenge (such as OWA for customers that decide to use IP addresses as the mechanism for preventing outbound connectivity instead of URL based blocking). The reason behind this is documented here; some of the IP addresses we Use are dynamic and could change without notice for non-secure traffic. Things such as images for OWA are retrieved from third party content delivery networks (a.k.a. CDN) outside of the Microsoft controlled IP address space to improve performance.

Here is the “important” snip from the above mentioned article:

“Microsoft Office 365 relies on third-party content caching engines to achieve good performance and response times. The types of content cached with these third parties are non-SSL resources, such as the images downloaded to draw the Outlook Web App user interface. As stated above, it's possible and supported to use IP-based filtering for the SSL content downloaded from Office 365 and for the Office 365 end-points that make in-bound calls to an on-premises environment. However, it isn’t possible or supported to use IP-based filtering for the non-SSL resources hosted on third-party content caching engines. To express filtering rules that allow those non-SSL resources to be downloaded to clients on your intranet, you need to use hostname-based filtering (as opposed to IP-based filtering). This is because the IPs used by the third-party content caching engines change frequently in a manner which makes it impractical to track each individual IP change. Allow the following hostnames for these non-SSL resources:
r3.res.outlook.com
r4.res.outlook.com
prod.msocdn.com”

You may ask if URL based filtering requires no upkeep and will continue to work after you initially configure it.

The answer is no, not 100% anyway. There will still be times when URL’s for the service could change and changes will be reflected in the articles mentioned above. However, the frequency of the changes is dramatically decreased when URL filtering is used.

Often a customer will choose IP based filtering not because it is easier, but instead because their outbound proxy device cannot do URL based filtering. While this may be true for many older devices, we have seen many times where the application of a software update to your device may allow for URL based filtering functionality. URL based filtering is becoming more prevalent and most devices do give you the opportunity to adapt to this style of filtering.

Some features may not work depending on where users are physically situated is the third major challenge. In some situations we have seen customers attempt to allow only the IP addresses of only Office 365 geographical region datacenters they signed up in. For example, a North American Office 365 customer’s IT staff may attempt to allow only the North American datacenter IP ranges by watching what IP addresses their clients are connecting to then removing the IP ranges they do not see being accessed from the allowed IP list. At first, this may appear to work until (hypothetically speaking) you get a call from your CEO on business travel to Europe asking if Office 365 is down because they cannot access any of the services. Let’s first talk about why this may happen.

Microsoft utilizes geographically aware DNS (aka Geo-DNS) to respond to incoming DNS queries. Microsoft DNS servers first compare the IP address of an incoming request from the querying device against a database of industry standardized IP ranges and what regions of the planet they reside in. Microsoft DNS servers will then issue a DNS response back to the querying device that will be the closest physical entry point to the Office 365 services in the identified IP region. For example a traveling North American Office 365 customer in Europe will be given a European Office 365 entry point. This will prevent the user’s network traffic from having to travel long distances across the public Internet before reaching the desired services. Using Geo-DNS we get the user’s network traffic into the closest Microsoft datacenter and then pass their network traffic across our low latency global private network back to the user’s home datacenter region. This allows Microsoft to provide the best user experience possible for all regions of the planet by reducing dependencies on the public Internet where possible.

So back to our CEO example. The CEO may be in a European hotel on business using the hotel Wi-Fi with a company laptop to do some work. The DNS response they get back for outlook.office365.com will point them at European datacenter. Due to the fact you have only allowed the IP ranges of the North American datacenters, the user is unable to connect to services. You may think forcing the laptop to always utilize corporate DNS servers would help here. Remember that by going down that path you would be forcing the client to traverse the public Internet to reach the North American datacenter and not be able to take advantage of Microsoft’s low latency private network between its datacenters, thus giving your user a less optimal experience.

What is the best or recommended option is based on the challenges outlined above. Microsoft would like to see all customers use URL based filtering to overcome those challenges. URL based filtering will provide you with the fewest number of changes over time, prevent unwanted situations when some content may be unreachable due to changes at the third-party CDNs level, and allow users outside of their home region to always access the most appropriate datacenter for their client connectivity.

Thanks to Brian Day and Joshua Maher for review / comments on this post.

Timothy Heeney

23 Comments
Not applicable

The name of this site is "The Exchange Team Blog", NOT "The Office 365 Team Blog".

Why doesn’t Microsoft have all the Office 365 stuff on "the Office 365 Blog".

Most of us here are Exchange On-Premises guys.

We DO NOT care about Office 365 & NOT going to the Public Cloud.

Not applicable

Well I must say I agree with Fred.

It would be very nice to have everything about Exchange On-Premises here & Office 365 on another site (Office 365 blog). So Exchange On-Premises admins worldwide will have a dedicated site here :)

Not applicable

I agree with Fred.

This is a great idea.

Please give us a dedicate site for Exchange On-Premises admins worldwide.

Not applicable

@Fred, Joe, Patel (and whoever comes after), the majority of posts on this site remain to on-premises related articles. There is and will continue to be overlap between on-premises only admins, cloud-only admins, and hybrid admins where it often makes sense for us to post O365 (Exchange Online) related material. On-premises only Exchange deployment may also be using online resources through the new Office App model within Outlook/OWA if they have chosen to use the applications, or through Federation Relationships to other organizations. We'll continue to post on-premises articles as we have been so I hope you'll continue to tune into the RSS feed when it most interests you.

Not applicable

@Brian & Microsoft

I think this site has become a communication bridge between Exchange On-Premises admins worldwide & Microsoft. And I hope Microsoft will bank on it & continue with great Exchange On-Premises articles here.

By allowing us to leave comments here, we will have this great communication bridge between us here.

I do agree with Fred, and let's continue this great communication bridge between Microsoft & Exchange On-Premises admins worldwide in this site :)

Thanks

Not applicable

I actually do not agree with Fred. If you're thinking that Exchange & O365 don't go hand in hand then you haven't been paying much attention lately. I found this post very informative & useful for someone who works with Exchange & doesn't have their head buried in the sand. The number of Exchange Organizations with at least some presence in O365 is growing rapidly so there's no sense in ignoring it.

If something is relevant to multiple blogs then post it twice; all you have to do is not read a post if it is of no interest to you.

Not applicable

@Andy

It is not about ignoring Office 365, it is about having a site which focuses on Exchange On-Premises.

If you like Office 365 articles, then you can go to Office 365 Blog site.

It is all about making it easy for us Exchange On-Premises admins to come to one single site, and it will be all about Exchange server :)

Not applicable

"Using Geo-DNS we get the user’s network traffic into the closest Microsoft datacenter and then pass their network traffic across our low latency global private network back to the user’s home datacenter region."

I challenge this statement as it's certainly not my experience! I am working with a client with an Office 365 tenant hosted in European data centers. They have an Office in India with external DNS configured for APAC based DNS servers. A network monitor trace shows a client based in India will attempt to access APAC data centers (due to MS GLDNS) before eventually timing out (+30 seconds) and being redirected (not proxied) to the correct EMEA DC's.

We raised a call with MS support who advised we use a conditional forwarder on our internal India DNS servers to resolve outlook.office365.com via the London office.

I will be re-opening the call with MS having read this article.

Also, I often find the clients require investment in their proxy infrastructure to make use to URL based filtering, as all email traffic (including internal email) will now pass through their proxies - in some cases that's a significant amount of data.

Not applicable

@Allan, that may have depended on W14 or W15 of the service, when during each 'W' it was, and what protocol was in use at the time. This is an area the service has matured over time.

Not applicable

Hi, I think this is a great post, which helped me to understand the issues we are currently facing with a customer that wants to implement IP based filtering.

Is there somewhere more information about URLs that are pointed to CDNs for other O365 services? In the blog you are mentioning CDNs for OWA (r3.res.outlook.com, r4.res.outlook.com, prod.msocdn.com). What about the O365 logon page, admin portal and other O365 services? Where can I find URLs pointed to CDNs for those?

The reason I’m asking is, because we are thinking to split the traffic going to O365 and to CDNs. Traffic routed to O365 will be routed directly, but to CDNs through a proxy…

Thx, Sandi

Not applicable

I agree, "The Exchange Team Blog"  means "The Exchange server Team Blog", so this site is for Exchange On-Premises articles.

BTW Microsoft, congrats on Exchange 2013 CU3 On-Premises, great job :) no major issues reported.

Not applicable

I moved most of my customers to Office 365, so please continue with cross-posts between Exchange on-premise and Exchange online

Thanks

Not applicable

@Brian, can you guys work with the TMG2010 people and come up with a comprehensive article for best practices on how to configure TMG2010 for Office 365?  I think that would be very helpful.

Not applicable

Please make it simple for us, "The Exchange server Team Blog" should be a hub site for everything regarding Exchange On-Premises.

Not applicable

Agree with Fred, no interest in hack workarounds required to get Office 365 working, because our customers will never use it

Not applicable

Good advice. I'm finding more and more systems where URL-based firewall access rules are required. Seems this is the new reality.

FWIW I am happy with the mix of Exchange on-premises and Exchange Online (Office 365) content here, since this is the "Exchange Team" blog and you are the team that works on both products/services.

Not applicable

Good advice. I'm finding more and more systems where URL-based firewall access rules are required. Seems this is the new reality.

FWIW I am happy with the mix of Exchange on-premises and Exchange Online (Office 365) content here, since this is the "Exchange Team" blog and you are the team that works on both products/services.

Not applicable

Good advice. I'm finding more and more systems where URL-based firewall access rules are required. Seems this is the new reality.

FWIW I am happy with the mix of Exchange on-premises and Exchange Online (Office 365) content here, since this is the "Exchange Team" blog and you are the team that works on both products/services.

Not applicable

Please keep this site for Exchange On-Premises only, as it has always been.

Please continue with great Exchange On-Premises articles :)

Not applicable

Agree, "The Exchange server team blog" site is for Exchange On-Premises.

Not applicable

One big problem, most proxies cannot do URL based filtering for https sites because they ONLY see the IP, not the hostname/URL.   Therefore, "unblocking" access to office365 over the SSL channel REQUIRES that IP based filtering be used, UNLESS you are using a DLP type product that is playing "man in the middle" decrypting your SSL traffic, and therefore is able to see the URL in addition to the ip address.  Very few products out there do this-- we manage it with Websense Content Gateway along with WCCP, but this is an advanced configuration and in my experience represents a small amount of highly regulated corporate networks.

We went through this exact BS with Office365--hopelessly trying to unblock various IP ranges.  Once we implemented the aforementioned SSL decryption product we were able to use URL's, however the recommendations in your article will be unusable my most networks that do not use such a product.

Not applicable

Correct, site name is "The Exchange Server Team Blog". So please give us deep dive Exchange On-Premises articles.

Not applicable
This is an interesting blog entry that covers off the security perspective from the Exchange Team for outbound security. While it is "possible" to filter based on URL's through a modern proxy server you need to ensure your proxy not only supports URL filtering

but also has the ability to act as a man-in-the-middle for SSL inspection as the URL is embedded within the encrypted payload. There is very little traffic in the O365 solution that isn't encrypted and I'd expect moving forward there will be more encryption

using stronger cryptography. Using an SSL proxy is also going to require a trust relationship between the client and the proxy; requiring the "potential" distribution of certificates to devices, many of which are outside IT's control. It would be possible

to filter based on the SSL Certificate request from the client also, however this is typically cached at the client side and is dependant on the application or browser to refresh the certificate, so this approach cannot be relied upon and isn't mentioned in

any Microsoft documentation. Finally (And I'm aware that this is an Exchange Team blog) there are many protocols used by other clients that simply don't have URL's (IMAPS, POP3S, SMTPS) that don't have URL's for filtering, cannot be proxy'ed and use a potentially

different approach to establishing the SSL or TLS encrypted session. Sorry for posting this to the hosted exchange blog, there simply isn't anywhere on the O365 that has half of what is in this post.

Version history
Last update:
‎Jul 01 2019 04:16 PM
Updated by: