Getting the best connectivity and performance in Office 365
Published Nov 06 2017 12:02 PM 142K Views
Microsoft

Traditional enterprise networks are designed primarily to provide users access to applications and data hosted in company operated datacenters. A secondary use has been as a gateway for access to the Internet for communications and web browsing. In this model, there is minimal or no network security between users and the company operated datacenters, and a substantial security perimeter between users and the Internet with many network devices such as firewalls, anti-virus scanners, data loss prevention, and intrusion detection devices.

Because of the large network security stack, Internet connectivity for branch offices is commonly centralized and backhauled over the customer’s wide area network (WAN). This model worked well for secure access from users within the office to corporate on-premises apps such as email and document sharing where bandwidth could be assured, and minimal network security within the WAN meant network traffic was not impeded.

Traditional enterprise network backhauling Internet bound traffic over its WANTraditional enterprise network backhauling Internet bound traffic over its WAN

As enterprise adoption and reliance on SaaS apps like Office 365 continues to grow, and as employees work from more varied locations, the old methods of backhauling traffic to a central location for inspection creates latency and leads to a poor end user experience. The shift from accessing enterprise applications in a customer operated central datacenter to Office 365, and the differences in traffic patterns, performance requirements, and endpoint security needs to be acknowledged and planned differently when compared with simple Internet communications and web browsing research connectivity.

 

The Microsoft global network and Office 365

The Microsoft global network is one of the largest network backbones in the world consisting of high bandwidth links that have minimal network congestion, with thousands of miles of privately owned dark fiber, multi-terabit network connections between datacentres, and application front doors servers spread around the world. Over 100 public Internet peering interconnection locations on this network makes it easy for all users, regardless of location, to connect into the network using the Internet and access services such as Office 365, Azure, Xbox, Bing, Skype, Hotmail and more.

Microsoft continues to invest in the network, the geographical locations of the application front doors, public peering partnerships with ISP’s, and traffic backhauling capabilities. This allows user network traffic to enter the Microsoft global network very close to the user, and then the traffic is backhauled at Microsoft’s cost over high bandwidth lines within the network to the location where the user’s data is stored.

 

Microsoft global network with each of the blue dots representing Office 365 front end servers around the worldMicrosoft global network with each of the blue dots representing Office 365 front end servers around the world

Office 365 connectivity principles

Microsoft recommends using the Internet and a simple network design for optimal connectivity and performance in Office 365. A key goal in the network design should be reducing the round-trip time (RTT) from your network into the Microsoft global network and ensure that the network traffic is not hair pinned or centralized to specific locations. Use the Office 365 connectivity principles to manage your traffic and get the best performance when connecting to Office 365.

 

1. Identify and differentiate Office 365 traffic using Microsoft published endpoints


Office 365 URLs and IP addresses aka.ms/O365IPOffice 365 URLs and IP addresses aka.ms/O365IPAs a SaaS application Office 365 has a large number URL’s and IP Addresses representing Office 365 service front end servers. We refer to these URL’s and IP addresses as endpoints and customers can use them to identify specific network traffic that is destined for Office 365.

Identifying Office 365 network traffic is the first step in being able to differentiate that traffic from generic Internet-bound network traffic. Microsoft publishes the Office 365 endpoints and guidance on how best to use this data. An Office 365 administrator can use a script to fetch the endpoint details and apply it to a perimeter firewall and other network devices. This will ensure that traffic bound for Office 365 is identified, treated appropriately and managed differently to network traffic bound for generic and often unknown Internet web sites that employees may browse. See the Office 365 endpoint categories and Office 365 IP Address and URL web service to automate endpoint management.

 

2. Egress Office 365 data connections as close to the user as practical with matching DNS resolutionLocal Internet egress into Microsoft's networkLocal Internet egress into Microsoft's networkMany enterprise WANs are designed to backhaul network traffic to a central company head office for processing before network egress to the Internet. Because Office 365 runs on Microsoft’s large global network that includes many front end servers around the world, there will often be a network connection and front end server close to the user’s location.

When compared to backhauling data across the corporate WAN, the user is most likely going to get better performance by egressing Office 365 network traffic to the Internet close to their location where it can be connected to Microsoft’s global network. Additionally, many Office 365 applications use DNS requests to determine the user’s geographic location. If the users DNS lookups are not done at the same point as the network egress the user may be directed to a distant Office 365 front end server.

By providing users with local Internet egress and local DNS resolution their network traffic destined for Office 365 can connect to Microsoft’s global network and Office 365 front end servers as close as possible to the user. Shortening the network path to Microsoft’s global network and to Office 365 front end servers in this way should be expected to improve connectivity performance and the end user experience in Office 365.

 

3. Avoid network hairpins and optimize connectivity directly into the nearest entry point into Microsoft's global network
Enterprise network hairpinning Office 365 bound Internet trafficEnterprise network hairpinning Office 365 bound Internet traffic
Microsoft is continuously working on reducing the distance between users and Office 365 endpoints, driving down latency and improving end user experience. There are two types of network route hairpin that may occur in connecting users to Office 365. These network hairpins greatly lengthen the network path between a user and Microsoft’s global network, and this increases network latency and reduces performance of Office 365.

As discussed, the second type can result from a cloud based network security infrastructure device. If the network device vendor has limited hosting locations and directs a user to a specific one that is distant from them they may create a hairpin route where network traffic goes from the user to the distant network device and back to an Office 365 front end server that is near the user. This can be avoided by asking cloud based network security vendors about the specific locations of their hosting and being critical of the network paths that this creates that may be different to the direct route to Office 365 endpoints on Microsoft’s global network.

The first type results from misaligned network egress and DNS lookups for a user. This can result in the user being directed to an Office 365 front end server that is close to them, but via a distant corporate egress location at a head office. This can be avoided by local egress and local DNS as outlined in the principle above.

 

4. Assess bypassing proxies, traffic inspection devices and duplicate security which is available in Office 365Bypassing additional security for Office 365Bypassing additional security for Office 365

Generic Internet web browsing traffic to unknown Internet sites can have substantial security risk and most enterprises implement network security, monitoring, and traffic evaluation technology at their Internet egress locations. Network security technology includes proxy servers, inline SSL break and inspect of network traffic, network layer based data loss prevention, and more. Network security devices is a strongly growing industry. Unfortunately, whilst all this equipment reduces the enterprise risk of Internet connectivity, it also increases the cost and resources required for Internet connectivity, and it reduces the performance for network connections.

Office 365 servers are all hosted in Microsoft datacenters and Microsoft is very transparent about datacenter security, operational security and risk reduction around those servers and the network endpoints that they represent. These security details can be found in the Microsoft Trust Center. Office 365 also has many other methods available for reducing that network security risk including the built-in security features in Office 365 such as, Data Loss Prevention, Anti-Virus, Multi-Factor Authentication, Customer Lock Box, Advanced Threat Protection, Office 365 Threat Intelligence, Office 365 Secure Score, Exchange Online Protection, Network DDOS Security, and other many other security features.

Enterprise customers should review these risk reduction methods specifically for Office 365 bound traffic and use the Office 365 in-built security features to reduce the reliance on intrusive, performance impacting, and expensive network layer security technologies for network traffic that is identified as Office 365.

 

The Office 365 networking product group would like to learn about your networking challenges when  connecting to Office 365. Please comment on this blog to start a conversation.

 

Resources

15 Comments

Nice, thanks for sharing

Copper Contributor

There are many Cloud connectivity Service providers or Internet access Security providers claiming to have optimized connectivity with Office 365, having direct network peering with Microsoft datacenters or present in the same datacenters as Microsoft, having local DNS resolutions of Office 365 services etc....  Is Microsoft doing such kind of arrangements with such providers or it's only with ISPs?

Also in case of geographical spread organization with Office tenant in one region, the traffic in any case has to travel to central tenant location. What is the best way to connect:

- Connect directly through Internet in different geographical locations. ISPs connect you to local MS datacenters  and then reach your central tenant.

Or

- Route your office 365 traffic on WAN to the location nearest to Tenant and then reach your tenant from there.

Microsoft

Hi @rajesh arora - Microsoft has peering agreements with over 2500 ISPs in 150+ peering locations around the world where they can exchange traffic into the Microsoft 8075 network https://aka.ms/8075. The recommended way for optimal Office 365 connectivity is to egress locally using the Internet, and perform DNS name resolution locally, which then connects the user into the nearest Office 365 application front door, after which the connection travels on the Microsoft global network to the datacenter location where the user's data is stored.

Copper Contributor

Hi Sameer,  Thanks for your clear response.

One more points to clarify:

What you mentioned is about the "Client Connectivity", but what about Server to Server connectivity. For Exchange & Skype for Business Hybrid integrations, the on premise servers need to connect with Online servers and vice versa.  What is the recommended way to connect and why? Direct Internet or Peering/ExpressRoute? This is especially important from Security point of view as well since exposing your servers direct on internet (through reverse proxy) is not secure and also brings reliability/robustness issues into picture (e.g DOS attack on servers). A dedicated connectivity can prevent these type of issues. Please share your thoughts around that....

Thanks,
Rajesh

Microsoft

Hi Rajesh,

             Our recommendation for most customers is to use the internet path for Office 365 traffic as ExpressRoute is inherently complex to implement and invariably the internet path provides the optimal method in terms of cost and complexity. You also may want to think about connectivity which is required over the internet, for example if you move Exchange or Skype Hybrid endpoints to be accessible over ExpressRoute, they are then not reachable to your users or other entities you want to expose them to publicly. As an example, if you move Autodiscover to be only exposed over ER then your users would have to VPN back into the corporate network to get their mail in some scenarios.

Even when a customer uses ExpressRoute, we generally recommend inbound (MS to customer) connections are kept on the internet path to keep the service availability the same and also simplify the routing model.

From a security standpoint, we publish the IPs used for the service so access can be restricted to Microsoft endpoints where appropriate. 

 

Hope that helps,

 

Paul

Brass Contributor

"An Office 365 administrator can use a script to fetch the endpoint details and apply it to a perimeter firewall and other network devices."

 

Has this been actually observed in the wild?  I tried this, and engaged my (not cheap) VAR for support, and they said it was too complex.  The manufacturer said it was too complex.  I'm managing the 1,400+ IPs and URLs by hand.


Microsoft

Hi @david williams - We are making it simpler for customers to identify, differentiate and treat Office 365 endpoints much more smoothly. Please see the new announcement at aka.ms/ipurlBlog. + @Paul Andrew for awareness.

Thanks Paul for this article. With you permission i will translate it in French for my customers. This article is useful and actually we have to cope with in a 40 K users 0365 international onboarding. Geo location helps but not always. In some places especially in south Asia we have poor connectivity to the first Microsoft entry point. For security reasons we will also deploy Zscaller services in an large scale but we know that it will not fix all our bandwidth and latency issues. Bandwidth and latency is still an issue less for Outlook than Skype for Business (Sorry Teams ;) and choosing 0365 will implies additional challenges and will also challenge the old network configurations. Lost of customers do not envision that before launching their 0365 unfortunately. Laurent TERUIN [MVP] lteruin@hotmail.com Global architect in a (NonDisclosure) company.;-)
Microsoft

@laurent Teruin - Glad to see that the article is useful, please feel free to go ahead and translate this post into French! @Paul Andrew for awareness

Copper Contributor

Awesome to see MS providing such detail about something we ran into several year ago.

 

I wrote an article about this specifically for Outlook experience to O365 and we wrote an endpoint testing tool to help organizations understand if their physical users/campuses are getting served "nearest endpoint" values from DNS.

 

One thing that i think was left out was the importance of choosing an external DNS resolver that is location specific.  We first hit this issue because a customer had their internal DNS servers forwarding to Googles 8.8.8.8 hosts and for which Microsoft cannot really determine geo-location from.

 

 

Copper Contributor

Good article! 

Is possible to obtain the PPT presentation (Key elements of Office 365 connectivity strategy based on real-life examples - BRK3041)  showed on the video https://youtu.be/19a8s90HboQ ?

Thanks in advance,

Adrian

 

Microsoft

Hi @AdrianLarsen - a newer version of that session was presented at Ignite 2018. Please see https://myignite.techcommunity.microsoft.com/sessions/64275 - you will also see a link to the slide deck on the right. Let me know if you have any questions. Thanks!

Copper Contributor

We have a single O365 tenant that is hosted in W-Europe. However, we have a business unit with approximately 1200 users in Japan. This raises some concerns about SPO performance. Assuming that DNS is properly implemented to point them to the closest SPO "service front door", would  adding O365 CDN capabilities (https://docs.microsoft.com/en-us/office365/enterprise/use-office-365-cdn-with-spo) further improve performance and end user experience? It was always my understanding that caching is done automatically as part of some "internal CDN". So, I am a bit confused about the need for an explicit CDN.

 

Thanks for clarifying.

 

KR

Paul Stawski

Microsoft

Hi @Paul Stawski  With local breakout for the Office 365 endpoints in Japan you should hit the local front door for SharePoint Online and also the CDN content Microsoft publishes (scripts/generic images etc), regardless of where your tenant resides.

This doesnt use DNS but an Anycast address to find the nearest SPO front door which should be in Japan near your egress. SharePoint CDN as per the link you sent, can be used if you wish to host elements of your SharePoint pages within a CDN as opposed to within SPO itself. In which case it may help with performance by delivering them locally. Not all customers/pages require this additional step so i'd start first with checking performance with a local breakout in Japan for the required endpoints. 

 

Regards,

 

Paul

Copper Contributor

Hi MS Team / @Paul Collinge 

 

Is access to the SharePoint front door service better when your trace route include answers from spo-msedge.net?

 

Example output:

 

1167-ipv4e.clump.prod.aa-rt.sharepoint.com. 60 IN CNAME 19899-ipv4e.farm.prod.aa-rt.sharepoint.com.
19899-ipv4e.farm.prod.aa-rt.sharepoint.com. 60 IN CNAME 19899-ipv4e.farm.prod.sharepointonline.com.akadns.net.
19899-ipv4e.farm.prod.sharepointonline.com.akadns.net. 300 IN CNAME 19899-ipv4.farm.prod.aa-rt.sharepoint.com.spo-0004.spo-msedge.net.
19899-ipv4.farm.prod.aa-rt.sharepoint.com.spo-0004.spo-msedge.net. 240 IN CNAME spo-0004.spo-msedge.net.
spo-0004.spo-msedge.net. 240    IN      A       13.107.136.9

I have noticed when spo-msedge.net is not involved, other IPs such as 40.108.195.53 is returned, which is further hops to reach inside the MS network.

 

Tracing route to 13.107.136.9 over a maximum of 30 hops

  1   148 ms   165 ms   146 ms  10.29.0.1
  2   198 ms   199 ms   174 ms  162.253.68.126
  3   186 ms   183 ms   156 ms  vl62.br02.lax03.as46562.net [184.170.244.8]
  4   174 ms   186 ms   187 ms  184.170.244.0
  5   202 ms   200 ms   202 ms  8075-lax.msn.net [206.223.123.17]
  6   207 ms   183 ms     *     ae27-0.ear01.lax30.ntwk.msn.net [104.44.41.74] <<<<< Entry point in MS network
  7   197 ms   173 ms   246 ms  ae30-0.lax-96cbe-1b.ntwk.msn.net [104.44.40.152]
  8   161 ms   170 ms   161 ms  13.104.141.87
  9     *        *        *     Request timed out.
 10   151 ms   167 ms   171 ms  13.107.136.9
Tracing route to 40.108.195.53 over a maximum of 30 hops

  1   155 ms   166 ms   160 ms  10.29.0.1
  2   189 ms   181 ms   154 ms  162.253.68.126
  3   163 ms   179 ms   174 ms  184.170.244.6
  4   173 ms   162 ms   155 ms  8075-lax.msn.net [206.223.123.17]
  5   145 ms   153 ms     *     ae23-0.ear01.lax31.ntwk.msn.net [104.44.41.107]
  6   313 ms   364 ms   293 ms  be-20-0.ibr01.lax31.ntwk.msn.net [104.44.32.15]
  7   329 ms   313 ms   321 ms  be-9-0.ibr03.cys04.ntwk.msn.net [104.44.16.227]
  8   327 ms   315 ms   345 ms  be-3-0.ibr03.dsm05.ntwk.msn.net [104.44.17.165]
  9   341 ms     *      316 ms  be-2-0.ibr02.dsm05.ntwk.msn.net [104.44.17.31]
 10   300 ms   315 ms   331 ms  be-6-0.ibr02.ch2.ntwk.msn.net [104.44.18.216]
 11   300 ms   368 ms   533 ms  be-3-0.ibr02.cle02.ntwk.msn.net [104.44.29.252]
 12   319 ms   310 ms   330 ms  be-6-0.ibr04.bl20.ntwk.msn.net [104.44.30.5]
 13   322 ms   303 ms   313 ms  be-9-0.ibr02.nyc30.ntwk.msn.net [104.44.28.54]
 14   291 ms   341 ms   311 ms  be-7-0.ibr02.lon22.ntwk.msn.net [104.44.18.155]
 15   351 ms   308 ms   317 ms  be-1-0.ibr02.lon24.ntwk.msn.net [104.44.16.56]
 16   295 ms   306 ms   300 ms  be-14-0.ibr02.ams21.ntwk.msn.net [104.44.30.111]
 17   310 ms   296 ms   290 ms  ae126-0.icr04.ams21.ntwk.msn.net [104.44.23.247]
 18     *        *        *     Request timed out.
 19     *        *        *     Request timed out.
 20     *        *        *     Request timed out.
 21     *        *        *     Request timed out.
 22     *        *        *     Request timed out.
 23   303 ms   310 ms   311 ms  40.108.195.53

Trace complete.
Co-Authors
Version history
Last update:
‎Feb 10 2023 12:29 PM
Updated by: