Announcing: Office 365 endpoint categories and Office 365 IP Address and URL web service
Published Apr 02 2018 09:00 AM 166K Views
Microsoft

[Originally published for the preview on 4/2/2018 and updated on 7/6/2018. Updated for GA on 9/5/2018]

 

Announcing: The IP Address and URL web services are generally available from 5th September, 2018.

 

Microsoft recently published a set of connectivity principles for Office 365 which provides concise guidance on the recommended ways of achieving optimal performance and connectivity to Office 365. The first of these principles is to Identify and differentiate Office 365 network traffic using Microsoft published endpoints. Endpoints include IP Addresses and URLs that are used to connect to Office 365.

 

The primary benefits of using these web services are that they share the endpoint categories which significantly simplify network perimeter configuration, they are fully automated including automated validation testing them, they can be loaded directly into network devices, and they help automate change management to avoid change related outages. The endpoint categories identify a vital few key network endpoints in the Optimize and Allow categories for Office 365 for which we recommend direct Internet egress.

 

We use web services because they are easier for customers scripts and network devices to call than web pages. Specific scenarios where you might need this data include:

  1. Updating your perimeter firewall to allow Office 365 network connectivity.
  2. Updating your enterprise proxy server to allow connectivity to Office 365 URLs.
  3. Edit PAC files on your users computers to bypass proxy servers.
  4. Bypassing an SSL decrypting network device for Office 365 network traffic.
  5. Bypassing a CASB service for Office 365 network traffic.
  6. Selecting endpoints for bypassing proxy servers and routing for direct internet access at a branch office user location.

These web services directly offer Office 365 IP Address and URL data in JSON, and CSV format for all five Office 365 service instances including Office 365 worldwide commercial, Office 365 operated by 21 Vianet, Office 365 Germany, Office 365 U.S. Government DoD, and Office 365 U.S. Government GCC High. We also generate HTML pages from the data and RSS feeds are available from the web services to help with change notification.

Here’s a few quick links to the web services you can access right in a web browser. These links are provided for the worldwide Office 365 commercial instance as examples only.

 

The current XML file and the old RSS feed will be available until October 2nd, 2018. If you have automation that uses the XML format, you should update that to use the JSON format data. If you are using the old RSS feed you should either move to the new RSS feed, or use the sample Microsoft Flow we have published for getting emails on changes. Developer usage documentation for the IP Address and URL web services are detailed in Managing Office 365 Endpoints – Web Service.

 

The web services include three categories for Office 365 network endpoints as attributes of this data which can be used to simplify management of perimeter network devices:

  • Optimize for a small number of endpoints that require low latency unimpeded connectivity which should bypass proxy servers, network SSL break and inspect devices, and network hairpins. Direct Internet access, such as with SDWAN, is recommended for these endpoints.
  • Allow for a larger number of endpoints that benefit from low latency unimpeded connectivity but do not require it. It is required to bypass SSL break and inspect on these endpoints and to avoid proxy authentication. Although not expected to cause failures, we also recommend bypassing proxy servers entirely, network hairpins, and other network intermediary devices on these endpoints. Good connectivity to these endpoints is required for Office 365 to operate normally.
  • Default for other Office 365 endpoints which can be directed to the default internet egress location for the company WAN.

Use of these categories, how they simplify connectivity to Office 365, and what actions you can take to make use of them is detailed in Office 365 Network Connectivity Principles.

 

The web services and data contained in it are supported by Microsoft. However, you do not need to connect to these web services in order to use Office 365. Keep a local copy of the data and just call them again to check for changes. If you are ever unable to connect to the web services, just use the data you have previously downloaded. When a change is notified, you should have 30 days to make updates.

 

Documentation links:

89 Comments
Great news @Paul Andrew, many thanks for sharing !
Copper Contributor
Thanks Paul, good read! – that should give our network team aa bit to look at - We've had a few challenges with connectivity and latency down at the end of the wet string that comes to the long white cloud - especially having an Express Route alongside for Azure connectivity and playing whack-a-mole with asymmetric routes. Hope you and the family are well :)
Microsoft

Hi Adrian, great to hear from you. Send me your Office 365 network performance challenges offline. Would be great to chat about them and catch up.

Copper Contributor

This is great news for automation. For smaller shops that don't have dedicated network admins (like my organization), will there be some how-to's or FAQ's to create human-readable versions from the web service? Current documentation is a bit arcane for those of us who don't deal with REST or other web API interactions on a regular basis.

Microsoft

@Ryan Sheldon Absolutely. We're keen to get the feedback. What scenarios are on your wish list?

Brass Contributor

it will make many of us life easier - automation is the way to go 

Spoiler
 
Iron Contributor

Good job!!


 

Brass Contributor

Please keep the RSS feed alive. We need human control over this process, especially given how many times in the past there have been typos, mistakes, discrepancies between sources (XML, RSS, reference page), etc. with the current process. We cannot only rely on a hardly readable JSON file for this, especially when there are numerous changes every month. Thanks.

Any EOL on the RSS feed? Many vendors now have solutions parsing those.

Microsoft

The current XML/RSS files and the tables with URLs and IP Address ranges in the HTML page are planned to stay current until October 2nd, 2018. The new /changes endpoint contains structured changes which are more easily parsed for specific endpoint changes than the RSS feed which only structured as RSS format. There's no impact to Office 365 services with this change but people will need to migrate their RSS readers to something that reads the new web services before then to get updates to Office 365 endpoints. Message Center post MC133236 is published detailing the change required.

Copper Contributor

This is great news for automation. For smaller shops that don't have dedicated network admins (like my organization), will there be some how-to's or FAQ's to create human-readable versions from the web service? Current documentation is a bit arcane for those of us who don't deal with REST or other web API interactions on a regular basis.

Absolutely. Please do not end the web page view. We copy and past the IP's directly into firewalls now.

Copper Contributor

Absolutely. We're keen to get the feedback. What scenarios are on your wish list?

 

@Paul Andrew Most common for us is needing to validate which URL's or IP ranges are in use when we encounter network traffic issues, usually with a firewall or AV product. The security appliances we use from Meraki and SonicWall also don't currently seem to have hooks to point to web services to update lists for exceptions, so we need to be able to parse/copy/paste the current lists.

 

The currently hinted at method in the existing documentation to export the current ranges to a CSV table could work, I just feel like I'm missing some necessary knowledge or tools to actually go through that process.

 

Will this change also affect the non-global Office 365 URL/IP lists? (21 Vianet, Germany, GovCloud, etc.) Documentation that explicitly shows how to access those specific values or at least shows where to plug in the appropriate variables would help.

The main point for me is that deprecate also an HTML version is something away from the good documentation.

 

Also, the more important thing here is, that a lot of services like PowerApps, Azure AD, and services stuff around Office 365 and Microsoft cloud services has separate Docs where they publish their URL without the same level of comfort.

 

I tried in past here in forums to make a list of links to all these pages, but after the move from TechNet to Docs, some articles were changed. Maybe Microsoft can a little help with that.

Microsoft

@Thomas Reynolds you will be able to copy/paste from the CSV output with the new web services. There's an example CSV output URL in the above article right at the bottom "The worldwide commercial instance endpoints in CSV format." You can save that as a file.csv and open it in Excel, or just read it on screen.

 

@Ryan Sheldon please use the CSV file to do CTRL-F searching if you need that. Yes, this includes the other service instances and the web service documentation shows how to specify those.

 

@Petr Vlk thank you for the feedback.

Copper Contributor

As others have said, please do not end the web view and the RSS feed.  Trying to update my access lists and WCCP lists from a JSON file will be brutal.

Copper Contributor

@Paul Andrew Great news, we've been anticipating this since the Networking Focus Group session. Do you know if vendors like Palo Alto involved in this implementation? In these cases we rely on the vendor to update their firewall and VPN products to be aware of these changes.    

Microsoft

@John Erwin we are working with a number of network device and service providers to use the new web services directly. I don't have anything to share on the status of that today though.

Copper Contributor

@Paul Andrew any more updates on this automation process and the view of vendors you are working with? Would the plan be to have this natively applied to their product?

Copper Contributor
'You can save that as a file.csv and open it in Excel, or just read it on screen.' Have you clicked on that link Paul? The only thing I need is the ip's in #3. It's a mess. I'm trying to figure a way to pull that out in automatic way for my customer every month. Not fun... :(
Microsoft

@E, Amachaghi We're working with several network vendors with the ask of having them connect and get this data directly.

 

@Thomas Reynolds Thanks for the feedback, we'll look for ways to make this easier.

Copper Contributor

@Paul Andrew Do you have the Palo Alto Networks in the list of vendors you're woking with?

Thanks

Microsoft

@Alexey Goncharov We're talking to a lot of people, but we don't want to list specific vendors until we have testing results for their products. Hope you can understand.

Copper Contributor

@Paul Andrew Sure, no problem. Thanks Paul. 

Copper Contributor
So you see on this page where it shows when the list has been updated: https://docs.microsoft.com/en-us/office365/SecurityCompliance/eop/exchange-online-protection-ip-addr... Where can we check that in the future? CSV, JSON, XML, ENDPOINTS web methods do not show that. I've tried them all. Again I'm looking for the date of that SPECIFIC group, (Exchange Online Protection IP addresses) not the entire US ip list for every service there is as I'm sure that's updated all the time, probably daily.
Microsoft

@Thomas Reynolds You will be able to get the detailed changes from the /changes web method when there's a new version. Today on July 12th, 2018 there is only the original version published so you don't see any changes. Note that the page you referenced is going to be removed in favor of the web services.

 

If this doesn't meet your needs, please can you expand on your scenario so we can understand what data you need and why?

 

Regards,

Paul

Steel Contributor

Is there a list somewhere of vendors that Microsoft has confirmed to have integration with in terms of auto / zero touch updates?

Microsoft

@Dustin Halvorson Not yet, we're still early in that work. Please go ask your network device vendors when they're planning the integration.

Copper Contributor

Paul, We need to update our firewall, email security appliance and archiver appliance with these Exchange IP's in order for email to flow in and out of our organization. Adopting one of the unfortunate new methods we go here: https://endpoints.office.com/endpoints/Worldwide?ServiceAreas=Exchange&NoIPv6=true&ClientRequestId=b... (using a different new unique GUID) and manually copy the section for *.mail.protection.outlook.com(ID 10) into notepad, remove the spaces, quotes, and commas (basically get it back to the NICE clean version that exists now at https://docs.microsoft.com/en-us/office365/SecurityCompliance/eop/exchange-online-protection-ip-addr...) so that we can paste it into our devices. While that new method solves that part of the equation, what it does not solve is knowing when those IP changes other than manually comparing them from time to time with what was published previously. As you can see it's bad enough that we have to go through many more steps to get the IPs, but add to that no time stamp further complicates things. HELP.

Microsoft

@Thomas Reynolds there are three new web methods. The /endpoints method that you reference always gets the latest data. There is also a /version method which shows the version of the data, and a /changes method which returns details of all version to version changes. You can read about all three of the web methods in the user guide which is linked from this page, and here.

Copper Contributor
Paul, That appears to not work. Try the example shown: https://endpoints.office.com/changes/worldwide?version=2018062700&ClientRequestId=b10c5ed1-bad1-445f... You will NOT get the output shown in that example. You get 'No HTTP resource was found that matches the request URI...' Please advise.
Microsoft

@Thomas Reynolds thanks for pointing out the error which I'll get corrected. The usage for /changes is correct in the rest of the usage document. The URL you're using should read: https://endpoints.office.com/changes/worldwide/2018062700?ClientRequestId=b10c5ed1-bad1-445f-b386-b9...

 

Paul

Copper Contributor
Paul, Thanks for checking into that. However the URL you posted does not help me. That just shows the changes. I need to know the date they changed, and again specifically JUST that service area.
Microsoft

@Thomas Reynolds The /changes web method does provide all of that, but there's no output right now since we just launched. You should read the user docs for the web services and learn about the /changes result schema.

Copper Contributor

@Paul Andrew

 

I have updated my script. Now it uses new web service. It will generate lists with all ports and all IPs used by O365 (in format ready to import into firewall).

 

Regards, Halis

 

Steel Contributor

@Paul Andrew

Have you asked customers what they want? If customers want to consume IP changes via HTML, RSS, or whatever, why not give paying customers what they want? How much extra could it really cost to maintain HTML or RSS feeds in addition to offering a web service? Why does it have to be mutually exclusive? Why the rush to retire these extremely popular human readable options? My guess is that you did not speak to actual Office 365 administrators who also maintain firewalls, and have zero programming REST/Web service experience. Please stop and listen to customers. 

 

Microsoft

@Joe Stocker Thanks for the feedback. We've taken feedback from many Office 365 administrators, network administrators where Office 365 is used, and network device vendors. If you scroll back to the top of this page, you can see that it was updated on 7/6/2018. One of the updates listed is that based on feedback, we are going to continue to provide both RSS and HTML output which will show the same information as the web services. -- Regards, Paul

Brass Contributor

When will this be coming out of preview?  

Microsoft

@Joseph Halpy We will be out of preview in the next couple of months. The data exposed by the web services is already out production data and we will no longer change the web service schema without appropriate versioning so you should start working against the web services now. What we still don't have in preview is support such that if there is an outage of the web services we only have 8x5 PST work hours that they will be supported by my team.

Regards,

Paul

Brass Contributor

@Paul Andrew Thank you for the clarification.  Do you know when the support hours will go 24x7?   We are trying to time the change over from the RSS feed to JSON since the MC article says the RSS feed will stop on 10/2/18.  

Microsoft

@Joseph Halpy If I can get a more specific date than "next couple of months" then I'll share that here. I encourage you to move to the new publishing right away. 

Regards,

Paul

Copper Contributor

Hi,

 

The current RSS feed gives an effective date in the future of upcoming additions/removals etc.

 

It appears that the REST service only provides changes that "have happened" and that you're only going to know about changes that require implementing immediately - putting you on the back foot, especially if you require change requests/liaising with 3rd party network teams etc. to implement firewall changes.

Is there any mechanism to get notifications of upcoming changes in advance like the current system provides?

 

Thanks

 

Steve

 

Copper Contributor

There are duplicates in the data. eg. sets 11 and 21 appear identical.

 

The effective date is no longer available. How can we determine when the URL's will go live?

 

 

Microsoft

@Steve Ianson Additions are published with 30 days for you to take action. There's no change to this. Please read the description of this policy at http://aka.ms/o365ip.

 

@Ian Williams We have a number of corrections to the data being worked on. Right now we need you to filter out duplicates after downloading. The effective date is listed in the /changes/ web method.

 

 

Regards,

Paul

Copper Contributor

Thanks Paul - still a bit confused though:

 

If you look at the RSS feed there's a publish date and an effective date and the effective date is in the future.

For example:

<item>
<description>Adding 1 IP_Set; 1/[Effective 9/1/2018. Required: Exchange
Online Protection. ExpressRoute: Yes. 23.103.144.0/20]. Notes: Adding an IP
prefix.</description>
<guid>18ef9105-dfb3-403b-bd50-29fe7106efa2</guid>
<link>http://aka.ms/o365endpoints</link>
<pubDate>Thu, 02 Aug 2018 09:46:50 GMT</pubDate>
<title>Exchange Online Protection</title>
</item>

 

So I assume this means on the 02 Aug we got notified of a change which will become effective on 01 Sep?

But if I look at the same thing from the changes rest call (I believe this relates to the same change):

{
    "id": 134,
    "endpointSetId": 9,
    "disposition": "Change",
    "version": "2018080200",
    "add": {
      "effectiveDate": "20180802",
      "ips": [
        "23.103.144.0/20"
      ]
    }
  },

 It looks like the pub date =02 Aug (from version number), but so is the effective date which I would expect to be 30 days later?

So with the rest service, does this mean that IP addresses will actually go live 30 days after whatever the effective date is?

 

Thanks

Microsoft

Hi @Steve Ianson,

 

The effective date doesn't apply in this particular case, and was therefore set to the same date as publishing. The RSS feed effective date is an error and should also have been 2nd August.

 

This IP Address range was part of a removal that occurred on 30th July, but a range was removed by mistake. The change you quote above is adding back that range. Here is the 30th July remove change: https://endpoints.office.com/changes/Worldwide/2018073000?singleVersion&clientRequestId=b10c4332-562... 

 

Regards,

Paul

Copper Contributor

Hi Paul,

 

If this one's an anomoly/edge case then fair enough.

However I've not managed to find any examples where the effective date is 30 days in advance of the publish date. In the link you pasted, the effective dates range from 1 day in the future to several days in the past. Or am I missing something?

 

Thanks

 

Steve

Copper Contributor

as RSS feed will be replaced by new RSS feed so the URL of RSS feed also change or it will be the same as it is currently- https://support.office.com/en-us/o365ip/rss ?

Microsoft

@Abhinay Sharma we will provide the new RSS URL's soon

 

Regards,

Paul

Brass Contributor

Hey everyone, just posted a new blog on how to automate access to the new web service: https://lucian.blog/2018/09/office-365-urls-and-ip-address-updates-for-firewall-and-proxy-configurat...

Copper Contributor

Hi,

 

I think there's a mistake with the recent published changes but would like to confirm this please:

 

Viewing changes for "Worldwide" since 02 August with this url: https://endpoints.office.com/changes/Worldwide/2018080200?clientrequestid=b10c5ed1-bad1-445f-b386-b9...

This includes:

{
    "id": 167,
    "endpointSetId": 24,
    "disposition": "Remove",
    "version": "2018082900",
    "previous": {
      "expressRoute": "true",
      "serviceArea": "Skype",
      "category": "allow",
      "required": "true",
      "tcpPorts": "80,443"
    },

When I then went to cross reference this with the endpoint information for Worldwide (looking for id 24 as this is listed as the endpointsetid), it is not present (jumps from 22 to 25) https://endpoints.office.com/endpoints/Worldwide?ClientRequestId=b10c5ed1-bad1-445f-b386-b919946339a....

I have found endpoint id 24, but it is only returned when querying EPs for Germany - i.e.

https://endpoints.office.com/endpoints/Germany?ClientRequestId=b10c5ed1-bad1-445f-b386-b919946339a7

 

So I assume either this change should not have been listed against the worldwide changes call, or the id should be included under the worldwide endpoints and not just under Germany?

 

Thanks

Version history
Last update:
‎Sep 23 2018 10:40 AM
Updated by: