Exchange message transport & routing has at times been a bit challenging to understand. Whether it was the X400 based Mail Transport Agent (MTA), Open Shortest Path First (OSPF) based routing in Exchange 2000+, or the effect of Database Availability Groups (DAGs) on mail delivery and resilience. Of course if your mailboxes are completely hosted in Office 365, then this is no longer your problem to worry about. But for many of our customers, Hybrid or some form of coexistence is still a concern. If your Office 365 mail routing is anything but the most basic case, you probably need to know a little something about how we do message attribution.
What is message attribution and why do we need it?
One of the harder things to wrap your mind around is how Office 365 attributes messages to particular organizations (tenants). Office 365 is a completely multi-tenant environment – meaning virtually all infrastructure can be shared with other tenants. This includes IP addresses, certificates, transport servers to name just a few components.
When a message arrives at Office 365, one of the first things we need to do is figure out which organization it belongs to. At first, this sounds simple – just look at the recipient, right? Well, it is more complicated than that, because of Hybrid and complex routing scenarios.
For this entire post, we’ll refer to a message that is sent by Office 365 customer contoso.com, destined to another Office 365 customer, tailspintoys.com. Both customers may be hybrid, and so it is possible that the mailbox of the initial sender at contoso.com is hosted on-premises:
By the way, it’s also possible that the configuration could be this:
Or even this:
There are many other possibilities, some of which are more complex. The point is that Office 365 is extremely flexible. So knowing the rules can keep you out of trouble. For a list of all the scenarios which we support, see our best practices document.
Important Note: just because it is possible to route mail in a complex manner does not necessarily mean that the required headers and permissions will be preserved. We frequently see customers configured in complex routing scenarios who are having a poor experience with spam, spoof, and phishing – either false positives or false negatives. This blog only focuses on the mail attribution related pieces. Successful attribution doesn’t make any statement about the security of your configuration.
Before we get into the exact logic, I think it’s important to dispel some myths we frequently hear.
FAQ #1 – Why can’t you use DNS to determine attribution?
When you pick your initial onmicrosoft.com domain, you also pick a primary geography for your tenant. Office 365 then assigns you a unique DNS entry for your onmicrosoft.com domain (and any other domains you verify). This means:
- Currently, all of your domains should resolve to the same set of IP addresses (so technically, you can use a single A record for all MX domain records)
- As with other Exchange protocols, your IP addresses are all the same as every other Office 365 customer in that same geography; assigning unique IP addresses is not an option – we would require roughly 3 IPs per customer, and that would be a too many, given IPv4 addresses are hard to come by these days
- These records & IPs are used for all mail flow, coming from your tenant (originating) or going to your tenant (incoming)
- SMTP has no concept at the application layer that is equivalent to HTTP host headers, so our SMTP endpoint has no idea which DNS entries were used to find us
In other words – we can’t use DNS as an attribution method.
FAQ #2 – Is Office 365 an open relay?
Office 365 has a feature that allows you to relay messages from your on-premises environment. The way you enable this is by creating an inbound connector of type on-premises. This connector can authorize relay either by IP address, or by certificate. The latter is highly preferred, particularly when you’re sending email for domains which you don’t own or haven’t added to your tenant. Certificates are significantly more secure and identify you better than just a IP addresses – as it is very difficult to prove IP ownership definitively. Should two Office 365 tenants specify the same IP or overlapping ranges, then things start to go wrong – for example, if inbound on-premises connectors are created for a shared service.
Sometimes when checking to see if Office 365 is an open relay or not, the IP address from which the test is being run is listed on a connector – inside your tenant, or someone else’s. And since the relay feature is enabled for that IP, the ‘open relay’ test will fail. More on how to protect against abuse later.
Office 365 will delay the rejection of an “unauthorized relay” until “End Of Data” instead immediately after RCPT TO command in the SMTP conversation. This may cause some third party tools to incorrectly flag Office 365 as a potential open relay. You can be assured, however, that this is not the case.
FAQ #3 – Does mail between Office 365 tenants always use MX records?
Yes - except when we’re not expected to. Going back to our example, if a message is truly attributed to contoso.com initially, then when we’re ready to send the message to tailspintoys.com, we will absolutely look up the MX record. However, there are some exceptions you should be aware of:
- Contoso.com can override the routing logic by creating an outbound connector for tailspintoys.com. This case is the most obvious and is no different than Exchange’s (or any other SMTP server’s) routing logic.
- If the mail appears to already be incoming to tailspintoys.com, then we wouldn’t want to route it back out to the tailspintoys.com MX record. This can happen if contoso.com doesn’t have a properly configured inbound connector. Without a proper connector, we see the message as incoming to tailspintoys.com.
- We also give each tenant an onmicrosoft.com address. These addresses can always be used to route directly to Office 365, no matter where the MX record points (this is how cross premise mail flow works between on-premises and the cloud).
If tailspintoys.com does not point their MX record to Office 365, they can tell Office 365 to reject messages which do not originate from the endpoint they want all mails to first route through (including onmicrosoft.com domains). You can do this by creating an inbound connector something like this:
New-InboundConnector –Name "Block delivery unless via MX record" -ConnectorType Partner -SenderDomains * -RequireTls:$true -RestrictDomainsToCertificate $true -TlsSenderCertificateName <the domain or SAN of the certificate used by the endpoint where all mails to first route through>
Or, if you prefer to use IP address instead of certificate:
New-InboundConnector –Name "Reject mail not routed through MX" -ConnectorType Partner -SenderDomains * -RestrictDomainsToIPAddresses $true -SenderIpAddresses <static, full list of on-premises IPs, or IP ranges of third-party filtering service their MX may be pointing to>
Note that by creating a connector like this will cause NDRs to be generated for messages which are submitted directly to your tenant and don’t match the certificate name you provide above. Certificate is always the preferred method. Whether you end up using certificates or IP(s), if either certificate or IP(s) changes, you will reject good messages.
FAQ #4 – Can I use production domains for testing without risk?
If you create a tenant in Office 365/Azure and add production domains to the tenant, you need to be careful. It is likely that your organization already sends email to many other Office 365 customers. As such, real production email will be sent to Office 365 at some point. As soon as you verify the domain, it is possible that mail will be associated with your test tenant, especially if there is an inbound connector inside your test tenant, or if the other party has a configuration issue that causes the mail to be attributed differently than they intend.
Put another way, we’ll start accepting mail for verified domains – unless you tell us not to accept or tell us to only accept conditionally.
FAQ #5(a) – Why does Office 365 allow spoofing?
First, SMTP is inherently not very secure. It is also possible that someone else is spoofing the sender of the message – whether this spoofing is legitimate (like a bulk email service, a partner organization, 3rd party service, etc.) – or not (spam, phish, etc.). While EOP does it’s best to protect you from the latter, we must be extremely careful not to block the legitimate messages. This is no different than any other SMTP server, multi-tenant or not. Very few email servers will outright block email at the SMTP level based on a spoofed MAIL FROM, because there are so many legitimate uses.
FAQ #5(b) – Is Office 365 more insecure because of this?
There are two main standards for detecting bad spoofing. SPF based spoof detection isn’t the best method for a multi-tenant service and is certainly the weaker of the two. We don’t typically use SPF alone to determine that mail is suspicious – it has proven too unreliable. The other standard which we use, DKIM, is more secure. We have some level of anti-spoofing by default using DKIM, and you can always make it better by publishing and using both DKIM and DMARC. We have a Spoof Intelligence feature that uses our knowledge of mail flow patterns as the world’s largest email provider, and also using SPF, DKIM, and DMARC as inputs.
When you use one (or both) of these standards to authorize Office 365 to send mail on your behalf, you are essentially saying that you trust us to prevent bad spoofing of your domain by other Office 365 customers. Read through the links above to build your confidence that we do not take this trust lightly.
FAQ #6(a) – When does the hybrid wizard create an inbound connector with a wildcard in the TLSSenderCertificateName?
Most customers have been using certificates on-premises that take the form of <host>.<rootdomain>, for example, exchange.contoso.com. However, exchange.contoso.com is not usually registered inside of the Office 365 tenant when the hybrid wizard is run. In order to make sure that mail works, we optimized the wizard to instead create the connector with *.contoso.com as the matching certificate. That way (a) the on-premises certificate matches and (b) the accepted domain (contoso.com) also matches. Hybrid wizard does tell you that it is going to do this.
FAQ #6(b) – I don’t want outbound email originating from my on-premises gateway going through my Office 365 tenant. How do I fix this?
Unfortunately, for customers with multiple tenants, or complicated outbound on-premises routing, having a wildcard match may be catching too much mail and attributing it all to the matching Office 365 tenant. The simplest fix – if the hybrid wizard created the connector – is:
- Figure out the specific certificate name utilized by Exchange on-premises (e.g. exchange.contoso.com).
- Add & verify the Exchange certificate name (e.g. exchange.contoso.com) as an accepted domain inside of the correct Office 365 tenant.
- Make sure that the on-premises gateway is utilizing a different certificate (e.g. mailgateway.contoso.com).
- Re-run the hybrid wizard. If you don’t want to re-run the hybrid wizard, then update the TlsSenderCertificateName on the inbound connector to match the exchange on-premises certificate only (e.g. exchange.contoso.com).
How does message attribution work?
So, back to the reason for the title of the post - when a message is sent to Office 365, the Front-end transport servers that receive the message need to determine if the message is originating (coming from a customer) or incoming (going to a customer). In other words, is the current hop inside of contoso.com’s routing configuration or inside of tailspintoys.com?
Definitions
- Originating: A message that originates from a tenant’s on-premises server (it will match the tenant’s inbound connector that is of type OnPremises), or from an Exchange Online mailbox. If Office 365 is unable to identify the message as originating and the recipient domain is an accepted domain in an Office 365 organization, the service will identify the message as incoming to the recipient organization.
- Incoming: Messages sent to accepted domains that are in Office 365 that do not match a tenant connector that is of type OnPremises.
How Office 365 determines to which tenant a specific message belongs
This is the basic logic used. This has been simplified to a small degree in order to cover the top scenarios and make it easier to understand.
Originating
- First, we check to see if the message is Originating. The service uses the following envelope information:
- Certificate presented by the sending server or the Connecting IP
- MailFrom domain
- RcptTo domain
- The certificate name, or Subject Alternative Name (SAN) on the certificate, is used to lookup matching tenant Inbound connectors that are of type OnPremises across all tenants.
- For all matched connectors we next look at which tenant has the TlsSenderCertificateName domain configured as an accepted domain. The most specific match will win.
- If one is found the messages is attributed as Originating from that tenant.
- If no tenant matches were found, we next look at the Connecting IP and find all tenants with Inbound OnPremises connectors which match the IP.
- For all matched connectors we next look at what tenant has the MailFrom domain configured as an accepted domain. If one is found the message is attributed as Originating from that tenant.
Incoming
- If the message has NOT been attributed as Originating, at this point the service assumes the message is coming to us either as MX routed or via relay smart host (we have no way of telling). We then lookup if the RcptTo matches an accepted domain in a tenant and attribute the message as Incoming to that tenant. We then look to see if the message matches any inbound partner connectors in that tenant, and if a match is made, the message is attributed to that connector. If no match is made, then the message arrives at that tenant through the default inbound connector.
- If no tenant is found which has the recipient domain has a validated domain, the message will be rejected by the service.
So now you know how Office 365 Message Attribution works.
With contributions & feedback from many others, especially Bruce Wilson, Stan Aleksiev, Markus Dahlweid