Forum Discussion
Email not being delivered to M365 and being forwarded back on-prem
Hi All
Hopefully I can explain the issue given it is a bit puzzling and a complex setup.
We have 2 environments/tenants. contosedev.com for dev work and contoso.com for production. We have an on-prem Exchange 2019 infrastructure for contosodev.com and a on-prem Exchange 2016 infrastructure for contoso.com.
Between the on-prem environment we have an Exchange 2019 edge server (not AD Sync'd) for each environment (dev and production) that takes email from on-prem and sends to M365. The on-prem Exchange server has a send connector that routes email destined for contosodev.mail.onmicrosoft.com (dev) or contoso.mail.onmicrosoft.com (production) via these edge servers. The edge servers have a receive connector to take this email and a send connector to then send on to M365. The connectors use certificate validation in each case.
The M365 tenants have an inbound connector to receive this email also with certificate validation. All connectors are setup the same apart from the obvious difference in domains. The tenants are authoritative for their respective domains. For dev contosodev.com & contosodev.mail.onmicrosoft.com (and also the default contosodev.onmicrosoft.com). For production contoso.com & contoso.mail.onmicrosoft.com (and also the default contoso.onmicrosoft.com).
The tenants have outbound connectors to route all email via on-premise Exchange servers. So any email in M365 for say contosodev.com (dev) and contoso.com (production) get routed to the outbound connector and hence on-prem Exchange where they can either be delivered locally or if it an external address they are routed out via our gateway infrastructure.
Each tenant has a test mailbox (shared). The mailbox has been migrated from the on-prem infrastructure to M365. Each has email addresses of contosodev.mail.onmicrosoft.com & contosodev.com for the dev environment and contoso.mail.onmicrosoft.com & contoso.com for production.
Now the puzzling bit.
In the dev environment, if I send an email from an on-prem mailbox to email address removed for privacy reasons, Exchange on-prem sees this as a remote mailbox and sends the email via the edge servers. It arrives in M365, sees it has a mail.onmicrosoft.com address and is delivered successfully to the test mailbox.
In the production environment, If I send an email from an on-prem mailbox to email address removed for privacy reasons, Exchange on-prem sees this as a remote mailbox and sends the email via the edge servers. It arrives in M365, sees it has a mail.onmicrosoft.com address, but instead of delivering it to the mailbox, it then routes it back to on-prem using the contoso.com address, which then causes a mail loop that eventually fails.
The message trace seems to indicate the email is being forwarded, however there are no forward rules or inbox rules. I've even tried another completely blank mailbox that I migrated to M365 with the same result.
Now I've been over the config of both environments, looked at various articles in regards to attribution, but cannot see any difference between what I've setup in the dev environments vs the production one.
I just can't work out why, when the mailbox obviously exists in M365 with all the correct email addresses, it just doesn't get delivered. M365 seems to ignore that and decide to send it out via the outbound connector. The other weird part is if I disable that outbound connector in M365, the email is delivered to the mailbox correctly!
Anyway, lengthy I know and hopefully have explained the infrastructure, so if anyone has any ideas where I might check next it would be greatly appreciated.
Cheers
Peter
Yes the summary trace shows the email being forwarded rather than being delivered to the mailbox even though there are no forward or inbox rules. I'm starting to look at security now as I can see incoming email with @contoso.com is being tagged as spoofed. It's possible the records setup for our dev environment are different to prod in relation to SPF and DMARC. This is controlled by a 3rd party and maybe they've changed something we aren't aware of. I'm not sure how M365 works with spoofed emails. Would that be something preventing delivery and sending back to on-prem given we are using a CMT for all our email flow?
Subject: test
Sender: email address removed for privacy reasons
Recipient: email address removed for privacy reasons
Received -> Processed -> Delivered
Status: The message was forwarded to the Inbox folder of the following address:<br/><br/><b>Redirected to:</b> email address removed for privacy reasons
More information: <div>If the sender didn't expect the mail to be forwarded, ask them to follow these steps to stop forwarding messages to another email address:<ol><li>In Outlook on the web, on the <a href='https://outlook.office365.com/owa/#path=/options/mail' target='_blank'>Options</a> page, go to <b>Mail</b> > <b>Accounts</b> > <b>Forwarding</b>. (See Note.)</li><li>Choose <b>Stop forwarding</b>.</li><li>Choose <b>Save</b>.</li></ol>Note: If you don't see <b>Forwarding</b> under Accounts, then your mail is probably being forwarded due to an Inbox rule. For more information on changing Inbox rules, see <a href='http://go.microsoft.com/fwlink/p/?LinkId=708514'>Organize mail using Inbox rules in Outlook on the web</a>.</div>
Date (UTC+10:00) | Event | Detail |
------------------------------------
10/17/2024, 3:17 PM | Receive | Message received by: SY0P300MB0069.AUSP300.PROD.OUTLOOK.COM using TLS1.2 with AES256
10/17/2024, 3:17 PM | Resolve | The message was resolved to email address removed for privacy reasons.
10/17/2024, 3:17 PM | Send external | Message sent to autodiscover.contoso.com at 1.2.3.4 using TLS1.2 with AES256
- OK. Still have no clue what was screwed up. I re-ran the Hybrid config wizard, which thankfully now has the option to just redo certain bits. It doesn’t really cater for our CMT scenario with edge boxes between on-prem and M365, so fixed up the on-prem send connector, fixed the inbound and outbound connectors and hey presto all works. Go figure.
I'll have to spend a bit of time and try and find out what specific thing has changed with one of those connectors, Ugh.
