March 2020 significant update to Hybrid Configuration Wizard
Published Mar 19 2020 09:21 AM 41.6K Views

(Post updated on 3/26/2020 to add more information about installation process)

We wanted to let you know that we are releasing what we consider a significant update to Exchange Hybrid Configuration Wizard (HCW). Along with a handful of small bug fixes, there are four major changes coming that we wanted to share with you:

  1. HCW will no longer enable Federation Trust by default for all installations. Instead, it will only enable Federation Trust if there are Exchange 2010 servers on premises. HCW will call Get-ExchangeServer and if no Exchange 2010 servers are reported, the workflow to enable Federation Trust and subsequently require domain proof will not execute. Note that organization relationships are still created.
  2. When uninstalling the hybrid agent and switching to Classic in the HCW, this action would sometimes fail with a “null reference” error. We have fixed this!
  3. How many of you have hit the HCW 8064 error – unable to configure OAuth, and subsequently had no idea why OAuth failed to configure? Yes, we heard you loud and clear! In this release, we have completely changed the way we enable and configure OAuth. Instead of enabling OAuth at the service layer, we now enable OAuth via a Graph API under the context of the Tenant Admin. This in turn removes the error obfuscation we had with the service layer enablement and allows us to include a detailed error entry in the HCW log. So while you still see the HCW 8064 error in the HCW UI, you can now review the log for the specific error detail which will make it easier to troubleshoot and resolve.
  4. When verifying DNS, we had a fallback mechanism that would reach out to an external site to verify domains. While this fallback mechanism was rarely hit, we received overwhelming feedback to not use this mechanism/site as it was not listed in our IPs & URLs web page. We have removed that fallback and now only use the endpoint “mshybridservice.trafficmanager.net”, which is listed in our endpoints documentation.

Because this is a major version update, the build begins with 17.x vs 16.x. The build number can be found in the top right corner once you download and open the HCW.

Because of the web-based distribution nature HCW uses and this version is a brand new package, you will get all this goodness simply by installing the new HCW from here. The current builds of HCW (16.x) will not automatically update to 17.x build, in fact – you could run the two side-by-side. Once you are on 17.x build – the HCW will then auto-update as usual.

A few additional notes: At this time, we do not anticipate new HCW 16.x builds. Therefore, to continue getting new HCW builds in the future, uninstall the current version of HCW (16.x) and then install the new version (17.x). The new version of HCW has a new dependency, .NET 4.7.2. The installer should take care of this for you, but just so you are aware.

On installation of the new version of HCW

Generally speaking, the HCW is a standalone, stateless app that gathers configuration information and applies the necessary changes to get to your topology into the desired state. These changes are applied at the Exchange Organization level (both on premises and in the cloud/tenant configuration). Those settings are not stored or managed by the HCW after the run has completed. The Hybrid Agent, once installed and configured runs on the specified machine and is also not tied to the HCW app after installation completes.

Unless you need to re-run HCW, there is actually no need to uninstall and reinstall the new version. In fact, once hybrid is configured, the HCW itself could be uninstalled and it would have no effect on the hybrid configuration it deployed.

In case that at some time you want to re-run HCW, you would want to run the latest version… and that is why we mentioned that you should uninstall the 16.x version and install the 17.x version. But if you do not plan to re-run HCW, you do not need to actually do anything.

Exchange Hybrid Team

30 Comments
Copper Contributor

Good Afternoon,

 

I've posted before but you've retired the original announcement and corrected as I expected.

So we have 2 scenarios:

1) Mixed Environment with Exchange 2010: The HCW enable Federation Trust between Exchange on-premise and Exchange online

2) No Exchange 2010 in the Organization: THe HCW enable OAuth-based authentication between Exchange on-premise and Exchange online

 

Organization relationships is always created.

 

Is correct?

 

Thanks in advance

 

Best regards

 

Danilo Belcastro

Microsoft

@DaniloBelcastro, yes, Organization Relationship is still created.

Brass Contributor

Hi,

So I have to write it again :D

We are facing a problem with HCW which we did not have before, dunno why, but we have our guess, so:

1. We have over 250 domains and until recently HCW run correctly and created 2 organization relationships, one with 238 domains and the other with rest of them

2. Now, HCW creates the 2 org relationships still like it used to, but fails at the New-IntraOrganizationConnector step for the reason of "over 250 domains in the tenant"

Why Org relationship can be created with HCW and divide the domains between two relationships, but the intra org connector cannot be created and HCW fails without possibility to continue?

Even if we create the intraorg connector manually in PowerShell, HCW removes that and tries to create the intraorg connector and of course fails becauese of more than 250 domains.

Before we did not have the problem but is it due to the fact that with EXCH2010 in the organization, the intraorg connector is not created? Cause we removed EXCH2010 at the beginning of the year.

Kind regards,

Krzysztof Sienkiewicz, Microsoft Core Infrastructure Specialist @Demant

Brass Contributor

Thanks for the detailed error message about HCW 8064 error :smile:

 

2020.03.20 07:53:54.177 *ERROR* 10133 [Client=UX, Page=Configuring, fn=RunWorkflow, Workflow=Hybrid, Task=IntraOrganization, Phase=Configure, Step=OAuth, Thread=16]
Graph service call (Graph API) has failed with the exception: System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
File name: 'System.Net.Http, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
at Microsoft.Online.CSE.Hybrid.Common.GraphServicePrincipalProvider.UpdateSPN(String[] acceptedDomains, String[] servicePrincipalNames, X509Certificate2 authCertificate)
at Microsoft.Online.CSE.Hybrid.StandardWorkflow.IOCConfigurationTask.UpdateGraphSPN(ILogger logger, String[] acceptedDomains, String[] servicePrincipalNames, IExchangeCertificate certificate, IGraphService graphService)
WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

Is it possible to distribute an "Update" for the HCW Version 16, which tells the admin about the new version and the manuall uninstall and download? So everybody knows, that they are riding a dead horse and have to switch.

Copper Contributor

What are the consequences of uninstalling the current version of HCW (16.x)? It brokes the Hybrid Configuration or the Hybrid Configuration object stored in Active Directory? Or all continue to function normally?

Do you have a page (I searched for it but didn't find it) for HCW builds history and changelog?

 

Thanks in advance and best regards

 

Danilo Belcastro

Microsoft

@Peter Forster , if you download the net 4.7.2 framework, does the problem go away?

@Frank Carius , I'm looking into exactly this right now.

@Krzysztof Sienkiewicz , Please open a support case so we can identify the source of this.

Brass Contributor

@xC0000005 I already did, was closed with "this is by design" kind of answer ;)

Steel Contributor

Can you please advise if, for Exchange 2013 and newer, Federation Trust's provided functionality is better provided by either A) OAuth between Exchange/EXO, or B) the Hybrid Agent (for non-full Hybrid)?

 

I'm trying to find documentation that states Federated Trust is not required for 2013 and newer hybrid deployments.  This article is close :

 https://docs.microsoft.com/en-us/exchange/configure-oauth-authentication-between-exchange-and-exchan...

 

Question 2, in the absence of Federation Trust's, is OAuth being setup a prerequisite to Organization Relationship's, or do these simply work without any other backing system, for 2013 and newer?

Copper Contributor

@xC0000005 and @The_Exchange_Team 

Can You please give me a clarification about my doubts:

- What are the consequences of uninstalling the current version of HCW (16.x)? It brokes the Hybrid Configuration or the Hybrid Configuration object   

   stored in Active Directory? Or all continue to function normally?

- Do you have a page (I searched for it but didn't find it) for HCW builds history and changelog?

- With HCW v. 17 we have 2 scenarios?

   1) Mixed Environment with Exchange 2010: The HCW enable Federation Trust between Exchange on-premise and Exchange online

   2) No Exchange 2010 in the Organization: THe HCW enable OAuth-based authentication between Exchange on-premise and Exchange online

 

Thanks in advance and best regards

 

Danilo Belcastro

Copper Contributor

When this will be rolled out for all users ?

Microsoft

@DaniloBelcastro I just added a few paragraphs into the original post related to installation of HCW. Hope that clarifies it.

Copper Contributor

@Nino Bilic thank you so much, the clarification was easy to understand and well explained. :)

You've solved all my doubts about HCW uninstall/install procedure.

 

Could you please, if you're eligible to do so, kindly clarify also this question:

- With HCW v. 17 we have 2 scenarios?

   1) Mixed Environment with Exchange 2010: The HCW enable Federation Trust between Exchange on-premise and Exchange online

   2) No Exchange 2010 in the Organization: THe HCW enable OAuth-based authentication between Exchange on-premise and Exchange online

 

Thanks in advance and best regards

 

Danilo Belcastro

 

@DaniloBelcastro The HCWv17 performs:

 

  DAuth OAuth Note
Ex2010/Ex2013+ Mixed Yes No (manual if needed w/Ex2013+)  
No Ex2010 No (new) Yes will still create OrgRel



Copper Contributor

When downloading the new hcw with the new Edge browser I got the error: “Deployment and application do not have matching security zones”

When downloading the file with Internet Explorer the file starts correct. Found this solution on: https://bwit.blog/fix-deployment-application-not-matching-security-zones/

Brass Contributor

I need to add a new hybrid domain to the configuration in a few weeks.

 

My action plan would be:

 

1.) Install new hybrid wizard

2.) Run new hybrid wizard and add new hybrid domain - See if everything works

3.) If everything is O.K. i would uninstall the old one

4.) If it fails i would use the old (16.x) one to add the new hybrid domain

 

Two Questions:

 

1.) Is there actually any change (federation trust, oauth, anything else?) to the hybrid config when  running the new hybrid wizard in an existing hybrid setup that was configured with the 16.x version?

2.) Will the proposed action plan work as described?

Copper Contributor

Hello Exchange team,

 

Currently upgrading hybrid exchange 2010 to hybrid exchange2016 CU15.

My plan is to run HCW on Exchange 2016 EAC and add two Exchange 2016 servers and not to remove Exchange 2010 servers from hybrid configuration.

Is it going to work and later next will run the HCW again and remove one server Exchange 2010 at a time and will under observation and next time run the HCW and remove the last exchange 2010 server.

 

Is it going to break the connection between my onprem and O365 cloud.

Regards

Anand Sunka

Brass Contributor

Great, awesome :)


We received error HCW8064 when configuring the HCW on our Exchange 2016 environment. After reading this article we removed version 16 (which we installed via http://aka.ms/taphcw), we installed version 17 (http://aka.ms/hybridwizard) and the wizard was successful. Why does Microsoft use multiple URLs for the HCW wizards? This is confusing?

Brass Contributor

Great to have version that again works with OAUth. However, I would be happy if you could finally switch from this application type, that reqires Internet Explorer to download. It is a mess! Modern applications like the Windows Admin Center don't even support IE anymore... Why not simply provide an EXE that can also be download with other browsers?

Steel Contributor

@Christian Schindler fyi new Edge handles clickonce apps no problem.  You have to enable it then it works fine.

Brass Contributor

@Jeremy Bradshaw have you received or found an answer to your questions? I have the same question.

In addition, I have noticed that the HCW v17 does not update the Default Email Address Policy, is more known about this? Unfortunately, I don't find this anywhere on the internet or on docs.microsoft.com.

Steel Contributor

@ArendvanDijk I never did hear back anything here nor did I find out anywhere else.  My assumption is that the answer is "Yes" for OAuth being setup effectively replaces Federation Trust's functionality.  I have 2010 in my lab and am hesitating to decommission it as I have so many 2010-related projects (and too lazy to spin up a separate other lab).  Eventually I'll lab out all these questions to see the differences between Fed. Trust and OAuth setup.

Copper Contributor

We are running Exchange Server 2013 CU 23 with all updates. When opening the ECP, clicking 'Hybrid' and selecting 'Modify', version 16.x of the wizard installs and runs, even though we have 17.x installed.

 

This causes issues with various processes! Please can the link be updated ASAP?

Brass Contributor

@ChrisMAF It's frustrating because you do exactly what Microsoft asks you to do (https://docs.microsoft.com/en-us/exchange/hybrid-deployment/deploy-hybrid#use-the-exchange-admin-cen...).

I always use this link now, never the built-in link via ECP: http://aka.ms/hybridwizard

Microsoft could provide some more clarity about v16 and v17 and how to download it. It is now often looking for the right information.

Steel Contributor

The way this blog post words it, at the very end of the post specifically, you need not re-download the HCW from either place.  Instead, you do that once and then every time you run it later, just run it from the installed shortcut.  It'll update itself automatically if a newer version is there (within the same major build).

 

So if you go and click the link in the EAC, it makes sense that it installs the version that is there.  

 

So this is two issues - one issue is that Exchange team are having two different versions available in two different places.  The other is customers re-downloading from both places and getting surprised that the version is different.

 

I would just use the external link for v17, then just relaunch it from the installed shortcut or return to the same link, but if going the latter route, keep going back to the same link.

 

Exchange has always been a big customer booby trap like this.  Everywhere you look there is something odd that you just have to deal with as the customer.  This is the product that first refused GUI items in favor of PowerShell, after all.  So it's love hate for all.

Copper Contributor

As one of the other users mentioned. I have over 250 domains and the HCW fails at the New-IntraOrganizationConnector command. The error is below. I can remove some domains from the HCW and it completes successfully, however how can the remaining domains be added, or additional domains in the future?

 

HCW0000 PowerShell failed to invoke 'New-IntraOrganizationConnector': The total number of explicit and implicit subfilters exceeds maximum allowed number of 250. Processing stopped.

 

What is the work around or fix for this?

Brass Contributor

Hi @prestoYO 

Yeah, I described the same problem. What we do now, cause this is affecting both the IntraOrg Connector and the Org Relationship:

1. Run the HCW with first 238 domains selected

2. Afterwards we need to run the Set-OutboundConnector against the hybrid connector created that sends mails from O365 to On-premises Exchange defining all the domains we have (cause it will only have 238 domains after HCW is run)

3. Then we run the New-IntraOrganizationConnector that has the same config as the IntraOrg Connector that HCW created but defined with the rest of the domains above the 238 limit

4. Then we run the New-OrganizationRelationship that has the same config as the Org Relationship that HCW created but defined with the rest of the domains above the 238 limit

But this should really be fixed at some point cause this is major pain in the **bleep** and soon we will also reach 476 domains and it will become even more trickier.

Please fix that!

Copper Contributor

Thanks @Krzysztof Sienkiewicz 

 

So in the end you have two IntraOrganizationConnectors and two OrganizationRelationships? 

 

Are you doing something like this?

 

$or=Get-organizationrelationship "Name of the organization relationship created by hybrid" New-organizationrelationship -Name "O365 to On-premises2" -DomainNames "Remaining domains to be added separated by commas" -FreeBusyAccessEnabled:$true -FreeBusyAccessLevel "LimitedDetails" -DeliveryReportEnabled:$true -MailTipsAccessEnabled:$true -MailTipsAccessLevel All -PhotosEnabled:$true -ArchiveAccessEnabled:$true -TargetApplicationUri $or.TargetApplicationUri.OriginalString -TargetAutodiscoverEpr -$or.TargetAutodiscoverEpr.OriginalString

Brass Contributor

@prestoYO 

Yeah, basically something like that. And yes, then we have 2 Intra Org Connectors, 2 Org Relationships but with same settings and then 1 outbound mail connector that has all domains added.

Steel Contributor

@The_Exchange_Team Hoping you might circle back to this thread to help me with a quick question.  For Exchange 2013 and new environments setting up Hybrid for the first time, meaning no Federation Trust gets created, where do the EXO-based Organization Relationship find the value for "TargetApplicationUri"?  The Demistify Hybrid Free/Busy post states:

 

"

  • TargetApplicationURI - This must match the ApplicationUrI from On-Prem. Example: FYDIBOHF25SPDLT.contoso.com
    • You can find the value by cross-checking On-Prem's (Get-FederationTrust).ApplicationUri

"

 

We've encountered a client environment whose Organization Relationship (the EXO-based one) has blank TargetApplicationUri, and so when we do (in EXO v2 PowerShell) Test-OrganizationRelationship, we get a Watson dump.  If we populate TargetApplicationUri with FYDIBOHF25SPDLT.<anyOfTheirAcceptedDomains>, this lets the Test-OrganizationRelationship run without Watson dump, but it then fails saying no matching application with URI <our fabricated attempt at the URI> could be found.

 

I'm unaware of where else to look for what the TargetApplicationUri should be.  Any tips would be greatly appreciated.

Version history
Last update:
‎Mar 26 2020 07:02 AM
Updated by: