Log Parser Studio 2.0 is now available
Since the initial release of Log Parser Studio (LPS) there have been over 30,000 downloads and thousands of customers use the tool on a daily basis. In Exchange support many of our engineers use the tool to solve real world issues every day and in turn share with our customers, empowering them to solve the same issues themselves moving forward. LPS is still an active work in progress; based on both engineer and customer feedback many improvements have been made with multiple features added during the last year. Below is a short list of new features: Improved import/export functionality For those who create their own queries this is a real time-saver. We can now import from multiple XML files simultaneously only choosing the queries we wish to import from multiple query libraries or XML files. Search Query Results The existing feature allowing searching of queries in the library is now context aware meaning if you have a completed query in the query window, the search option searches that query. If you are in the library it searches the library and so on. This allows drilling down into existing query results without having to run a new query if all you want to do is narrow down existing result sets. Input/Output Format Support All LP 2.2 Input and Output formats contain preliminary support in LPS. Each format has its own property window containing all known LP 2.2 settings which can be modified to your liking. Exchange Extensible Logging Support Custom parser support was added for most all Exchange logs. These are covered by the EEL and EELX log formats included in LPS which cover Exchange logs from Exchange 2003 through Exchange 2013. Query Logging I can't tell you how many times myself or another engineer spent lots of time creating the perfect query for a particular issue we were troubleshooting, forgetting to save the query in the heat of the moment and losing all that work. No longer! We now have the capability to log every query that is executed to a text file (Query.log). What makes this so valuable is if you ran it, you can retrieve it. Queries There are now over 170 queries in the library including new sample queries for Exchange 2013. PowerShell Export You can now export any query as a standalone PowerShell script. The only requirement of course is that Log Parser 2.2 is installed on the machine you run it on but LPS is not required. There are some limitations but you can essentially use LPS as a query editor/test bed for PowerShell scripts that run Log Parser queries for you! Query Cancellation The ability to submit a request to cancel a running query has been added which will allow you to cancel a running query in many cases. Keyboard Shortcuts There are now 23 Keyboard shortcuts. Be sure to check these out as they will save you lots of time. To display the short cuts use CTRL+K or Help > Keyboard Shortcuts. There are literally hundreds of improvements and features; far too many to list here so be sure and check out our blog series with existing and upcoming tutorials, deep dives and more. If you are installing LPS for the first time you'll surely want to review the getting started series: Getting started with Log Parser Studio Getting started with Log Parser Studio, part 2 Getting started with Log Parser Studio, part 3 If you are already familiar with LPS and are installing this latest version, you'll want to check out the upgrade blog post here: Log Parser Studio: upgrading from v1 to v2 Additional LPS articles can be found here: http://blogs.technet.com/b/karywa/ LPS doesn't require an install so just extract to the folder of your choice and run LPS.EXE. If you have the previous version of LPS and you have added your own custom queries to the library, be sure to export those queries as a backup before running the newest version. See the "Upgrading to LPS V2" blog post above when upgrading. Kary WallIntroducing Remote Desktop Connection Manager (RDCMan) 2.2
Inside Microsoft, we maintain a repository of tools written by our engineers and technical staff. Many of the tools that are posted are very specific to Microsoft engineering— tools to help developers and testers better manage their project in our internal source control system, provide better visibility into our internal bug/issue tracking system, etc. Since these tools are very specific to the Microsoft environment, most of them don't get released externally. About eight months ago, I came across a tool in the repository called Remote Desktop Connection Manager ("RDCMan" for short) written by Julian Burger, one of our principal developers on the Windows Live Experiences team. RDCMan is a central place where you can organize, group, and manage your various Remote Desktop connections. This is particularly useful for system administrators, developers, testers, and lab managers who maintain groups of computers and connect to them frequently. As an example - my customer manages over 200 Exchange servers worldwide. Today, they maintain a configuration file for the Remote Desktops MMC with many of their servers. Of course, with 200 servers, it becomes difficult to maintain and navigate, as seen in the following screenshot. Figure 1: Managing RDP connections in the Remote Desktops MMC After I installed RDCMan, it was very clear that our customers and partners would benefit greatly from it, as it fills the gap nicely that the standalone Remote Desktop Connection application and the Remote Desktops MMC snap-in leave behind. Here's a screen shot of an organized RDCMan configuration with the servers organized by version (Exchange 2007, Exchange 2010), region (Chicago, Redmond) and then finally by Exchange role (Client Access, Hub Transport, Mailbox, etc.) Figure 2: An organized RDCMan configuratoin You'll also notice that there's a grid on the right side that has a thumbnail of each of the servers. Yes, RDCMan supports a live thumbnail view of your connected servers, as seen in the following screenshot. Figure 3: RDCMan displays live thumbnails of your connected servers I'll leave the other features for you to discover. With Julian's blessing, I worked with our legal department, trademark group, engineering compliance, release support, and others to get RDCMan licensed for external distribution... and while it's been months in the works - today, I'm excited to announce that Remote Desktop Connection Manager is now available externally on the Microsoft Download Center - get it from http://go.microsoft.com/?LinkID=9733636. David ZazzoIntroducing: Log Parser Studio
To download the Log Parser Studio, please see the attachment on this blog post. Anyone who regularly uses Log Parser 2.2 knows just how useful and powerful it can be for obtaining valuable information from IIS (Internet Information Server) and other logs. In addition, adding the power of SQL allows explicit searching of gigabytes of logs returning only the data that is needed while filtering out the noise. The only thing missing is a great graphical user interface (GUI) to function as a front-end to Log Parser and a ‘Query Library’ in order to manage all those great queries and scripts that one builds up over time. Log Parser Studio was created to fulfill this need; by allowing those who use Log Parser 2.2 (and even those who don’t due to lack of an interface) to work faster and more efficiently to get to the data they need with less “fiddling” with scripts and folders full of queries. With Log Parser Studio (LPS for short) we can house all of our queries in a central location. We can edit and create new queries in the ‘Query Editor’ and save them for later. We can search for queries using free text search as well as export and import both libraries and queries in different formats allowing for easy collaboration as well as storing multiple types of separate libraries for different protocols. Processing Logs for Exchange Protocols We all know this very well: processing logs for different Exchange protocols is a time consuming task. In the absence of special purpose tools, it becomes a tedious task for an Exchange Administrator to sift thru those logs and process them using Log Parser (or some other tool), if output format is important. You also need expertise in writing those SQL queries. You can also use special purpose scripts that one can find on the web and then analyze the output to make some sense of out of those lengthy logs. Log Parser Studio is mainly designed for quick and easy processing of different logs for Exchange protocols. Once you launch it, you’ll notice tabs for different Exchange protocols, i.e. Microsoft Exchange ActiveSync (MAS), Exchange Web Services (EWS), Outlook Web App (OWA/HTTP) and others. Under those tabs there are tens of SQL queries written for specific purposes (description and other particulars of a query are also available in the main UI), which can be run by just one click! Let’s get into the specifics of some of the cool features of Log Parser Studio … Query Library and Management Upon launching LPS, the first thing you will see is the Query Library preloaded with queries. This is where we manage all of our queries. The library is always available by clicking on the Library tab. You can load a query for review or execution using several methods. The easiest method is to simply select the query in the list and double-click it. Upon doing so the query will auto-open in its own Query tab. The Query Library is home base for queries. All queries maintained by LPS are stored in this library. There are easy controls to quickly locate desired queries & mark them as favorites for quick access later. Library Recovery The initial library that ships with LPS is embedded in the application and created upon install. If you ever delete, corrupt or lose the library you can easily reset back to the original by using the recover library feature (Options | Recover Library). When recovering the library all existing queries will be deleted. If you have custom/modified queries that you do not want to lose, you should export those first, then after recovering the default set of queries, you can merge them back into LPS. Import/Export Depending on your need, the entire library or subsets of the library can be imported and exported either as the default LPS XML format or as SQL queries. For example, if you have a folder full of Log Parser SQL queries, you can import some or all of them into LPS’s library. Usually, the only thing you will need to do after the import is make a few adjustments. All LPS needs is the base SQL query and to swap out the filename references with ‘[LOGFILEPATH]’ and/or ‘[OUTFILEPATH]’ as discussed in detail in the PDF manual included with the tool (you can access it via LPS | Help | Documentation). Queries Remember that a well-written structured query makes all the difference between a successful query that returns the concise information you need vs. a subpar query which taxes your system, returns much more information than you actually need and in some cases crashes the application. The art of creating great SQL/Log Parser queries is outside the scope of this post, however all of the queries included with LPS have been written to achieve the most concise results while returning the fewest records. Knowing what you want and how to get it with the least number of rows returned is the key! Batch Jobs and Multithreading You’ll find that LPS in combination with Log Parser 2.2 is a very powerful tool. However, if all you could do was run a single query at a time and wait for the results, you probably wouldn’t be making near as much progress as you could be. In lieu of this LPS contains both batch jobs and multithreaded queries. A batch job is simply a collection of predefined queries that can all be executed with the press of a single button. From within the Batch Manager you can remove any single or all queries as well as execute them. You can also execute them by clicking the Run Multiple Queries button or the Execute button in the Batch Manager. Upon execution, LPS will prepare and execute each query in the batch. By default LPS will send ALL queries to Log Parser 2.2 as soon as each is prepared. This is where multithreading works in our favor. For example, if we have 50 queries setup as a batch job and execute the job, we’ll have 50 threads in the background all working with Log Parser simultaneously leaving the user free to work with other queries. As each job finishes the results are passed back to the grid or the CSV output based on the query type. Even in this scenario you can continue to work with other queries, search, modify and execute. As each query completes its thread is retired and its resources freed. These threads are managed very efficiently in the background so there should be no issue running multiple queries at once. Now what if we did want the queries in the batch to run concurrently for performance or other reasons? This functionality is already built-into LPS’s options. Just make the change in LPS | Options | Preferences by checking the ‘Process Batch Queries in Sequence’ checkbox. When checked, the first query in the batch is executed and the next query will not begin until the first one is complete. This process will continue until the last query in the batch has been executed. Automation In conjunction with batch jobs, automation allows unattended scheduled automation of batch jobs. For example we can create a scheduled task that will automatically run a chosen batch job which also operates on a separate set of custom folders. This process requires two components, a folder list file (.FLD) and a batch list file (.XML). We create these ahead of time from within LPS. For more details on how to do that, please refer to the manual. Charts Many queries that return data to the Result Grid can be charted using the built-in charting feature. The basic requirements for charts are the same as Log Parser 2.2, i.e. The first column in the grid may be any data type (string, number etc.) The second column must be some type of number (Integer, Double, Decimal), Strings are not allowed Keep the above requirements in mind when creating your own queries so that you will consciously write the query to include a number for column two. To generate a chart click the chart button after a query has completed. For #2 above, even if you forgot to do so, you can drag any numbered column and drop it in the second column after the fact. This way if you have multiple numbered columns, you can simply drag the one that you’re interested in, into second column and generate different charts from the same data. Again, for more details on charting feature, please refer to the manual. Keyboard Shortcuts/Commands There are multiple keyboard shortcuts built-in to LPS. You can view the list anytime while using LPS by clicking LPS | Help | Keyboard Shortcuts. The currently included shortcuts are as follows: Shortcut What it does CTRL+N Start a new query. CTRL+S Save active query in library or query tab depending on which has focus. CTRL+Q Open library window. CTRL+B Add selected query in library to batch. ALT+B Open Batch Manager. CTRL+B Add the selected queries to batch. CTRL+D Duplicates the current active query to a new tab. CTRL+ALT+E Open the error log if one exists. CTRL+E Export current selected query results to CSV. ALT+F Add selected query in library to the favorites list. CTRL+ALT+L Open the raw Library in the first available text editor. CTRL+F5 Reload the Library from disk. F5 Execute active query. F2 Edit name/description of currently selected query in the Library. F3 Display the list of IIS fields. Supported Input and Output types Log Parser 2.2 has the ability to query multiple types of logs. Since LPS is a work in progress, only the most used types are currently available. Additional input and output types will be added when possible in upcoming versions or updates. Supported Input Types Full support for W3SVC/IIS, CSV, HTTP Error and basic support for all built-in Log Parser 2.2 input formats. In addition, some custom written LPS formats such as Microsoft Exchange specific formats that are not available with the default Log Parser 2.2 install. Supported Output Types CSV and TXT are the currently supported output file types. Log Parser Studio - Quick Start Guide Want to skip all the details & just run some queries right now? Start here … The very first thing Log Parser Studio needs to know is where the log files are, and the default location that you would like any queries that export their results as CSV files to be saved. 1. Setup your default CSV output path: a. Go to LPS | Options | Preferences | Default Output Path. b. Browse to and select the folder you would like to use for exported results. c. Click Apply. d. Any queries that export CSV files will now be saved in this folder. NOTE: If you forget to set this path before you start the CSV files will be saved in %AppData%\Microsoft\Log Parser Studio by default but it is recommended that y ou move this to another location. 2. Tell LPS where the log files are by opening the Log File Manager. If you try to run a query before completing this step LPS will prompt and ask you to set the log path. Upon clicking OK on that prompt, you are presented with the Log File Manager. Click Add Folder to add a folder or Add File to add a single or multiple files. When adding a folder you still must select at least one file so LPS will know which type of log we are working with. When doing so, LPS will automatically turn this into a wildcard (*.xxx) Indicating that all matching logs in the folder will be searched. You can easily tell which folder or files are currently being searched by examining the status bar at the bottom-right of Log Parser Studio. To see the full path, roll your mouse over the status bar. NOTE: LPS and Log Parser handle multiple types of logs and objects that can be queried. It is important to remember that the type of log you are querying must match the query you are performing. In other words, when running a query that expects IIS logs, only IIS logs should be selected in the File Manager. Failure to do this (it’s easy to forget) will result errors or unexpected behavior will be returned when running the query. 3. Choose a query from the library and run it: a. Click the Library tab if it isn’t already selected. b. Choose a query in the list and double-click it. This will open the query in its own tab. c. Click the Run Single Query button to execute the query The query execution will begin in the background. Once the query has completed there are two possible outputs targets; the result grid in the top half of the query tab or a CSV file. Some queries return to the grid while other more memory intensive queries are saved to CSV. As a general rule queries that may return very large result sets are probably best served going to a CSV file for further processing in Excel. Once you have the results there are many features for working with those results. For more details, please refer to the manual. Have fun with Log Parser Studio! & always remember – There’s a query for that! Kary Wall Escalation Engineer Microsoft Exchange SupportHow to Export and Import mailboxes to PST files in Exchange 2007 SP1
There might be times when an Exchange Administrator will need to export the contents of individual mailboxes to offline files in order to present specific users with a format that is easily portable and ready to consume using Outlook clients. To fulfill this need Exchange 2007 SP1 will have a new set of features to export and import mailboxes to and from PST files. As I know you will ask - yes, those PST files can be bigger than 2 GB, which was a limitation of Exmerge tool used for this purpose in previous versions of Exchange. Export/Import to PST Requirements In order to export or import mailboxes to PST files the following requirements must be met: Export/Import to PST must be run from a 32 bit client machine with Exchange Management Tools installed (Version Exchange 2007 SP1 or later). The 32bit requirement comes from a dependency with the Outlook client. Either Outlook 2003 or Outlook 2007 must be installed on the client machine. The user running the task must be an Exchange Organization Admin or an Exchange Server Admin on the server where the mailbox to export/import lives. Exporting mailboxes to PST files The most basic cmdlet to export a mailbox to a PST file is as follows: Export-Mailbox –Identity <mailboxUser> -PSTFolderPath <pathToSavePST> PSTFolderPath must be a full path pointing either to a directory or to a (.pst) file. If a directory is specified a PST file named after the mailbox alias will be used as the target of the export. Note that if the PST file already exists the contents of the mailbox will be merged into it. Example: After the cmdlet finishes execution, the .pst file will be ready in the specified location: To export multiple mailboxes to their respective .pst files at once you can pipe in the identities of those mailboxes to the export task. Notice that when bulk exporting the PSTFolderPath parameter must forcefully point to a directory since one .pst file will be created for each mailbox. Example: Get-Mailbox -Database 'MDB' | Export-Mailbox -PSTFolderPath D:\PSTs Importing mailboxes from PST files The process for importing mailbox contents from a PST file is quite similar: Import-Mailbox -Identity <mailboxUser> -PSTFolderPath <PSTFileLocation> Again, PSTFolderPath must be the full path to the directory where the .pst file lives or to the (.pst) file itself. In the case where PSTFolderPath points to a directory the cmdlet will try to match the mailbox alias with the name of an existing .pst file in the specified directory and import the content of that file. Example: Just as with the export to PST scenario, when bulk importing mailboxes the PSTFolderPath must forcefully point to a directory and the task logic will try to match mailboxes alias with the .pst file names under that location. If no match is found for a particular mailbox, that mailbox will be skipped. Example: Get-Mailbox -Database 'MDB' | Import-Mailbox -PSTFolderPath D:\PSTs Filtering content in Export/Import to PST When only specific content is desired in the PST file (or back into the mailbox) a common set of filters can be used to leave out the rest of the messages. Export/Import to PST support the following filters: Locale, StartDate, EndDate, ContentKeywords, SubjectKeywords, AttachmentFileNames, AllContentKeywords, SenderKeywords, and RecipientKeywords. Example: Import only those messages that were created between 1/1/06 and 12/1/06 and contain the word "review" in the subject and any of the words {"project","alpha"} in the body. Import-mailbox -Identity ricardr -PSTFolderPath D:\PSTs -StartDate 1/1/06 -EndDate 12/1/06 -SubjectKeywords:'review' -ContentKeywords:'project','alpha' Now, we realize that you would like to try this today, but please be patient! - Ricardo Rosales GuerreroResolving WinRM errors and Exchange 2010 Management tools startup failures
As was discussed in the previous (related) blog post "Troubleshooting Exchange 2010 Management Tools startup issues", in Exchange 2010 the Management tools are dependent on IIS. As was discussed in that blog, we have seen situations where the management tool connection to the target Exchange server can fail, and the error that is returned can be difficult to troubleshoot. This generally (but not always) happens when Exchange 2010 is installed on an IIS server that is already in service, or when changes are made to the IIS server settings post Exchange Install. We have seen that these changes are usually done when the IIS administrator is attempting to "tighten up" IIS security by editing the Default Web Site or PowerShell vdir settings. The situation is further complicated by the fact that some of the errors presented have similar wording; most seem to originate with WinRM (Windows Remote Management), and in some cases different root problems can produce the exact same error message. In other words, depending on how knowledgeable you are with these errors, troubleshooting them is all around... not much fun. I was approached by a good friend of mine and he asked what I thought we could do to make these errors a little easier to troubleshoot. I was studying PowerShell and PowerShell programming at the time (I just happened to be reading Windows PowerShell for Exchange Server 2007 SP1), and I thought that this would be a perfect opportunity to try and apply what I was learning. This is the result. Introducing the Exchange Management Troubleshooter (or EMTshooter for short). What it does: The EMTshooter runs on the local (target) Exchange server and attempts to identify potential problems with management tools connection to it. The troubleshooter runs in 2 stages. First, it will look at the IIS Default Web Site, the PowerShell vdir, and other critical areas, to identify known causes of connection problems. If it identifies a problem with one of the pre-checks it will make a recommendation for resolving the problem. If the pre-checks pass, the troubleshooter will go ahead and try to connect to the server in the exact same way that the management tools would. If that connection attempt still results in a WinRM-style error, the troubleshooter will attempt to compare that error to a list of stored strings that we have taken from the related support cases that we have seen. If a match is found, the troubleshooter will display the known causes of that error in the CMD window. Here is an example of how this might look like: When I was designing the troubleshooter, I could have just written a little error lookup tool that handed over the appropriate content for the error you were getting, but I felt that was not as robust of a solution as I was aiming for (and not much of a learning experience for me). So the tool runs active pre-checks before moving on to the error look-up. The amount of pre-checks it can run depends on the flavor of OS you are running on and the options you have installed on it, such as WMI Compatibility. Basically, I have taken all of the documentation that has been created on these errors to date, and created a tool that will make the information available to you based on the error or problem it detects. Hopefully this will cut down on the amount of time it takes to resolve those problems. Event reporting: When you run the EMTshooter it will log events in the event log. All results that are displayed in the CMD window are also logged in the event log for record keeping. Events are logged to the Microsoft-Exchange-Troubleshooters/Operational event log and are pretty self-explanatory. Things to remember: Depending on your current settings, you may need to adjust the execution policy on your computer to run the troubleshooter, using: Set-ExecutionPolicy RemoteSigned Or Set-ExecutionPolicy Unrestricted Remember to set it back to your normal settings after running the troubleshooter. This version of the troubleshooter needs to run on the Exchange Server that the management tools are failing to connect to. While our final goal is that the troubleshooter will be able to run anywhere the Exchange Management tools are installed, the tool isn't quite there yet. We have seen instances where corruption in the PowerShell vdir or in IIS itself has resulted in errors that seemed to be caused by something else. For instance, we worked on a server that had an error that indicated a problem with the PowerShell vdir network path. But the path was correct. Then we noticed that the PowerShell vdir was missing all its modules, and quite a few other things. Somehow the PowerShell vdir on that Exchange Server had gotten severely... um... modified beyond repair. WinRM was returning the best error it could, and the troubleshooter took that error and listed the causes. None of which solved the problem. So be aware that there are scenarios that even this troubleshooter cannot help at this time. The troubleshooter is still a bit rough around the edges, and we plan to improve and expand its capabilities in the future. We also hope to be able to dig a little deeper into the PowerShell vdir settings as time goes on. Also note that the troubleshooter will NOT make any modification to your IIS configuration without explicitly asking you first. Permissions required: In order to run the troubleshooter, the user will have to have the rights to log on locally to the Exchange server (due to local nature of the troubleshooter at this time) and will need permissions to run Windows PowerShell. Installing the troubleshooter: First, you will need to download the troubleshooter ZIP file, which you can find attached to this post. Installing the EMTshooter is pretty easy. Just drop the 4 files from the ZIP file into 1 folder, rename them to .ps1 and run EMTshooter.ps1 from a normal (and local) PowerShell window. I personally just created a shortcut for it on my desktop with the following properties: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -noexit -command ". 'C:\EMTshooter\EMTshooter.ps1'" However, as most users probably won't run this more than a few times you might not need or want an icon. Just remember that EMTshooter.ps1 is the main script to run. Providing feedback: As I mentioned before, the troubleshooter is still a work in progress. If you wish to provide feedback on it, please post a comment to this blog post. I will be monitoring it and replying as time allows, and also making updates to the troubleshooter if needed. If you run into errors that are not covered by the troubleshooter, please run the troubleshooter, reproduce the error through it and send me the transcript.txt file (you will find it in the folder where the 4 scripts have been placed), along with what you did to resolve the error (if the problem has been resolved). My email is sbryant AT Microsoft DOT com. Errors currently covered: Connecting to remote server failed with the following error message : The WinRM client cannot process the request. It cannot determine the content type of the HTTP response from the destination computer. The content type is absent or invalid. Connecting to remote server failed with the following error message: The connection to the specified remote host was refused. Verify that the WS-Management service is running on the remote host and configured to listen for requests on the correct port and HTTP URL. Connecting to remote server failed with the following error message: The WinRM client received an HTTP server error status (500), but the remote service did not include any other information about the cause of the failure. For more information, see the about_Remote_Troubleshooting Help topic. It was running the command 'Discover-ExchangeServer -UseWIA $true -SuppressError $true'. Connecting to remote server failed with the following error message : The WinRM client received an HTTP status code of 403 from the remote WS-Management service. Connecting to the remote server failed with the following error message: The WinRM client sent a request to an HTTP server and got a response saying the requested HTTP URL was not available. This is usually returned by a HTTP server that does not support the WS-Management protocol. Connecting to remote server failed with the following error message : The client cannot connect to the destination specified in the request. Verify that the service on the destination is running and is accepting requests. Consult the logs and documentation for the WS-Management service running on the destination, most commonly IIS or WinRM. If the destination is the WinRM service, run the following command on the destination to analyze and configure the WinRM service: Connecting to remote server failed with the following error message : The WS-Management service does not support the request. Connecting to remote server failed with the following error message : The WinRM client cannot process the request. The WinRM client tried to use Kerberos authentication mechanism, but the destination computer - Steve BryantIntroducing the Microsoft Office 365 Hybrid Configuration Wizard
For up-to-date information on Hybrid Configuration Wizard steps, please see Microsoft Docs. The Exchange hybrid team has been working hard over the past year getting the 3 rd version of the Hybrid Configuration Wizard (HCW) ready. This new version is called the Microsoft Office 365 Hybrid Configuration Wizard. This article tells you what’s new and shows you how to run the wizard. We also explain the various issues that have been addressed with the new Wizard, and touch on some of the telemetry we pull with every run of the wizard. We think this new wizard has enough of the old to reduce the learning curve while adding plenty of enhancements to make your hybrid deployment as friction free as possible. Microsoft Office 365 Hybrid Configuration Wizard Stand-Alone Application This version of the HCW is a standalone application that is downloaded from the service. This is an important change because one of the bigger limitations of the previous versions of the HCW was that it was included with the on-premises product. This led to the following issues: Up-To-Date hybrid experience: When you ran the HCW you got the experience consistent with your on-premises version of Exchange Server. This meant that if you are running Exchange 2013 CU7 you got the CU7 experience. If you ran Exchange 2013 CU9 you got the CU9 experience in HCW. Each customer would have a different HCW experience. Solution: The new HCW will download the latest version every time it is run, therefore providing the latest and improved experience. As soon as we make changes to, or fix any issues in the HCW, customers will see the benefits immediately. HCW not tied to Cumulative Updates: Since the previous versions of the HCW were part of the on-premises product they were updated per the regular Exchange Serviceability model. This means that the hybrid team had to wait for a new Cumulative Update (Rollup for Exchange 2010) every three months to deliver any enhancements or changes. For a component like hybrid that is a problem, we have to be agile enough to handle changes not just to on-premises, but also in the service. Solution: Again, every time you attempt to run the HCW we will ensure you have the latest version. This version will of course go through its rounds of validation, but it is in no way tied to the releases of a CU. No more waiting months for fixes! Piloting Changes: As we move forward with this new HCW we will be making some aggressive changes. In the months ahead we want to add more capabilities to HCW. One of the most important changes in HCW will be the ability to roll out feature changes slowly and in a controlled manner. Solution: We have built in the capability to allow customers who are on “first Release” and any other customers we specify (for example TAP customers) to see the latest version of the HCW. Often the latest release and the production release will be the same version, but we do have the ability to pilot versions of the HCW as needed. Improvements to error handling The HCW has a lot of dependencies and relies on various prerequisites for a successful completion. For example, you have to add an external TXT record for the HCW to create the Federation Trust, you have to have your certificates properly installed on your Exchange servers, and you have to have Internet access from your Exchange servers to name a few. I am not trying to scare you away from hybrid, in fact the wizard does walk you through most of the prerequisites. I am instead trying to point out that there are many failure points for the HCW to contend with. Up till now the solution was to provide you an error message that included a stack trace. These error messages are extremely difficult to decipher and often the first reaction after a couple of failed Internet searches was to call into support. Figure 1 shows the old error Experience for those that may not be familiar with it. Figure 1: Old error messages Our goal is to allow you to successfully configure without an error, but we also want to make sure that we give you the information needed to get past any hurdles you may face. Figure 2 shows a sample of the new (much more informative) error experience. In the sample you can see the following major improvements to the error experience: Improved Title: We have added the ability to see what Phase and Task were being completed at the time of the failure. For instance, you can know if we failed at the prerequisite check or configuration phase. You will also immediately know if you failed to create the Organization Relationship or Outbound Connectors. Error code: We have added a new error code for all the possible error messages in the new wizard. You will now see all errors prepended with a code HCW8***. This change allows for our errors to be easily searched and it allows them to remain searchable even if we change the context of the errors. Humans can read the errors: One of the previous challenges was that we provided a stack trace as the error message instead of just a friendly actionable string. We now keep the stack trace in the logs for anyone who may want that information. New “More info” feature: We added the “More info…” option under the error message. We have recently associated a KB or TechNet article as the most likely solution to EVERY error message the HCW throws. Simply click the “More Info…” link and you will be taken to that solution Access to log Files: You can easily access the HCW log file by clicking on the link that says “Open Log File”. In addition, you will find the log file on the system were you ran the new wizard from by going to “%appdata%\Microsoft\Exchange Hybrid Configuration”. Keep in mind the old location for the logs in the Exchange install directory is not used. Coolest addition: When you run the HCW you will more than likely have the Exchange Admin Center already open, but there is a chance that if you run into an issue you will need to use either your on-premises or Exchange Online PowerShell. The new HCW error experience includes a link that will open the on-premises and/or Exchange Online PowerShell. We already have the credentials you entered into the wizard, so you can seamlessly open PowerShell by using those credentials. In addition, we open the Exchange Online PowerShell with a blue background and the Exchange on-premises PowerShell with a black background so you can easily differentiate the two. Figure 2: Awesome error experience Top issues solved by the new HCW About a year ago we came out with a toolto assist you to troubleshoot your hybrid experience. This tool collects and parses the HCW log and provided a link to an article that gave a solution to your issue. The tool has been run thousands of times and has given us great insights into what the top failure points are for the HCW. This telemetry tells us what we need to focus on and allows us to see any failure trends, but in the end we were limited to the information gathered from folks that ran the HCW troubleshooter. Because we want to be as helpful as possible, we now by default upload the HCW logs to the service when you run the new wizard. Gathering this data will allow us to serve you better by limiting the amount of time it takes for someone in support to find out more about your environment and it allows us to see any trending issues and failure points that we need to address. Even with the limited amount of logs we have collected from the troubleshooter, we have been able to identify the following issues and are addressing them in the new HCW. I think you will see why the log collection is so important to the hybrid team. Note: If you want to opt out of uploading the Hybrid logs you can do that by using the registry key below on the machine were you are running the HCW from: 1. Navigate to the following location in the registry, create the path if needed: Exchange 2016: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v16\Update-HybridConfiguration Exchange 2013: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\Update-HybridConfiguration 2. Create REG_DWORD “DisableUploadLogs” with value 1 TXT proof string issues Any time you are required to add an DNS entry you are dealing with a potential for failure. HCW includes a step were you need to add a record to an external DNS to prove to the Azure Authorization Service (known as Microsoft Federation Gateway) that you own the domain. This step may seem trivial but it accounts for ~15% of our HCW failures. Usually the TXT proof string get messed up in one of two ways: Incorrect string entered: when creating the DNS record to provide domain ownership we often see that the incorrect value was provided. This is in large part due to the way the HCW copied the value. In the previous version of the HCW, when you copied the TXT string, we prepended the words “Domain Proof” so it looked similar to “Domain Proof = t4jnhkjdesy78hrn…”. Solution: While simple, moving forward we are only going to copy the part of the string that is needed from the “copy link” option in HCW, which should lead to less issues with incorrect TXT strings. Domain name lockouts: The point of providing this TXT string to the external DNS is so the service can validate that you own the domain and federation certificate. After a few failed attempts to validate a domain we lock you out from federating that domain for a few hours. The purpose of this lockout is to prevent a denial of service attack. Often this issue occurs because someone put the wrong value in DNS (see the first bullet), someone created the record and did not wait for replication of the record, or someone created the record in internal (not external) DNS. Solution: To resolve this we created a new external endpoint in the service that will perform the DNS lookup for the TXT record and only try to federate the domain if the record is correct or if that new service endpoint cannot be found. The logic for this is as follows: First we try to hit the new external service endpoint and see if the TXT record is resolvable externally and is correct in DNS. If so, we move forward with federating the domain. If the record is either wrong or not resolvable, we inform you that you need to verify the record and wait for replication. If the new external TXT validation service is not reachable, we will warn you that we could not verify the TXT record but allow you to continue anyway. Figure 3 show the new TXT experience you will be getting with the Microsoft Office 365 Hybrid Configuration Wizard. Figure 3: TXT records Missing Certificate in Wizard The HCW has a screen that asks you for the “Transport Certificate”. The HCW looks to ensure this certificate is installed on every server that you designated to be part of the Send and Receive Connector Configuration, as shown on the pages in Figure 4. Figure 4: Send and Receive Connector In order for the certificate to properly display you need to ensure that the following has been completed on all of the servers designated in the wizard pages shown in figure 4: The Certificate must be a third party trusted certificate. The proper names must be on the certificate such as mail.consoto.com or *.contoso.com. The SMTP service must be assigned to the certificate on each of the sending and receiving servers. The certificates must have a private key. These requirements are nothing new, but if you have a large environment, getting all of this correct on a large number of servers can be a tough task. If even one server was missing any of the requirements, we would fail to show you the certificate. In previous versions of the HCW you were left with a blank screen (see figure 5) which offered no direction or solution. Figure 5: Blank certificate The Microsoft Office 365 Exchange Hybrid Configuration Wizard experience will not remove the certificate requirements, but it will help you solve the issue. The HCW will now show you a list of certificates that meet the requirements, and it will show you the servers that do not have a proper certificate installed (see figure 6). This will allow you to either remove those servers from the HCW receive and send connector pages, or you can properly install the certificate on those servers. Figure 6: Better certificate error A more efficient Hybrid experience One of the things we tried to do with the HCW is ensure that we are performing the various configurations in the most efficient way possible (this is our on-going green effort). A good example of an inefficient task that the HCW previously performed was the Mailbox Replication Service (MRS) enablement process. In the HCW logs collected from the troubleshooter, we could see that this cmdlet was often taking an extremely long time to complete. What we do now, is enable the Migration endpoint on the servers in your environment so that you can start moving mailboxes when the HCW is complete without having to enable the endpoint. One of the cmdlets that we used in the previous version of HCW was get-WebServicesVirtualDirectory. In a larger often geographically dispersed environment this cmdlet could take over eight hours to run. In many cases you would end up getting the following error: ERROR: Updating hybrid configuration failed with error 'Subtask Configure execution failed: Configuring organization relationship settings. Execution of the Set-WebServicesVirtualDirectory cmdlet had thrown an exception. This may indicate invalid parameters in your Hybrid Configuration settings. Unable to access the configuration system on the remote server. Make sure that the remote server allows remote configuration Solution: We have resolved this issue in the new HCW using the -ADPropertiesOnly option with Get-WebServicesVirtualDirectory. This changes things so the HCW reads the MRS settings using a local directory call instead of waiting for a response from every server in the environment. This change along with a few others in this area, makes the process take around 15 minutes instead of 8 hours (your deployment times will vary) in these large environments. This is just one example of the type of cleanups we did in the HCW to improve the reliability and speed of the configuration tasks. Autodiscover issues in HCW The single most common failure point for the HCW is the inability to retrieve the Federation Information via the Autodiscover call initiated by the Get-FederationInformation cmdlet. The output of this cmdlet is needed in order to create the Organization Relationships so you can do things like free busy sharing. This accounts for nearly 30% of all HCW failures based on the logs collected from the troubleshooter (are you starting to see the importance of these log files?). When looking at the issue there are certain things the wizard cannot directly address. For instance, at times the issues are related to an improperly configured firewall, or someone doesn’t have a third-party certificate for IIS on the Exchange servers. However, a good portion of you have had things configured correctly and still we failed to complete the Get-FederationInformationcmdlet. One of the things this cmdlet does is use DNS settings from the server you are connected to in order to resolve the Autodiscover endpoint and retrieve the federation information. Many customers do not have a DNS record created for Autodiscover internally since there is often no need for this. The internal Outlook client will use the Service Connection Point to find the Autodiscover endpoint so there is no need for this from an outlook standpoint, however the Get-FederationInformation cmdlet does not use the Service Connection Point. Therefore, if there is no forwarding configured for this zone in DNS the Get-FederationInformationcmdlet will be unable to resolve the autodiscover endpoint and the HCW will fail. Solution: To resolve this issue, we have added a new method for checking for the federation information. We still try to use local DNS first and if it fails we then will try to hit an external service to see if we can get the federation information externally. This will ensure that if you have Autodiscover published properly externally the HCW will complete as expected. See figure 7 for details: Figure 7: Get-FedInfo OAuth Integration Another common failure point is the OAuth portion of the HCW. The HCW today shows you an option to configure OAuth if you are Exchange 2013 native, but not if you coexist with previous versions of the Exchange. OAuth is required for some featurestoday, such as cross premises discovery and automatic archive retention. Because of that, we want to ensure that OAuth is by default configured so all of the Hybrid features work when you complete the HCW. One downside to this is that the current OAuth configuration experience previously had a high rate of failure. We have gone through and fixed a good portion of the experience and we have also added logic to the new HCW so that if the OAuth portion fails we will disable the OAUTH configuration by disabling the IntraOrganizationConnector and let you know we disabled it and give you remediation steps. This will ensure that a failed OAuth configuration does not prevent other hybrid features such as cross premises Free Busy from working. Many more… The above are just a few of the issues that have been addressed with the latest version of the HCW. There are many example that we could have used such as a couple of issues we addressed with mail flow, Multi-Forest deployments, and many more. In this latest version we strived for feature parity, while improvement the failure rate, and allowing for future innovation. We think we have hit the mark. Running the HCW Now that we have covered some of the new features and benefits of running the Microsoft Office 365 Exchange Hybrid Configuration Wizard, let’s take a guided tour. We are not going to go through each option in depth as most of them have not changed from Exchange 2013. How to find the new HCW We have not moved the location of the HCW in the Exchange Admin Center, the entry point look and feel is consistent with previous version of the Exchange 2013 HCW. The only difference is that instead of calling local code when you click “configure” or “modify” in the hybrid node of EAC, we now initiate the click once application. Figure 8 shows the entry point. Figure 8: Entry Point HCW Landing Page The next screen you will see is the HCW landing page, which is a page that serves two purposes. The first and most important purpose is that we can redirect a small subset of customers (based on pre-defined criteria) to an alternate HCW experience. As discussed previously in this blog, this allows us to pilot new features without affecting the production HCW experience. The second benefit of this landing page is that it allows us to provide a proper error message if the browser version, popup blockers, etc. are not configured in a way that would support the HCW. When you are on the landing page you will select the “click here” option to download the HCW. See figure 9 for a view of the landing page. Figure 9: Landing page Welcome Screen The Welcome screen (see figure 10) will provide you with a link that will inform you about what a Hybrid configuration is along with an additional link at the bottom that explains what the HCW application is going to do. The Second link is at the bottom-left of the screen and says What does this application do?On this screen you will simply click next to continue. Figure 10: Welcome screen Server Detection Page The next screen allows you to choose which server you will use to perform your hybrid configuration. This is the machine that the HCW will remote PowerShell into in order to perform all of the hybrid configuration tasks. The selected server must be running a version of Exchange that is within two releases of our currently released Cumulative update. This means at launch the new HCW will work if you are connecting to an Exchange 2013 CU8 or newer version of Exchange. However, when Exchange 2013 CU11 releases you will see that we will no longer allow you to run the new HCW from Exchange 2013 CU8 and will require a minimum of CU9. Keep in mind that even though the HCW will allow you to proceed if you are two versions older than the current release (n-2), we actually only support going one version back for Hybrid (n-1). If for you were to select a server that is running an unsupported version, the HCW will provide you with an error stating that you are not running a supported version. In addition, the HCW will provide you with a list of servers that are running a supported version (if any exist). Figure 11: Unsupported version The HCW will try to select the best server to perform the configuration tasks from using the following logic: First we look to see if the server we are on is running the latest supported version of Exchange in the organization. Next we look to see if there is an existing Exchange server in the site running the latest supported version of Exchange. Finally, we attempt to connect to an out of site Exchange server (typically in a different geographical location) running the latest supported version. If you do not like the server selection the HCW made via the above mentioned detection logic you can manually specify the server name that you want to connect to. You can use the short name (ServerName) or the long name (ServerName.Contoso.com) in the provided box to select the appropriate server running the supported version of Exchange. The last option on this page allows you to select the tenant location. For most the tenant location is simply “Microsoft Office 365” but if your Office 365 is operated by 21 Vianet, you can also use the “21 Vianet” option. Figure 12: Server detection Credentials page The main improvement on this page is the fact that we do not force you to type in your on-premises credentials. However, if you are not signed in as the user with the Organization Management Role you can manually override this behavior and provide separate credentials. Figure 13: Credentials page Connection Status page We will then show you the connection status window, which will let you know if improper credentials were provided on the previous step. Usually this is a pretty uneventful window and you just click next. Figure 14: Connection status Mail flow options page The rest of the questions in the HCW from this page on are related to the mail flow options. The experience and windows you see from this point forward may vary depending on the options selected. For more information on the mail flow options you have please review this article. Figure 15: Mail flow options Receive and Send Connector Configuration This page of the wizard allows you to select the Exchange 2013 and/or Exchange 2016 servers that you intend on configuring for sending and receiving mail for your on-premises environment. You can have a mix of 2013 and 2016 servers selected. We do not allow you to choose Exchange 2010 servers from these menus. Figure 16: Receive Connector Figure 17: Send Connector Certificate selection page We described the enhancements to this certificate selection page previously in the blog, we covered the experience you will get if a valid certificate cannot be found on any one of the Sending and Receiving servers selected on the previous page (figure 16 and figure 17). This certificate page is what you should expect to see when the certificates are installed properly on all servers. In this case you will get a list of certificates that are meeting all of the requirements and installed on all of the selected servers. In most cases the list includes only one certificate that meets the list of requirements. Figure 18: Certificate FQDN for Mail Flow The final question in the wizard will allow the HCW to properly configure the smart host settings on the outbound connector in Exchange Online. You will usually provide the FQDN that matches your MX record in this window. Figure 19: FQDN Update page Up to this point in the HCW there has been no modification made to your on-premises or Exchange Online environment. When you select the update option on this page we will start making the modification based on the answers to the questions you provided on the previous screen. Similar to the old version of the HCW we will store those answer in the local Active Directory in a configuration object known as your desired state. We will then read from that configuration object to make the modification. Figure 20: Update Wrapping this up The Exchange hybrid configuration process is something that has evolved rapidly over the past few years. We have done a lot over that time to simplify these complex configurations. With this latest version we have continued that trend by adding flexibility for innovation, more HCW stability, better HCW performance, a cleaner configuration experience, and (if needed) a proper error experience. However, our tools and services are built for you so let us know what you think, when you try out the wizard send us feedback through the feedback widget in the HCW. Just look for the “give feedback” link on the bottom of the page in the wizard and please rate the experience. The Exchange Hybrid TeamOffice 365 Hybrid Configuration wizard for Exchange 2010
The Office 365 Hybrid Configuration wizard has been updated to support Exchange 2010. This new wizard comes with the following advantages: An updated user experience that simplifies the hybrid configuration process The error handling experience allows for simple remediation of issues, meaning you can actually read and understand the error Fixes for HCW can happen quickly and are no longer tied to the on-premises product release cycle Inefficient code that caused the HCW to take hours to run has been completely reworked and now you should be in and out in minutes Many more enhancements explained in our previous blog post Please stop using the old HCW for Exchange 2010 that was bundled in the Exchange 2010 Exchange Management Console and instead use the Office 365 Hybrid Configuration wizard. Video Walkthrough of the new experience The following is a short video that walks you through the new Office 365 Hybrid Configuration wizard experience for Exchange 2010: How to run the HCW in your Exchange 2010 environment In order to run the new Office 365 Hybrid Configuration wizard, you need to change one behavior. Instead of going to the on-premises Exchange Management Console in Exchange 2010, open the Exchange Admin Center in Exchange Online. To do this follow these steps: 1. From a Domain joined machine, go to http://portal.office.com and log in with your tenant administrator credential 2. From the portal landing page select the Admin Icon 3. In the left tree menu of the admin page, expand ADMIN then select Exchange to open the Exchange Admin Center 4. On the Left side of the Exchange Admin Center select the Hybrid node, then select the Configure button to download the wizard Alternatively, you can click on this link: https://aka.ms/HybridWizard to directly download the new wizard instead. NOTE: We do plan on updating the Exchange Management Console in Exchange 2010 SP3 RU13 with the new HCW, but there is no reason to wait for that. Just click on the link to the HCW when you are ready to run it, there is no need to even open the EMC. A few common questions answered: Why did we update the Office 365 Hybrid Configuration wizard for Exchange 2010? The new Office 365 Hybrid Configuration wizard, which was released a few months back for Exchange 2013 and 2016, has allowed us to really understand the Hybrid Configuration experience. We can see if there was a failure or slow experience, what the issue was, and we collect and act on any feedback that is provided. All of this telemetry allows us on the engineering side to prioritize and address the issues that need to be addressed quickly. Since we have included Exchange 2010 support in this wizard, now all hybrid customers will see these benefits. To find out more about the benefits of running the new HCW you can review our previous blog were we introduced the Office 365 Hybrid Configuration wizard. What Update needs to be installed for the new wizard to work against Exchange 2010? All you need is Exchange 2010 service pack 3, we do not check for the existence of any rollups. However, newer rollups will have plenty of code and security fixes, so (while not required for HCW to complete) I would make sure you try to stay current. To see the list of the latest updates for each version of Exchange go here. Can I run the new wizard even th ough I have already run the old HCW in my environment? Yes, the new wizard will run even if the old wizard completed or partially completed in your environment. If there is no reason for you to update your old configuration you do not need to run the wizard now, but the next time you have an update to make you should use this new experience. Is this new hybrid wizard different than the Exchange 2013/2016 hybrid wizard that was announce a few months back? No, this is the same wizard. However, the wizard has been updated to support the unique configurations that are required for Exchange 2010 Hybrid environments. For example, in a 2010 Hybrid environment you need additional Remote Domain configurations for mail flow features. These Exchange 2010 Explicit configurations needed to be added. Is this new wizard considered BETA or test software? No, in fact this is the supported way to configure hybrid moving forward. Conclusion Do you want to shave hours off of your hybrid deployment, have a more stable environment, and want to simplify the hybrid configuration experience? Stop using the old Hybrid Configuration wizards and use the new Office 365 Hybrid Configuration wizard instead. If you have an issue or any feedback on the wizard, provide it, there is a feedback button on every page in the wizard and we are eager to read and act on it. Spread the word to anyone that you know who has run or is getting ready to run the Hybrid Configuration wizard. Office 365 Hybrid TeamNeed help converting your LDAP filters to OPATH?
After installing Exchange 2007 into your existing Exchange organization, the address lists and recipient policies must have OPATH filters specified in order to administer them from the Exchange 2007 tools. As discussed in the earlier blog post, OPATH is the basis for the filtering syntax used by PowerShell, and is therefore the filtering syntax used by Exchange Server 2007. Reading up on OPATH syntax can be a considerable time sink for an administrator who just wants to get his policies upgraded. I've written a PowerShell script that will perform these conversions for you, allowing you to save the OPATH documentation for the next time you're having trouble getting to sleep. To use the script, just drop it somewhere on the Exchange 2007 server, change the extension to ps1, and change to that folder at the PowerShell prompt. The top of the script shows some various syntax examples, ranging from the simple conversion of a manually entered filter to the automatic upgrading of every existing legacy filter. Of course, before you just automatically upgrade every filter and call it a day, you should consider testing the script in your lab and saving your existing filters, just in case. One of the syntax examples shows how to write out the name, legacy LDAP filter, and suggested OPATH filter for each legacy object to a tab-delimited file, which you could then open in Excel for viewing. This is one of several ways you could save out your old filters before upgrading. Some notes about the script: All syntax examples assume you've changed into the folder where the script is, and that it's not in the path. If you drop the script in Exchange Server\bin you can eliminate the .\ that precedes the script name in the examples. Some LDAP attributes are not available in OPATH. If the script encounters such an attribute in one of your filters it will report, "Could not convert LDAP attribute 'blah' to OPATH", and will fail out. The script does very direct conversions from LDAP to OPATH. For instance, it typically will not use the 'RecipientType' property in OPATH since there is no LDAP equivalent. There is one exception where it looks for a specific string and produces 'RecipientType -eq UserMailbox' in response. The script looks for filters matching the default address list filters and, if it finds one, it skips the normal conversion routine and produces the filter suggested in Evan's earlier post (http://msexchangeteam.com/archive/2007/01/11/432158.aspx). Hopefully this will save you some time during your Exchange 2007 upgrade! The link to the script is below (ConvertFrom-LdapFilter.txt - download and rename to .ps1 to run): LDAP to OPATH filter conversion script - attached to this blog post. - Bill Long
