Log Parser Studio 2.0 is now available
Since the initial release of Log Parser Studio (LPS) there have been over 30,000 downloads and thousands of customers use the tool on a daily basis. In Exchange support many of our engineers use the tool to solve real world issues every day and in turn share with our customers, empowering them to solve the same issues themselves moving forward. LPS is still an active work in progress; based on both engineer and customer feedback many improvements have been made with multiple features added during the last year. Below is a short list of new features: Improved import/export functionality For those who create their own queries this is a real time-saver. We can now import from multiple XML files simultaneously only choosing the queries we wish to import from multiple query libraries or XML files. Search Query Results The existing feature allowing searching of queries in the library is now context aware meaning if you have a completed query in the query window, the search option searches that query. If you are in the library it searches the library and so on. This allows drilling down into existing query results without having to run a new query if all you want to do is narrow down existing result sets. Input/Output Format Support All LP 2.2 Input and Output formats contain preliminary support in LPS. Each format has its own property window containing all known LP 2.2 settings which can be modified to your liking. Exchange Extensible Logging Support Custom parser support was added for most all Exchange logs. These are covered by the EEL and EELX log formats included in LPS which cover Exchange logs from Exchange 2003 through Exchange 2013. Query Logging I can't tell you how many times myself or another engineer spent lots of time creating the perfect query for a particular issue we were troubleshooting, forgetting to save the query in the heat of the moment and losing all that work. No longer! We now have the capability to log every query that is executed to a text file (Query.log). What makes this so valuable is if you ran it, you can retrieve it. Queries There are now over 170 queries in the library including new sample queries for Exchange 2013. PowerShell Export You can now export any query as a standalone PowerShell script. The only requirement of course is that Log Parser 2.2 is installed on the machine you run it on but LPS is not required. There are some limitations but you can essentially use LPS as a query editor/test bed for PowerShell scripts that run Log Parser queries for you! Query Cancellation The ability to submit a request to cancel a running query has been added which will allow you to cancel a running query in many cases. Keyboard Shortcuts There are now 23 Keyboard shortcuts. Be sure to check these out as they will save you lots of time. To display the short cuts use CTRL+K or Help > Keyboard Shortcuts. There are literally hundreds of improvements and features; far too many to list here so be sure and check out our blog series with existing and upcoming tutorials, deep dives and more. If you are installing LPS for the first time you'll surely want to review the getting started series: Getting started with Log Parser Studio Getting started with Log Parser Studio, part 2 Getting started with Log Parser Studio, part 3 If you are already familiar with LPS and are installing this latest version, you'll want to check out the upgrade blog post here: Log Parser Studio: upgrading from v1 to v2 Additional LPS articles can be found here: http://blogs.technet.com/b/karywa/ LPS doesn't require an install so just extract to the folder of your choice and run LPS.EXE. If you have the previous version of LPS and you have added your own custom queries to the library, be sure to export those queries as a backup before running the newest version. See the "Upgrading to LPS V2" blog post above when upgrading. Kary WallIntroducing: Log Parser Studio
To download the Log Parser Studio, please see the attachment on this blog post. Anyone who regularly uses Log Parser 2.2 knows just how useful and powerful it can be for obtaining valuable information from IIS (Internet Information Server) and other logs. In addition, adding the power of SQL allows explicit searching of gigabytes of logs returning only the data that is needed while filtering out the noise. The only thing missing is a great graphical user interface (GUI) to function as a front-end to Log Parser and a ‘Query Library’ in order to manage all those great queries and scripts that one builds up over time. Log Parser Studio was created to fulfill this need; by allowing those who use Log Parser 2.2 (and even those who don’t due to lack of an interface) to work faster and more efficiently to get to the data they need with less “fiddling” with scripts and folders full of queries. With Log Parser Studio (LPS for short) we can house all of our queries in a central location. We can edit and create new queries in the ‘Query Editor’ and save them for later. We can search for queries using free text search as well as export and import both libraries and queries in different formats allowing for easy collaboration as well as storing multiple types of separate libraries for different protocols. Processing Logs for Exchange Protocols We all know this very well: processing logs for different Exchange protocols is a time consuming task. In the absence of special purpose tools, it becomes a tedious task for an Exchange Administrator to sift thru those logs and process them using Log Parser (or some other tool), if output format is important. You also need expertise in writing those SQL queries. You can also use special purpose scripts that one can find on the web and then analyze the output to make some sense of out of those lengthy logs. Log Parser Studio is mainly designed for quick and easy processing of different logs for Exchange protocols. Once you launch it, you’ll notice tabs for different Exchange protocols, i.e. Microsoft Exchange ActiveSync (MAS), Exchange Web Services (EWS), Outlook Web App (OWA/HTTP) and others. Under those tabs there are tens of SQL queries written for specific purposes (description and other particulars of a query are also available in the main UI), which can be run by just one click! Let’s get into the specifics of some of the cool features of Log Parser Studio … Query Library and Management Upon launching LPS, the first thing you will see is the Query Library preloaded with queries. This is where we manage all of our queries. The library is always available by clicking on the Library tab. You can load a query for review or execution using several methods. The easiest method is to simply select the query in the list and double-click it. Upon doing so the query will auto-open in its own Query tab. The Query Library is home base for queries. All queries maintained by LPS are stored in this library. There are easy controls to quickly locate desired queries & mark them as favorites for quick access later. Library Recovery The initial library that ships with LPS is embedded in the application and created upon install. If you ever delete, corrupt or lose the library you can easily reset back to the original by using the recover library feature (Options | Recover Library). When recovering the library all existing queries will be deleted. If you have custom/modified queries that you do not want to lose, you should export those first, then after recovering the default set of queries, you can merge them back into LPS. Import/Export Depending on your need, the entire library or subsets of the library can be imported and exported either as the default LPS XML format or as SQL queries. For example, if you have a folder full of Log Parser SQL queries, you can import some or all of them into LPS’s library. Usually, the only thing you will need to do after the import is make a few adjustments. All LPS needs is the base SQL query and to swap out the filename references with ‘[LOGFILEPATH]’ and/or ‘[OUTFILEPATH]’ as discussed in detail in the PDF manual included with the tool (you can access it via LPS | Help | Documentation). Queries Remember that a well-written structured query makes all the difference between a successful query that returns the concise information you need vs. a subpar query which taxes your system, returns much more information than you actually need and in some cases crashes the application. The art of creating great SQL/Log Parser queries is outside the scope of this post, however all of the queries included with LPS have been written to achieve the most concise results while returning the fewest records. Knowing what you want and how to get it with the least number of rows returned is the key! Batch Jobs and Multithreading You’ll find that LPS in combination with Log Parser 2.2 is a very powerful tool. However, if all you could do was run a single query at a time and wait for the results, you probably wouldn’t be making near as much progress as you could be. In lieu of this LPS contains both batch jobs and multithreaded queries. A batch job is simply a collection of predefined queries that can all be executed with the press of a single button. From within the Batch Manager you can remove any single or all queries as well as execute them. You can also execute them by clicking the Run Multiple Queries button or the Execute button in the Batch Manager. Upon execution, LPS will prepare and execute each query in the batch. By default LPS will send ALL queries to Log Parser 2.2 as soon as each is prepared. This is where multithreading works in our favor. For example, if we have 50 queries setup as a batch job and execute the job, we’ll have 50 threads in the background all working with Log Parser simultaneously leaving the user free to work with other queries. As each job finishes the results are passed back to the grid or the CSV output based on the query type. Even in this scenario you can continue to work with other queries, search, modify and execute. As each query completes its thread is retired and its resources freed. These threads are managed very efficiently in the background so there should be no issue running multiple queries at once. Now what if we did want the queries in the batch to run concurrently for performance or other reasons? This functionality is already built-into LPS’s options. Just make the change in LPS | Options | Preferences by checking the ‘Process Batch Queries in Sequence’ checkbox. When checked, the first query in the batch is executed and the next query will not begin until the first one is complete. This process will continue until the last query in the batch has been executed. Automation In conjunction with batch jobs, automation allows unattended scheduled automation of batch jobs. For example we can create a scheduled task that will automatically run a chosen batch job which also operates on a separate set of custom folders. This process requires two components, a folder list file (.FLD) and a batch list file (.XML). We create these ahead of time from within LPS. For more details on how to do that, please refer to the manual. Charts Many queries that return data to the Result Grid can be charted using the built-in charting feature. The basic requirements for charts are the same as Log Parser 2.2, i.e. The first column in the grid may be any data type (string, number etc.) The second column must be some type of number (Integer, Double, Decimal), Strings are not allowed Keep the above requirements in mind when creating your own queries so that you will consciously write the query to include a number for column two. To generate a chart click the chart button after a query has completed. For #2 above, even if you forgot to do so, you can drag any numbered column and drop it in the second column after the fact. This way if you have multiple numbered columns, you can simply drag the one that you’re interested in, into second column and generate different charts from the same data. Again, for more details on charting feature, please refer to the manual. Keyboard Shortcuts/Commands There are multiple keyboard shortcuts built-in to LPS. You can view the list anytime while using LPS by clicking LPS | Help | Keyboard Shortcuts. The currently included shortcuts are as follows: Shortcut What it does CTRL+N Start a new query. CTRL+S Save active query in library or query tab depending on which has focus. CTRL+Q Open library window. CTRL+B Add selected query in library to batch. ALT+B Open Batch Manager. CTRL+B Add the selected queries to batch. CTRL+D Duplicates the current active query to a new tab. CTRL+ALT+E Open the error log if one exists. CTRL+E Export current selected query results to CSV. ALT+F Add selected query in library to the favorites list. CTRL+ALT+L Open the raw Library in the first available text editor. CTRL+F5 Reload the Library from disk. F5 Execute active query. F2 Edit name/description of currently selected query in the Library. F3 Display the list of IIS fields. Supported Input and Output types Log Parser 2.2 has the ability to query multiple types of logs. Since LPS is a work in progress, only the most used types are currently available. Additional input and output types will be added when possible in upcoming versions or updates. Supported Input Types Full support for W3SVC/IIS, CSV, HTTP Error and basic support for all built-in Log Parser 2.2 input formats. In addition, some custom written LPS formats such as Microsoft Exchange specific formats that are not available with the default Log Parser 2.2 install. Supported Output Types CSV and TXT are the currently supported output file types. Log Parser Studio - Quick Start Guide Want to skip all the details & just run some queries right now? Start here … The very first thing Log Parser Studio needs to know is where the log files are, and the default location that you would like any queries that export their results as CSV files to be saved. 1. Setup your default CSV output path: a. Go to LPS | Options | Preferences | Default Output Path. b. Browse to and select the folder you would like to use for exported results. c. Click Apply. d. Any queries that export CSV files will now be saved in this folder. NOTE: If you forget to set this path before you start the CSV files will be saved in %AppData%\Microsoft\Log Parser Studio by default but it is recommended that y ou move this to another location. 2. Tell LPS where the log files are by opening the Log File Manager. If you try to run a query before completing this step LPS will prompt and ask you to set the log path. Upon clicking OK on that prompt, you are presented with the Log File Manager. Click Add Folder to add a folder or Add File to add a single or multiple files. When adding a folder you still must select at least one file so LPS will know which type of log we are working with. When doing so, LPS will automatically turn this into a wildcard (*.xxx) Indicating that all matching logs in the folder will be searched. You can easily tell which folder or files are currently being searched by examining the status bar at the bottom-right of Log Parser Studio. To see the full path, roll your mouse over the status bar. NOTE: LPS and Log Parser handle multiple types of logs and objects that can be queried. It is important to remember that the type of log you are querying must match the query you are performing. In other words, when running a query that expects IIS logs, only IIS logs should be selected in the File Manager. Failure to do this (it’s easy to forget) will result errors or unexpected behavior will be returned when running the query. 3. Choose a query from the library and run it: a. Click the Library tab if it isn’t already selected. b. Choose a query in the list and double-click it. This will open the query in its own tab. c. Click the Run Single Query button to execute the query The query execution will begin in the background. Once the query has completed there are two possible outputs targets; the result grid in the top half of the query tab or a CSV file. Some queries return to the grid while other more memory intensive queries are saved to CSV. As a general rule queries that may return very large result sets are probably best served going to a CSV file for further processing in Excel. Once you have the results there are many features for working with those results. For more details, please refer to the manual. Have fun with Log Parser Studio! & always remember – There’s a query for that! Kary Wall Escalation Engineer Microsoft Exchange SupportPermanently Clear Previous Mailbox Info
We are introducing a new parameter that can be called by using the Set-User cmdlet in Exchange Online PowerShell. The feature is focused for customers doing migration of on-premises mailboxes to the cloud and you will be able to use it within three weeks or so (Edit 1/19: we updated this due to slower than expected rollout): Customers who have Hybrid or on-premises environments with AAD Connect / Dir Sync may have faced the following scenario: User Jon@contoso.com has a mailbox on-premises. Jon is represented as a Mail User in the cloud. You are synchronizing the on-premises directory to the cloud in preparation to migrate to Exchange Online. Due to issues with the on-premises sync or due to a configuration problem, the user Jon@contoso.com does not get the ExchangeGUID synchronized from on-premises to the cloud. If the Exchange GUID is missing from the object in the cloud, assigning an Exchange license to Jon@contoso.com will cause Exchange Online to give the user a mailbox, converting the object from a Mail User to a User Mailbox. (Adding the license is a step required for the migration of the mailbox from on-premises to the cloud.) The end result is the user that has 2 mailboxes: one on-premises and one in the cloud. This is not good. Mail flow issues will follow. Those doing these types of migrations will know that the ExchangeGUID value is very important as it helps Exchange Online identify that the user has a mailbox on-premises, and if an Exchange license is assigned in the cloud, a new mailbox should not be created. The immediate fix for this situation is to remove the Exchange License from Jon@contoso.com. This will convert the cloud object for Jon back to a Mail User. Mail flow should be restored at this point. The problem now is that you have an “unclean” cloud object for Jon. This is because Exchange online keeps pointers that indicate that there used to be a mailbox in the cloud for this user: PS C:\WINDOWS\system32> Get-User Jon@contoso.com | Select name,*Recipient* Name PreviousRecipientTypeDetails RecipientType RecipientTypeDetails ---- ---------------------------- ------------- -------------------- Jon UserMailbox MailUser MailUser Re-assigning the license after that will always err on the side of caution and Exchange Online will try to re-connect the (duplicate, temporary) mailbox in the cloud (and mailboxes can be reconnected for 30 days). Therefore Jon’s account in the cloud can’t be licensed in preparation for migration. Up to now, one of the few options to fix this problem was to delete *only in the cloud* Jon’s object and re-sync it from on-premises. This would delete jon@contoso.com from the cloud – but from all workloads, not only Exchange. This is problematic because Jon could have his OneDrive or SharePoint data in the cloud only and deleting his account means that this will be deleted too. If the account is then re-created, Jon and the tenant admin would have to work to recover to his new account all the data he used to have in OneDrive or SharePoint just because Exchange data needed to be “cleaned up”. The new parameter in the user cmdlet will allow tenant admin to clean up Exchange Online Jon’s object without having to delete it. To clean the object, you can run the following command: PS C:\> Set-User Jon@contoso.com -PermanentlyClearPreviousMailboxInfo Confirm Are you sure you want to perform this action? Delete all existing information about user “Jon@contoso.com"?. This operation will clear existing values from Previous home MDB and Previous Mailbox GUID of the user. After deletion, reconnecting to the previous mailbox that existed in the cloud will not be possible and any content it had will be unrecoverable PERMANENTLY. Do you want to continue? Yes Yes to All No No to All [?] Help (default is "Y"): Y Executing this leaves you with a clean object that can be re-licensed without causing the 2-mailbox problem. Now you can on-board Jon’s on-premises mailbox following the usual steps. An alternative – a call to support to do the clean-up for you - is also not needed. Remember, cleaning up the user means that the older associated disconnected (duplicate) cloud mailbox is not recoverable. If you want to keep it or be able to check it’s content, we recommend using Soft Deletion or Inactive Mailboxes to keep the mailbox. Note: This command is expected to be executed when you have an on-premises mailbox and a mailbox in the cloud for the same object due to bad AAD Connect configuration, to clean the object that can be re-licensed. The procedure allows you get out of the dual mailbox state and enable you to re-try on-boarding the mailbox immediately. If you execute this for a user whose mailbox is cloud only after delicensing the user (and intend to later re-license the same user and expect to have a new clean mailbox for the user) - then this will not happen immediately. To avoid potential loss of mailbox data due to unintended/mistaken execution of the command, we retain the mailbox data for 30 days so that you may recover it. If your intention is to clean-up cloud only mailbox then you may hard delete the user account to re-create a clean mailbox. Mario Trigueros SolorioResolving WinRM errors and Exchange 2010 Management tools startup failures
As was discussed in the previous (related) blog post "Troubleshooting Exchange 2010 Management Tools startup issues", in Exchange 2010 the Management tools are dependent on IIS. As was discussed in that blog, we have seen situations where the management tool connection to the target Exchange server can fail, and the error that is returned can be difficult to troubleshoot. This generally (but not always) happens when Exchange 2010 is installed on an IIS server that is already in service, or when changes are made to the IIS server settings post Exchange Install. We have seen that these changes are usually done when the IIS administrator is attempting to "tighten up" IIS security by editing the Default Web Site or PowerShell vdir settings. The situation is further complicated by the fact that some of the errors presented have similar wording; most seem to originate with WinRM (Windows Remote Management), and in some cases different root problems can produce the exact same error message. In other words, depending on how knowledgeable you are with these errors, troubleshooting them is all around... not much fun. I was approached by a good friend of mine and he asked what I thought we could do to make these errors a little easier to troubleshoot. I was studying PowerShell and PowerShell programming at the time (I just happened to be reading Windows PowerShell for Exchange Server 2007 SP1), and I thought that this would be a perfect opportunity to try and apply what I was learning. This is the result. Introducing the Exchange Management Troubleshooter (or EMTshooter for short). What it does: The EMTshooter runs on the local (target) Exchange server and attempts to identify potential problems with management tools connection to it. The troubleshooter runs in 2 stages. First, it will look at the IIS Default Web Site, the PowerShell vdir, and other critical areas, to identify known causes of connection problems. If it identifies a problem with one of the pre-checks it will make a recommendation for resolving the problem. If the pre-checks pass, the troubleshooter will go ahead and try to connect to the server in the exact same way that the management tools would. If that connection attempt still results in a WinRM-style error, the troubleshooter will attempt to compare that error to a list of stored strings that we have taken from the related support cases that we have seen. If a match is found, the troubleshooter will display the known causes of that error in the CMD window. Here is an example of how this might look like: When I was designing the troubleshooter, I could have just written a little error lookup tool that handed over the appropriate content for the error you were getting, but I felt that was not as robust of a solution as I was aiming for (and not much of a learning experience for me). So the tool runs active pre-checks before moving on to the error look-up. The amount of pre-checks it can run depends on the flavor of OS you are running on and the options you have installed on it, such as WMI Compatibility. Basically, I have taken all of the documentation that has been created on these errors to date, and created a tool that will make the information available to you based on the error or problem it detects. Hopefully this will cut down on the amount of time it takes to resolve those problems. Event reporting: When you run the EMTshooter it will log events in the event log. All results that are displayed in the CMD window are also logged in the event log for record keeping. Events are logged to the Microsoft-Exchange-Troubleshooters/Operational event log and are pretty self-explanatory. Things to remember: Depending on your current settings, you may need to adjust the execution policy on your computer to run the troubleshooter, using: Set-ExecutionPolicy RemoteSigned Or Set-ExecutionPolicy Unrestricted Remember to set it back to your normal settings after running the troubleshooter. This version of the troubleshooter needs to run on the Exchange Server that the management tools are failing to connect to. While our final goal is that the troubleshooter will be able to run anywhere the Exchange Management tools are installed, the tool isn't quite there yet. We have seen instances where corruption in the PowerShell vdir or in IIS itself has resulted in errors that seemed to be caused by something else. For instance, we worked on a server that had an error that indicated a problem with the PowerShell vdir network path. But the path was correct. Then we noticed that the PowerShell vdir was missing all its modules, and quite a few other things. Somehow the PowerShell vdir on that Exchange Server had gotten severely... um... modified beyond repair. WinRM was returning the best error it could, and the troubleshooter took that error and listed the causes. None of which solved the problem. So be aware that there are scenarios that even this troubleshooter cannot help at this time. The troubleshooter is still a bit rough around the edges, and we plan to improve and expand its capabilities in the future. We also hope to be able to dig a little deeper into the PowerShell vdir settings as time goes on. Also note that the troubleshooter will NOT make any modification to your IIS configuration without explicitly asking you first. Permissions required: In order to run the troubleshooter, the user will have to have the rights to log on locally to the Exchange server (due to local nature of the troubleshooter at this time) and will need permissions to run Windows PowerShell. Installing the troubleshooter: First, you will need to download the troubleshooter ZIP file, which you can find attached to this post. Installing the EMTshooter is pretty easy. Just drop the 4 files from the ZIP file into 1 folder, rename them to .ps1 and run EMTshooter.ps1 from a normal (and local) PowerShell window. I personally just created a shortcut for it on my desktop with the following properties: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -noexit -command ". 'C:\EMTshooter\EMTshooter.ps1'" However, as most users probably won't run this more than a few times you might not need or want an icon. Just remember that EMTshooter.ps1 is the main script to run. Providing feedback: As I mentioned before, the troubleshooter is still a work in progress. If you wish to provide feedback on it, please post a comment to this blog post. I will be monitoring it and replying as time allows, and also making updates to the troubleshooter if needed. If you run into errors that are not covered by the troubleshooter, please run the troubleshooter, reproduce the error through it and send me the transcript.txt file (you will find it in the folder where the 4 scripts have been placed), along with what you did to resolve the error (if the problem has been resolved). My email is sbryant AT Microsoft DOT com. Errors currently covered: Connecting to remote server failed with the following error message : The WinRM client cannot process the request. It cannot determine the content type of the HTTP response from the destination computer. The content type is absent or invalid. Connecting to remote server failed with the following error message: The connection to the specified remote host was refused. Verify that the WS-Management service is running on the remote host and configured to listen for requests on the correct port and HTTP URL. Connecting to remote server failed with the following error message: The WinRM client received an HTTP server error status (500), but the remote service did not include any other information about the cause of the failure. For more information, see the about_Remote_Troubleshooting Help topic. It was running the command 'Discover-ExchangeServer -UseWIA $true -SuppressError $true'. Connecting to remote server failed with the following error message : The WinRM client received an HTTP status code of 403 from the remote WS-Management service. Connecting to the remote server failed with the following error message: The WinRM client sent a request to an HTTP server and got a response saying the requested HTTP URL was not available. This is usually returned by a HTTP server that does not support the WS-Management protocol. Connecting to remote server failed with the following error message : The client cannot connect to the destination specified in the request. Verify that the service on the destination is running and is accepting requests. Consult the logs and documentation for the WS-Management service running on the destination, most commonly IIS or WinRM. If the destination is the WinRM service, run the following command on the destination to analyze and configure the WinRM service: Connecting to remote server failed with the following error message : The WS-Management service does not support the request. Connecting to remote server failed with the following error message : The WinRM client cannot process the request. The WinRM client tried to use Kerberos authentication mechanism, but the destination computer - Steve Bryant
