To download the script, please see attachment to this post.
EDIT 12/30/2016: This post has been updated for the new 2.0 version of the script.
Greetings Exchange Community! My name is Dan Sheehan, and I work as a Premier Field Engineer for Microsoft, specializing in Microsoft Exchange. Today I present to you the Generate Message Profile script which assists Exchange administrators/service owners with generating an Exchange “user message profile”. This message profile is a critical part of the information entered into the Exchange Server Role Requirements Calculator and the Exchange Client Network Bandwidth Calculator(more on those below). The script, which is published here on the TechNet Gallery, is designed to work in environments of all sizes, and has been tested in environments with hundreds of Exchange sites and servers. The current version works with the Management Shell of Exchange 2010 through Exchange 2016, and I am looking to create a one-off version for Exchange 2007. Without any further ado, on to the script.
An Exchange “user message profile” represents the amount of messages a user sends and receives in a day, and the average size of those messages. This critical information is used by the Role Requirements Calculator to determine the typical workload a group of users will place on an Exchange system, which in turn is used to properly size a new Exchange environment design. This information is also used by the Client Bandwidth Calculator to estimate potential bandwidth impact email users will have on the network, depending on their client type and versions used. Some Exchange service owners “guesstimate” a couple of different user message profiles based on the anticipated workload, while others use data from their existing environment to try and create a messaging profile based on recent user activity. Gathering the necessary information based on recent user activity and creating a user message profile is not an easy task, and quite often service owners turn to third-party tools for assistance with this process. This PowerShell script was created to assist Exchange service owners who want to generate average user message profiles based upon their current environment, but don’t have or want to use a third-party tool to gather the necessary information and generate a message profile. There are other messaging statistics gathering scripts published on the Internet, such as this this one by Mjolinor on the TechNet Gallery and this one by our own Neil Johnson (who BTW is responsible for the Client Bandwidth Calculator). Typically, those types of “messagestats” scripts create a per-user report of all messaging activity which takes a long time, includes information beyond what is required to create a user message profile, and the output requires further manipulation to come up with an average user message profile. The Generate Message Profile script on the other hand focuses on only gathering the messages sent and received by users, which is faster than gathering all messaging activity, and provides a user message profile per Exchange (AD) site versus individual user results.
The script uses native Exchange PowerShell cmdlets to extract the mailbox count from mailbox role servers and mailbox messaging activity from the Hub Transport role server message tracking logs for the specified date range. The information is then processed to obtain a per-site message profiles consisting of averages for sent messages, received messages, and message sizes. The script requires a start and end date, and can be run multiple times to accumulate groups/blocks of days into the final output. For instance, instead of gathering 30 straight days of data from the Exchange servers, which includes weekend days that generally negatively skew the averages due to reduced user load, the script can be run 4 consecutive times using the 4 groupings of weekdays within that 30 day period to help keep the averages reflective of a typical work day. The output to a CSV file can then be performed during the collection the 4th and final week, because the collection of message profiles are additive until the message profiles are exported to a CSV. The script can be run against Exchange servers in specific AD sites, collections of AD sites, or (the default) all AD sites, and the generated message profiles that are returned are organized by AD site name. The ability to specify a specific collection of AD sites is important for multi-site international Exchange deployments because not every location around the world follows a Monday through Friday work week. This selective site functionality can be combined with the script’s ability to accumulate and combine data from multiple data collections into a single report, even if some sites had to be queried using different date ranges. The script can optionally provide a “total” summary message profile for users across all collected sites using the site name of “~All Sites” (which will show up at the top of the output). The collected data can be exported to a CSV file at the end of each script run, otherwise it will be automatically stored as the PowerShell Console variable $MessageProfile for further manipulation or data collections. The script provides detailed output to the screen, including tiered progress bars indicating what site is currently being processed, how many server jobs in that site are being processed, and the running server processing results for that site. The script output also includes an execution time summary at the end so you can plan for timing of future data gathering efforts:
There are several script parameters (covered below) that can be used exclude certain types of mailboxes and messages from the data gathering and subsequent output of the script. For example, if you exclude all Journal data from the data gathering, then journal messages won’t be reflected resulting message profiles. This means the use of the words “all” and “total” below are in reference to the messages and mailboxes the script was told to gather and process, and not necessarily all of the data available on the servers. The data in the output is grouped into the following columns per Exchange site (as well as the optional “~All Sites” entry):
The script has several parameters to allow administrators control what goes into/is excluded from the user message profile generation process. Most of the parameters are grouped into one of three “parameter sets”, with the exception of one parameter that is in 2 sets and a couple that are not in any set. Parameter sets group related parameters together, so once a parameter is one set is chosen the only other available parameters are those in that same set and those that aren’t assigned to any set. Furthermore a required parameter is only required within its parameter set, meaning if you are using one parameter set, then the required parameters in other sets don’t apply. If the concept of parameter sets is a little confusing and you are using Exchange 2013 (andlater), then you can use the PowerShell 3+ cmdlet Show-Command with the script to create a graphical representation of the parameter sets like this:
|ADSites||Gather||Optional||Defaults to "*" which indicates all AD sites with Exchange should be processed. Alternatively, explicit site names, site names with wild cards, or any combination thereof can be used to specify multiple AD sites to filter on. The format for multiple sites is each site name in quotes, separated by a comma with no spaces such as: "Site1","Site2","AltSite*", etc...|
|StartOnDate||Gather||Required||Specifies the date (at 12:00AM) the message tracking log search should start on. The format is MM/DD/YYYY.|
|EndBeforeDate||Gather||Required||Specifies the date (at 12:00AM) the message tracking log search should end before. This means that if the desired search window is Monday through Friday, Saturday needs to be specified so the search "ends before" (stops) at 12:00AM Saturday. This will allow for all of Friday to be included in the search. The format is MM/DD/YYYY.|
|ExcludeHealthData||Gather||Optional||Excludes messages to or from Managed Availability "HealthMailbox" and the older SCOM "extest_" mailboxes, which could artificially inflate the message profile for a site. NOTE: Because the extest and HealthMailboxes can generate a lot of traffic, it is recommended to use this switch to get a more accurate message profile reflection of your users.|
|ExcludeJournalData||Gather||Optional||Excludes journal messages from the data collection. By default messages delivered to journal mailboxes will be included with the message profile, which could artificially inflate the message profile for a site.|
|ExcludePFData||Gather||Optional||Attempts to filter out messages sent to or from legacy Exchange 2007/2010 Public Folder databases. This is not needed if there are no legacy Exchange Public Folder databases. NOTE: This parameter is not recommended because its filter relies on message subject line filtering which could potentially filter out user messages. Additionally, this does not filter out all Public Folder messaging data because some Public Folder message subject lines were not included due to the high likelihood that users would use them in their own messages.|
|ExcludeRoomMailboxes||Gather||Optional||Excludes messages to or from room mailboxes. By default equipment and discovery mailboxes are excluded from the count as they negatively skew the average user message profile. Room mailboxes are included by default because they can send/receive email NOTE: This parameter is not recommended if you have active conference room booking in your environment as that means you have active message traffic to and from room mailboxes.|
|BypassRPCCheck||Gather||Optional||Instructs the script to bypass the additional RPC connectivity test to remote computers through Get-WMIObject. Basic PING tests are always used to initially test connectivity to remote computers. Bypassing the RPC check should not be necessary as long as the account running the script has the appropriate permissions to connect to WMI on the remote computers.|
|MaxServerTries||Gather||Optional||Specifies the maximum number of times to try to gather data from a server when there are issues gathering data. The default value of 3 means the script will try to gather data from each server up to 3 times before giving up on it and marking it as a skipped server.|
|MinServersPercent||Gather||Optional||Specifies the minimum percentage of servers in a site, defaulting to 100%, that must be accessible and also return data to adequately generate a message profile. If this percentage is not met, because too many servers are inaccessible or they exceed the MaxServerTries during data gathering, the site is skipped (recorded as a SkippedSite so the script can be quickly re-run against it) and not included in the final message profile collection. The format is a number value without the “%”. NOTE: It is highly recommended to leave this value at 100, because missing even one server could result in a potentially skewed message profile.|
|MaxThreads||Gather||Optional||Specifies the maximum number of simultaneous server data gathering jobs (threads). Each job increases the memory and CPU load on the server running this script. Therefore, the number of jobs defaults to 1/4 (rounded up) of logical cores if the system running the script is running Exchange services, or 1/2 if it is not. NOTE: Monitor CPU and memory impact and adjust as necessary.|
|Confirm||Gather||Optional||Bypasses the warning prompts for changes to the MinServersPercent and MaxThreads parameters.|
|ExcludeSites||Gather Import||Optional||Specifies which sites should be excluded from data processing. This is useful when you want to use a wild card to gather data from multiple sites, but you want to exclude specific sites that would normally be included in the wild card collection. Likewise, sites that do not house any user mailboxes, such as dedicated Hybrid sites, can be excluded. For data importing, this is useful when a site needs to be excluded from a previous collection. The format for multiple sites is each individual site name in quotes, separated by a comma with no spaces such as: "Site1","Site2", etc.. NOTE: Wild cards are not supported.|
|InCSVFile||Import||Required||Specifies the path and file name of the CSV to import previously collected data from.|
|InMemory||Existing||Required||Instructs the script to only use existing in memory data. This intended only to be used with the AverageAllSites parameter switch.|
|AverageAllSites||<None>||Optional||Instructs the script to create an "~All Sites" entry in the collection that represents an average message profile of all sites collected. If an existing "~All Sites" entry already exists, its data is overwritten with the updated data.|
|OutCSVFile||<None>||Optional||Specifies the path and file name of the CSV to export the collected data to. If this parameter is omitted, then the collected data is saved in the shell variable $MessageProfile. NOTE: Do not use this parameter if you are collecting multiple weeks of data individually, such as successive weeks to avoid weekends, until the last week so only the complete data set exported to a CSV and the $MessaProfile variable is not removed from memory.|
NOTE: This list of parameters will be updated on the TechNet Gallery posting as the script is updated.
The following are just some examples of the script being used:
Generate-MessageProfile.ps1 -StartOnDate 12/1/2014 -EndBeforeDate 12/6/2014 -ExcludeHealthData -OutCSVFile AllSites.CSV
Generate-MessageProfile.ps1 -ADSites East* -StartOnDate 12/1/2014 -EndBeforeDate 12/2/2014 -Verbose -Debug
Generate-MessageProfile.ps1 -ADSites "EastDC1","West*" -StartOnDate 12/1/2014 -EndBeforeDate 12/31/2014 -OutCSVFile MultiSites.CSV -ExcludePFData -ExcludeJournalData
Generate-MessageProfile.ps1 -InCSVFile .\PreviousCollection.CSV
Generate-MessageProfile.ps1 -InMemory -AverageAllSites
1. Why don’t I see any per-user information? Why is this site based?
2. Is the output generated by this script an accurate representation of my users’ messaging profile, which I can use in other tools such as the Role Requirements Calculator?
3. Should I inflate/enhance the message profile produced by this script to give myself some “elbow room” in my Exchange system design?
4. The messaging profile for my users seems lower than I expected. What are some factors that could attribute to this/how can I increase the values generated by the script?
5. Why did I get an alert that one or more sites were skipped or excluded?
6. Why can’t I specify the hours of a day I want to be searched in addition to the days?
7. Why does the EndBeforeDate need to be the day following the day I want to stop reporting on?
8. Why would I want to store data in a CSV file and then later import it with the script?
9. What is the purpose of the InMemory parameter?
10. Why do I get an error about “inconsistent number of days” when I try to use the AverageAllSites?
11. Why is the information saved to the $MessageProfile variable in the console if I don’t use the –OutToCSV parameter? Also how do I “wipe” the collected data from memory so I can start over?
$MessageProfile = $Null 12. Why does the output of the script include a value called “TimeSpan” and also the time zone of the remote site?
13. Why did you build in an ExcludePFData parameter switch if it doesn’t exclude all legacy Public Folder traffic?
14. I see Equipment and Discovery mailboxes are excluded, why aren’t Arbitration Mailboxes excluded?
So there you have it, a PowerShell script to assist you with generating an average user message profile for your environment, with a number of options for you to tailor it to your preferences. I hope you find it useful with the two calculators, but also any future troubleshooting efforts of your existing environment. As I make enhancements or other changes to the script, I will be updating the TechNet Gallery posting. So please check back with that posting periodically. Lastly I am always open to suggestions and ideas, so please feel free to leave a comment here, on the TechNet Gallery submission, or reach out to me directly. Dan Sheehan Senior Premier Field Engineer
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.