There are a number of ways an end-user may learn that they have new voice mail, such as from client software like Microsoft Outlook, OWA, Lync, Communicator, or different types of indicators on IP, smart and traditional phones. The term Message Waiting Indicator (MWI) is usually applied to notifications delivered by the user’s phone using mechanisms such as a light on the phone, a special icon or highlighted notification or even a special dial tone. Exchange 2010 includes built-in support for MWI.
For more details, see Understanding Message Waiting Indicator in Exchange 2010 documentation.
This article describes steps on how to use the event logs on Exchange 2010 Unified Messaging (UM) and Mailbox servers to troubleshoot MWI failures. Also included in this article are settings to validate MWI configuration on Exchange.
Flow of MWI notifications on Exchange servers
- A voice message is left for a user and is stored in the user's Exchange mailbox on a Mailbox server.
- The Mailbox Assistant service on the Mailbox server checks the status of voice mails in the user’s mailbox.
- The Mailbox Assistant service makes an RPC connection to a Unified Messaging server associated with the user’s UM Dial Plan.
- The UM server sends a SIP NOTIFY message with the MWI state change to a UM IP Gateway associated with the user’s UM Dial Plan.
The starting point of a MWI notification is after the voicemail has been placed in the end user’s mailbox. The focus of these troubleshooting steps are on the Exchange messaging side to diagnose MWI notification issues between a Mailbox server and a UM Server, and also between the UM server and a UM IP Gateway.
Contents
- Troubleshooting Steps:
- New Unified Messaging server event ID for SIP OPTION failures
- Summary of settings to validate
- Useful links
Troubleshooting Steps
-
Check the event log on the mailbox server
Confirming the Mailbox server is not logging MWI errors will help to isolate the problem to a later notification handling step. The first step in sending the MWI notification is the retrieval of the MWI state from the user’s mailbox on the mailbox database. Once the state of the MWI is known, the mailbox server must be able to make an RPC connection with the UM server to relay the MWI state information. Any of the following events, if found on the Mailbox server, would be the starting point for a deeper look at the Mailbox server. Also UM event logging can be increased on the mailbox server to confirm that the MWI notification was successfully sent to the UM server.
-
On the Exchange 2010 Mailbox server, look for Event ID 1346 in the Application Event log. It indicates there may be an issue with the database or the database might be dismounted.
Log Name: Application
Event ID: 1346
Source: MSExchange Unified Messaging
Date: 4/15/2011 6:55:52 PM
Task Category: MWI General
Level: Warning
Keywords: Classic
User: N/A
Computer: Ex2K10MailboxServer.Contoso.com
Description: The Message Waiting Indicator Assistant was unable to query the mailbox database 'MailboxDB01#GUID'. The MWI assistant will try to query the mailbox database again in 720 minutes. If you continue to see this warning, there could be a problem with the mailbox database or the Mailbox server. Additional information: Microsoft.Mapi.MapiExceptionMdbOffline: MapiExceptionMdbOffline: Unable to get mailbox information. (hr=0x80004005, ec=1142) Diagnostic context: Lid: 20354 -
Presence of Event ID 1345 on the Mailbox server indicates there may be a problem with the user’s mailbox. The ‘Additional information’ will contain MAPI error codes and additional information to diagnose the specific issue. Here's an example:
Log Name: Application
Source: MSExchange Unified Messaging
Date: 4/15/2011 6:45:52 PM
Event ID: 1345
Task Category: MWI General
Level: Warning
Keywords: Classic
User: N/A
Computer: Ex2K10MailboxServer.Contoso.com
Description: The Message Waiting Indicator Assistant was unable to obtain the MWI state for mailbox 'John Doe (#ExchangeGuid)' on database 'MailboxDB01#GUID'. You can safely ignore this warning if the mailbox is currently disconnected or inactive. Additional information: (Various MAPI error descriptions). -
Event ID 1360 on the Mailbox server indicates that there might be an RPC communication issue between the Mailbox server and the UM server. A communication issue between the mailbox and the UM server could be due to an issue on either server. For example:
Log Name: Application
Source: MSExchange Unified Messaging
Date: 4/15/2011 7:05:52 PM
Event ID: 1360
Task Category: UMCore
Level: Warning
Keywords: Classic
User: N/A
Computer: Ex2K10MailboxServer.Contoso.com
Description: The Message Waiting Indicator Assistant failed to deliver the MWI notification '1/29 (unread/read)' for the UM-enabled mailbox 'John Doe (#ExchangeGuid)' associated with UM extension '+15555555555'. Until this problem is corrected, the MWI state for this user may be out of sync. Additional information: Server Ex2K10UMServer failed with 1722 Error 0x6ba (The RPC server is unavailable) from SendMwiMessageNote: You may also see:
There are no more targets available to send an MWI message for user John Doe. -
If the UM MWI event logging level is increased on the Mailbox server, the success event for the MWI notification is logged to the Application event log with ID: 1343, which indicates that the Mailbox server has successfully submitted a MWI notification to the UM server.
This command shows the current logging level of MWI on the mailbox server:
Get-EventLogLevel "MSexchange Unified Messaging\MWI General" | FT -auto
The output:
Identity EventLevel
-------- ----------
MSexchange Unified Messaging\MWI General LowestThis command raises the MWI logging level to ‘Expert’ and you will start to see the MWI ‘Success’ events (Event ID: 1343) in the Application event log.
[PS] C:\>Get-EventLogLevel "MSexchange Unified Messaging\MWI General" | Set-EventLogLevel -Level:Expert
You can use the Get-EventLogLevel cmdlet again to verify that the MWI logging level has been raised to Expert level. For more details about diagnostic logging levels in Exchange 2010, see Manage Diagnostic Logging Levels.
The increase in logging show will show that mailbox server has successfully submitted the MWI notification to the UM server. For example:
Log Name: Application
Source: MSExchange Unified Messaging
Date: 4/15/2011 7:15:26 PM
Event ID: 1343
Task Category: MWI General
Level: Information
Keywords: Classic
User: N/A
Computer: Ex2K10MailboxServer.Contoso.com
Description: A Message Waiting Indicator (MWI) notification '1/29 (unread/read)' for the UM-enabled mailbox 'John Doe (#ExchangeGuid)' associated with UM extension 'johndoe@contoso.com' was succesfully sent to 'Ex2K10UMServer'.
-
-
Identify the UM IP Gateway and UM server
From the UM server: If you already know the UM server in use, then check the Application Event log for event 1344 identifying an MWI and the user affected. The PowerShell commands further below will help to validate Exchange configurations settings for the MWI.
Log Name: Application
Source: MSExchange Unified Messaging
Date: 4/15/2011 6:30:52 PM
Event ID: 1344
Task Category: MWI General
Level: Warning
Keywords: Classic
User: N/A
Computer: Ex2K10UMServer.Contoso.com
Description: The Unified Messaging server failed to deliver the MWI notification '1/29 (unread/read)' for the UM-enabled mailbox 'John Doe (#ExchangeGuid)' associated with UM extension 'johndoe@contoso.com'. Until this problem is corrected, the MWI state for this user may be out of sync. Additional information:
An MWI message couldn't be delivered in the allocated time for user John Doe.
Server MyIPGateway1 failed with 0 Unable to establish a connection.
Server MyIPGateway2 failed with 0 Unable to establish a connection.The ‘MyIPGateway#’ indicates one or more Exchange UM IP Gateways objects associated with the dial plan for the user. The UM IP Gateway should also be checked to confirm that the MWI is enabled with the MessageWaitingIndicatorAllowed set to True. See further below for an example.
From the recipient: If you have a specific user you are validating MWI functionality, the steps below are also useful to validate that each portion of the messaging configuration has MWI enabled for the user. The user must be UM enabled, and MWI must be enabled ‘True’ on the UM Mailbox Policy and the UM IP Gateway in Exchange, to be configured to deliver MWI notifications. Please be sure to check the health and status of the user’s mailbox server first.
Identify the UM server and UM IP Gateways that the user would be associated with.
-
First find the UM dial plan linked with the user. The recipients UM dial plan (and UM Mailbox Policy) can be found for a user with Get-UMMailbox. Get-Recipient will provide much of the same data. For example:
Get-UMMailbox johndoe@contoso.com | select UMEnabled, UMDialPlan, UMMailboxPolicy, UMSMSNotificationOption | FL
The output:
UMEnabled : True
UMDialPlan : MyUMDialPlan
UMMailboxPolicy : MyUMMailboxPolicy
UMSMSNotificationOption : VoiceMail
Alternatively, using the Get-Recipient cmdlet retrieves the following data.
[PS] C:\>Get-Recipient johndoe@contoso.com | select UMEnabled, UMMailboxPolicy, UMRecipientDialPlanId | FL
UMEnabled : True
UMMailboxPolicy : MyUMMailboxPolicy
UMRecipientDialPlanId : MyUMDialPlan -
Next, let's look at the UM Mailbox Policy by using the Get-UMMailboxPolicy cmdlet and confirm that the policy is MWI enabled (AllowMessageWaitingIndicator is set to True).
Get-UMMailboxPolicy -Identity MyUMMailboxPolicy | select AllowMessageWaitingIndicator, AllowSMSNotification | FL
AllowMessageWaitingIndicator : True
AllowSMSNotification : True -
The UM servers and IP Gateways associated with the recipient’s dial plan can be found with the Get-UMDialplan cmdlet. For example,
Get-UMDialplan -Identity MyUMDialPlan | select UMServers, UMMailboxPolicies, UMIPGateway | FL
UMServers : {Ex2K10UMServer, Ex2K10UMServer2}
UMMailboxPolicies : {MyUMMailboxPolicy}
UMIPGateway : {MyIPGateway1, MyIPGateway2} -
The UM IP Gateway in Exchange should also be confirmed that the MWI is enabled and set to ‘True.’
Get-UMIPGateway -Identity MyIPGateway1 | FL MessageWaitingIndicatorAllowed
MessageWaitingIndicatorAllowed : True
-
On the UM servers listed in the dial plan look for Application event log ID 1344. This event will show the MWI failure for the user. This will also indicate which UM IP Gateways that may have a communications issue where the SIP Options fail.
-
-
Look for a SIP OPTIONS failure on the Unified Messaging server
Connection issues between the UM server and physical UM IP Gateway device, may appear as SIP OPTIONS failure events on the UM server. With the UM IP Gateway from either the dial plan or directly on the UM server, you can look for Application event log ID 1400 that indicates a SIP OPTIONS failure to the specific Gateway. For example:
Log Name: Application
Source: MSExchange Unified Messaging
Date: 4/15/2011 6:31:28 PM
Event ID: 1400
Task Category: UMCore
Level: Warning
Keywords: Classic
User: N/A
Computer: Ex2K10UMServer.Contoso.com
Description: The following UM IP gateways did not respond as expected to a SIP OPTIONS request.
Transport = TLS, Address = MyIPGateway1.contoso.com, Port = 5060, Response Code = 0, Message = Unable to establish a connection.
Transport = TLS, Address = MyIPGateway2.contoso.com, Port = 5060, Response Code = 0, Message = Unable to establish a connection.Note: The error text will be different depending on the type of failure; you may also see "This operation has timed out.", “Unknown error (0x80131500)”, or as shown “Unable to establish a connection.” The Port may be different from ‘5060’. If mutual TLS has been configured the port is use will be 5061.
-
Increase the UM MWI event logging level on the UM server
To get confirmation that the Exchange UM server successfully submitted the MWI notification to the Gateway device, you will need to increase the logging level for MWI on the UM server. You may see a new Application event ID: 1343. The ID: 1343 is a ‘success’ entry for having notified the Gateway device of the MWI state. If the Exchange UM server successfully submitted the notification, then the next step would be to check the Gateway device’s functionality with the Gateway’s downstream devices.
Run the command below to view the current logging level of MWI on the UM server.
[PS] C:\>Get-EventLogLevel "MSexchange Unified Messaging\MWI General" | FT -auto
Identity EventLevel
-------- ----------
MSexchange Unified Messaging\MWI General LowestTo increase the MWI logging level to see the MWI success events you can run the following command. This will set the logging level to ‘Expert’ and you will start to see the UM MWI ‘Success’ ID: 1343 events in the Application event log.
[PS] C:\>Get-EventLogLevel "MSexchange Unified Messaging\MWI General" | Set-EventLogLevel -Level:Expert
You can now see that the MWI logging level has been increased.
[PS] C:\>Get-EventLogLevel "MSexchange Unified Messaging\MWI General" | FT -auto
Identity EventLevel
-------- ----------
MSexchange Unified Messaging\MWI General ExpertThe increase in the UM server’s MWI logging level show will show that UM server has successfully submitted the MWI notification to the UM IP Gateway with the following event. The failure to notify end users would then be with the Gateway device or downstream functionality.
For example,Log Name: Application
Source: MSExchange Unified Messaging
Date: 4/15/2011 7:15:45 PM
Event ID: 1343
Task Category: MWI General
Level: Information
Keywords: Classic
User: N/A
Computer: Ex2K10UMServer.Contoso.com
Description: A Message Waiting Indicator (MWI) notification '1/29 (unread/read)' for the UM-enabled mailbox 'John Doe (#ExchangeGuid)' associated with UM extension 'johndoe@contoso.com' was succesfully sent to 'MyIPGateway1'. -
Check connectivity between the UM server and the UM IP Gateway
There are a number of factors that impact communications between the UM server and the UM IP Gateway. If TLS (Transport Layer Security) is being used, certificates are a common item to check. When other functionality is working between the UM server and the Gateway, there could be an issue in how the SIP NOTIFY is being handled.
Basic network checks to confirm TCP/IP connectivity.
- If the UM Gateway listed is a FQDN, confirm that the correct IP address is resolved. Validate the address resolution across all DNS servers.
- From the UM server use ‘Ping’ and ‘Tracert’ to confirm the routing path and IP connectivity.
- To validate that the port is listening you can ‘Telnet’ to the port (5060/5061).
New Unified Messaging server event ID for SIP OPTIONS failures
The event ID for the SIP OPTIONS failure has changed between Exchange 2007 and Exchange 2010. Exchange 2010 logs event ID 1400 in the Application event log for SIP OPTIONS failures. The content of event ID 1400 is also different from the SIP OPTIONS failure event ID 1088 logged by Exchange 2007, which logged only one failure per event. Exchange 2010 lists multiple failures per event. The event ID1088 does not alert on the Exchange 2010 UM server.
In Exchange 2007 the Application log event ID 1088 for SIP OPTIONS failure:
Event Type: Warning
Event Source: MSExchange Unified Messaging
Event Category: UMService
Event ID: 1088
Date: 4/15/2011
Time: 6:18:13 PM
User: N/A
Computer: Ex2K7UMServer
Description: The IP gateway or IP-PBX "MyIPGateway1" did not respond to a SIP OPTIONS request from the Unified Messaging server. The error code that was returned is "0" and the error text is ":Unable to establish a connection.". Note: The error text will be different depending on the type of failure, you may also see ":This operation has timed out.".
In Exchange 2010 the new Application log event ID 1400 for SIP OPTIONS failure, for example.
Log Name: Application
Source: MSExchange Unified Messaging
Date: 4/15/2011 6:31:28 PM
Event ID: 1400
Task Category: UMCore
Level: Warning
Keywords: Classic
User: N/A
Computer: Ex2K10UMServer.Contoso.com
Description: The following UM IP gateways did not respond as expected to a SIP OPTIONS request.
Transport = TLS, Address = MyIPGateway1.contoso.com, Port = 5060, Response Code = 0, Message = Unable to establish a connection.
Transport = TLS, Address = MyIPGateway2.contoso.com, Port = 5060, Response Code = 0, Message = Unable to establish a connection.
- "This operation has timed out."
- “Unknown error (0x80131500)”
- “Unable to establish a connection.”
Summary of settings to validate
- Confirm that a voicemail has been placed in the end user’s mailbox.
- Confirm the user’s mailbox and mailbox database are online and responding.
- Look for any MWI failure events on the mailbox server.
- Increase the UM MWI event logging level on the mailbox server to view ‘success’ events.
- Confirm the user is UM enabled (Get-Recipient) and note the mailbox policy and dial plan.
- If the user receives an SMS text notification confirm that ‘UMSMSNotificationOption’ is set to a Voicemail option, and also that the UMMailboxPolicy setting ‘AllowSMSNotification’ is set to ‘True’.
- Confirm that the UM Mailbox Policy is MWI enabled.
- Find the UM server and UM IP Gateway being used from the UM Dial Plan.
- Confirm that the UM IP Gateways is MWI enabled in Exchange.
- Check the health of the UM server.
- Look for SIP OPTIONS failures on the UM server.
- Increase the UM MWI event logging level on the UM server to view ‘success’ events.
- Examine network connectivity between the UM server and the UM IP Gateway.
- Confirm the UM server is able to ping the UM IP Gateway and telnet to the port in use.
- The commands Test-UMConnectivity and Test-ExchangeUMCallFlow may be useful in testing functionality between the UM server and the UM IP Gateway but don’t specifically test MWI notifications.
These troubleshooting steps will help to eliminate any issues on the Exchange server side in passing MWI notifications to end users. With all MWI settings correctly configured, any Exchange side MWI failure should be able to be isolated by the errors in the Application event log. But this only validates the Exchange side of the MWI notifications, there could be an issue on the UM IP Gateway / SIP Peer side or how the SIP NOTIFY of the MWI notifications were handled between the UM server and the UM IP Gateway device. This article doesn’t go into the handling of the SIP NOTIFY between the UM server and a UM IP Gateway. For an in-depth description on the MWI SIP NOTIFY with the UM IP Gateway / SIP Peer, please see 'Understanding Message Waiting Indicator’.
Useful links
- Understanding Message Waiting Indicator -An in-depth description of the MWI.
- Configure the TCP Listening Port on a UM IP Gateway -Settings to use TLS (Transport Layer Security) with gateways.
- Error and Event Reference for Unified Messaging Servers -UM events and their description text.
- Understanding Unified Messaging IP Gateways -Description of IP Gateways and objects.
- An IP gateway or IP PBX didn't respond to a SIP OPTIONS request from the Unified Messaging server -Description of the event ID 1400 for the System Center Operations Manager Exchange Server 2010 Management Pack.
- MSExchange Unified Messaging 1400 -Description of the event ID 1400.