Troubleshooting Outlook RPC dialog boxes - revisited

[The_Exchange_Team]

It's been a while since we last blogged about this Outlook feature that most users feel is an error, the RPC Dialog Box. You might get a popup error that will say:

Outlook is retrieving data from the Microsoft Exchange Server <Server name>. You can cancel the request or minimize this message to the Windows taskbar until Outlook closes the message automatically.

There are many potential causes or combinations of areas that need to be addressed to properly diagnose and correct this issue. First, please keep in mind that the RPC Dialog Box is something that works off a timer. We have 5 to 8 seconds (depending upon Outlook version, 5 seconds in Outlook 2002 and 8 seconds in Outlook 2003 and 2007) to get an RPC request on the network, to the server, the server to respond, get its response on the network and the client to receive. With that being said, I call this a 10,000 foot problem. There is not necessarily a specific event generated within the application that you can just point your finger to. The idea behind troubleshooting these types of issues is you need to build a funnel to rule out what is not causing them. It's basically a process of elimination. The troubleshooting steps that I have outlined below are for what I would call a "remote client". This does not necessarily mean remote clients are connected via a VPN, it just means that you may be in another building, campus location, or on a different network segment. The affected clients are not "local" to the Exchange server. Typically if the RPC Dialog Box affected all users in your environment, I would also be looking a data from the server, along with the network and the client configuration. In troubleshooting a remote client scenario, it has already been pre-determined that "local" users are not impacted, and it is only users at a separate or external location. I can then narrow the 10,000 foot problem to let's say 5,000 feet. First I want to know about connectivity and the topology itself. There are some simple utilities that I run to acquire the information needed: 1. From the Outlook client workstation use Tracert to determine how many hops from the impacted Outlook Client to the Exchange Server? What devices do the IP Addresses map to? 258495 XCLN: Troubleshooting Client Connectivity Issues Using Command Line Utilities http://support.microsoft.com/default.aspx?scid=kb;EN-US;258495 I want to know what each networking device is relative to the IP's that are returned. Are any of these devices firewalls and do they have a "Session time out" setting similar to our value for TCPKeepAliveTime? Our defaults are 2 hours as documented in: 324270 How to harden the TCP/IP stack against denial of service attacks in Windows Server 2003 http://support.microsoft.com/default.aspx?scid=kb;EN-US;324270 2. From the Outlook client workstation test for a Black Hole Router by issuing a ping command with a packet size: Ping <exchange server_name> or <Exchange server ip address> -f -l 1472 By "pinging" with a packet size, I'm looking to see if there are any segments that have smaller MTU sizes configured. The return value if this is occurring would come back as:

Packet needs to be fragmented but DF set

You can find more information in the Knowledge Base: 314825 How to Troubleshoot Black Hole Router Issues http://support.microsoft.com/default.aspx?scid=kb;EN-US;314825 3. Does your Network include a WAN Accelerator? Can you bypass the device, or (depending on vendor) either exclude MAPI or RPC traffic? WAN Accelerators generally come in 2 flavors: Compress the compressed, or pattern matching. RPC is already compressed and the re-compressed data does not necessarily have any large performance gains based on independent testing. You can read more on the web on independent testing that has been done. I've also seen products that that will keep Outlook sessions open for users. If we exceed limits on sessions that are set in Exchange automatically, you will see event IDs 9646 on the server. These events are another subject that I'll cover in a later blog post to follow. 4. Are the remote users using Outlook in Cached Exchange Mode? If not, can we place the user in Cached Mode as a troubleshooting step? Yes, this will download to ost the first time and may take a few minutes however this sort of scenario is what cached mode is designed for. 5. Network Card. Configurations and settings need to be confirmed. These steps are to be taken on the affected workstation. (Note: these instructions are for how to obtain the settings in Windows XP, but these settings should be checked for any OS):

• Are there any External DNS Servers listed in the Primary or Secondary DNS Entries?
• Is your Local Area Connection at the top of the binding order? Start > Control Panel > Network Connections > Advanced > Advanced Settings - Make sure the in the connections tab the Local Area Connection is on top. Click OK.
• What card is installed and what driver? Local Area Connection > Right Click > Properties > Select the Configure Button > Select the Driver Tab. What is the Date and Version?
• Now Select the Advanced tab > In the Property Window, is TCP Checksum Offloading enabled? Please see screen shot below. In this instance the value for Checksum Offloading is not set. It can be listed as any of the following:

RX Checksum Offloading TX Checksum Offloading TCP Checksum Offloading Alternatively, you can also disable this in the registry: 904946 You experience intermittent communication failure between computers that are running Windows XP or Windows Server 2003 http://support.microsoft.com/default.aspx?scid=kb;EN-US;904946

Since a good portion of Outlook deployments are done via imaged builds, I check the following bindings and protocol registry keys to confirm they are correct: 6. Check the RPC Binding order on the Outlook Client: 163576 XGEN: Changing the RPC Binding Order http://support.microsoft.com/default.aspx?scid=kb;EN-US;163576 Here is what your registry key should look like: 7. Also check the Client Protocols registry keys on the Outlook Client: 321652 You receive error message and are continuously prompted for your Windows NT domain credentials in Outlook 2000, Outlook 2002, or Outlook 2007 http://support.microsoft.com/default.aspx?scid=kb;EN-US;321652 There a few of Outlook Features that are enabled by Default that could also make a difference when talking about RPC Dialog Popups being experienced depending on your environment. If these features are not being utilized, you should consider disabling as we have documented that they cause the RPC popups: 9. Try disabling IM Integration (Outlook 2002) / Smart Person Tags (Outlook 2003) in Outlook: 290025 A delay occurs when you open a message in Outlook 2002 with Instant Messaging enabled http://support.microsoft.com/default.aspx?scid=kb;EN-US;290025 10. Try turning off the Planner Options Feature: 324374 OL2002: You Receive the "Cancelable RPC" Dialog Box When You Use the Meeting Planner Options http://support.microsoft.com/default.aspx?scid=kb;EN-US;324374 11. How often is the Outlook client publishing Free/Busy information to the Exchange Server? In Outlook 2003 and 2007 the default is 45 minutes. In earlier client versions the default value is 15 minutes. Confirm or set the Free Busy Publishing Interval to 45 minutes as outlined here: 819108 Settings for minimizing periodic WAN traffic http://support.microsoft.com/default.aspx?scid=kb;EN-US;819108 12. Is the user accessing multiple Mailboxes or Calendars through their profile? I've seen a user that was loading 17 mailboxes within her profile since see needed to access Calendars for these 17 people. When we took a total number of items just in those 17 Inboxes, Sent Items, Deleted Items, and Calendars, we were looking at close to 1,000,000 items. 13. Look at a System Information file (saved as the default .nfo) while Outlook is open. Look through the Loaded Modules for things like multiple firewalls or other software that you might not expect. 300887 How to Use System Information (MSINFO32) Switches http://support.microsoft.com/default.aspx?scid=kb;EN-US;300887 The important question is "Do users have the ability to install their own software?" You may see items such as multiple Firewalls are all running at the same time. You may also see music or video file sharing services that actually do network broadcasts. 14. Is there a Host file and does it contain any External DNS servers? If the host name is correctly entered in a host file, it will be resolved before the client attempts to query DNS. So you will not be resolving to your own DNS servers. Rename the Host file and attempt your connection again. 15. Take a look at the Add-Ins or Comm Add-Ins that are loaded in Outlook. There are new third party and free utilities that come out all the time. Again, if users have the ability to install their own software, starting Outlook in safe mode on affected user workstations may help you identify the problem. Go to Start > Run > Outlook.exe /safe. Can you re-produce the RPC Dialog Box now? 16. Are any PST files stored on a network share? 297019 Personal folder files are unsupported over a LAN or over a WAN link http://support.microsoft.com/default.aspx?scid=kb;EN-US;297019 17. How many Items are in the Outlook Folders? What happens if subfolders are created and we keep the Inbox, Sent Items, and Deleted Items with less than 3500 items per folder? 905803 Outlook users experience poor performance when they work with a folder that contains many items on a server that is running Exchange Server http://support.microsoft.com/default.aspx?scid=kb;EN-US;905803 If you feel that the number of items within a folder is close to or beyond the recommended 3500 to 5000 items within a single folder, we have blog including step by step screen shots on how to use PFDavAdmin to extract this information found here: http://msexchangeteam.com/archive/2007/04/24/438170.aspx 18. If running Windows 2003 SP2 on the server (or workstation for that matter), or Windows 2003 SP1 with the Scalable Networking Pack, try disabling the TCP Chimney feature as follows: Open a command prompt and type the following: Netsh int ip set chimney DISABLED See the WebLog; Windows 2003 Scalable Networking pack and its possible effects on Exchange http://msexchangeteam.com/archive/2007/07/18/446400.aspx How this helps in your troubleshooting these type issues. Stay tuned for more... - Nicholas Basile

[#2]

Hi. Very good stuff (!!), but it looks like the most important #1 step isn't mentioned, I missed it, or our friendly local Exchange PFEs gave us some incorrect info. :)

Does the error say...

Outlook is retrieving data from the Microsoft Exchange Server <ServerHostName>. You can cancel the request or minimize this message to the Windows taskbar until Outlook closes the message automatically.

...Or...

Outlook is retrieving data from the Microsoft Exchange Server <ServerHostname.f.q.d.n>. You can cancel the request or minimize this message to the Windows taskbar until Outlook closes the message automatically.


Doesn't this usually mean the difference between you troubleshooting an issue with Outlook connectivity to the Exchange server versus Outlook connectivity to a Global Catalog?

Thank you.

[#3]

The difference in whehter an FQDN or NetBios name returned in the popup does not neccessarily differentiate how to troubleshoot.  The larger factor is your clients, locations, and actions being taken.  This post was designed to provide troubleshooting steps and actions based on "remote" clients or clients not local to the Exchange Server.  So let's say we have a location that is affected, and users local do not.  Isolation of problem and creating a funnel to rule out what it's not is the key point here

[#4]

Hi Nicholas,

is there a way to get directly in touch with you (email, IM, phone?) regarding this topic?

Thanks,
Abdelkader

[#5]

I would suggest that you open a case with Product Support and reference this blog post.

[#6]

Thanks. That will be my first task for tomorrow :-)

[#7]

Does anybody have any tips in troubleshooting why the remote outlook clients will not contact a local GC of the site, based on the post in how outlook mapi clients does the point system after SP2 upgraded on Exchange 2003

http://msexchangeteam.com/archive/2006/03/17/422350.aspx

We are seeing issues when users are looking at the GAL it brings up rpc dilog of GC that is at the Exchange site which is different for the remote users.

Any help or suggestions would be great? Thanks in advance

[#8]

Wondering if the issue may be on the Exchange server side.  In Exchange 2007, is there a limit to the number of RPC connections the Exchange server can support?

[#9]

Thanks ,
while checking the 3500- 5000 critical message in the outlook clients
like Inbox,send item,calender and task

but if you have more than 5000 message in the inbox we can create a new folder next to the inbox and move the mails to that folder

will this reduce the RPC call
because the max no of RPC call can be 100

suggestion