Fix available to alleviate event ID 9548
Published Mar 22 2006 11:55 AM 5,168 Views

No doubt many of you are familiar with the event commonly seen in application logs of Exchange 2000 and 2003 servers, MSExchangeIS event 9548, indicating that the information store came across a disabled user who is missing the msExchMasterAccountSid attribute while processing some various task.  There are many KB articles associated with this event, such as 291151, 326990, 278966, 328880, 316047, and possibly more.  There have also been countless support cases where this design was at least a contributing factor.  Almost every application log from an Exchange 2000 or 2003 server ever seen has likely been littered with 9548 events, to the point where it has become an annoyance event.  For additional information on the event and the related information, I suggest reading this article: http://www.msexchange.org/articles/NoMAS-Tool.html.

 

A CDCR (Critical Design Change Request) was accepted last year to resolve the issue without having to run tools or scripts, and the first version of this fix was released last week.  The KB article is 903158.

 

The problem was that a decision was made during the development of Exchange 2000 that every disabled user account with a mailbox had to have the msExchMasterAccountSid attribute.  This is because in order for a mailbox to function within the store, a SID must be associated with the mailbox.  The logic worked like this:

 

1. If the user account associated with the mailbox is disabled, and has msExchMasterAccountSid, AND msExchMasterAccountSid is not the well-known SELF SID, then msExchMasterAccountSid is the SID that is associated with the mailbox.

 

2. If the user account is enabled, or if it is disabled and has msExchMasterAccountSid set to the well-known SELF SID, then use objectSid.

 

3. If the user account is disabled, and has no msExchMasterAccountSid, OR if we cannot tell if the user is enabled or disabled due to access control on the user object, or if we cannot read the objectSid due to access control, fail the operation and log the 9548 event.

 

In the 3rd case, the vast majority of the 9548 events came from the first part - that is, almost all 9548 events are due to disabled user accounts which are missing msExchMasterAccountSid.  The new logic works as follows:

 

1. If the user account associated with the mailbox is disabled, and has msExchMasterAccountSid, AND msExchMasterAccountSid is not the well-known SELF SID, then msExchMasterAccountSid is the SID that is associated with the mailbox.  (NOTE: No change here)

 

2. If the user account is enabled, or if it is disabled and has msExchMasterAccountSid set to the well-known SELF SID, OR if the user is disabled has does not have msExchMasterAccountSid, then use objectSid.  (Big change here)

 

3. If we cannot tell if the user account is enabled or disabled due to access control on the user object, or if we cannot read the objectSid due to access control, fail the operation and log the 9548 event.

 

So now, the only way you will get the 9548 event is due to a real problem with the user account associated with a mailbox.

 

- Alex Seigler

47 Comments
Not applicable
Thank you, thank you, thank you.  I used to have a one-off script to set the attribute for disabled accounts (since we wanted the mailboxes to still function) until NoMAS came along.  Actually, I have always preferred MsAccntSidFixer, and I use it to this day.  Now I will be able to stop doing that once applied.  Thank you so much.

Scott.
Not applicable
I'm sure a lot of you have seen event 9548 in your event logs from time to time.
If 9548 isn't ringing a bell, maybe this example text will:

Disabled user /o= Organization Name /ou= Administrative Group Name /cn=Recipients/cn= Computer Name does not
Not applicable
Well done.

Now, let's examine #2 more closely:

You wrote: If the user account is enabled ...then use objectSid.

No problem with that.

You wrote:
if it is disabled and has msExchMasterAccountSid set to the well-known SELF SID...then use objectSid

My question: Which ObjectSID? The disabled account's SID or SELF?

You wrote:
if the user is disabled has does not have msExchMasterAccountSid...then use objectSid

Again, a question:
Which ObjectSID? The disabled account's SID or SELF?

The last 2 descriptions are not clear (to me) because, in one instance, there is MAS, and in another, there isn't. So, which SID are we using? The way I read KB903158 is that Exchange will always assume that MAS is equal to SELF SID and will ALWAYS use SELF SID even when there is no MAS associated with the disabled account. But your description does not make this clear when you mentioned objectSID.
Not applicable
Forget what I wrote, ladies and gentlemen. The effects of the morning coffee just set in :)
Not applicable
For both questions, the answer is the objectSID of the disabled user account.  When the store sees "SELF", it grabs the objectSID and uses that (because that is what SELF means).

So, for disabled user, if MAS is set to SELF, or if it is not set at all, we use the objectSID of the disabled user.  The only time we would use the SID in MAS as-is is when the attribute is populated and not SELF (like an NT4 SID or a SID from a different forest).

-aseigler
Not applicable
I think I am missing something. I got the hotfix from support. When I run the hotfix it says: "You can install this hotfix on Service Pack 1 only".
I didn't see mention of this limitation anywhere.
Not applicable
Not applicable
Beautiful, Andy.  The KBarticle indicates a pre-SP2 version, though, with a February date.  Will there be a post-SP2 version?
Not applicable
That should read "Alex", of course...
Not applicable
Correct, the build available right now is for SP1 only.  The fix for SP2 servers will be out very shortly.

-aseigler
Not applicable
Thankyou.
This has been a major annoyance for us when running daily backup jobs, as backup exec always complains when it finds a mailbox without the SELF permission thats associated to a disabled user. even just one disabled user and the backup job reports as having failed. Thankyou again for finally fixing this up.
Not applicable
Not applicable
5 years of waiting...finally. Way things are going AD and Exchange might start to be better together after all.
Not applicable
Hmm, have to wait until the PostSP2 version arrives, but since we've waited a long time for this a few more days/weeks don't mind.

I would like a peek on the 'Accepted CDCR' list....
Not applicable
Quick question, if we apply the pre SP2 fix onto a SP1 build is it likely that we will then have to apply the post SP2 fix after we apply SP2.

We are just planning our SP2 rollout and need to decide to apply the fix now or wait for the post SP2 hotfix version.

Very pleased to see this fix though as it's a big issue to us
Not applicable
Dave Trevallion, that is correct.  I expect the SP2 version to be released any day now.

-aseigler
Not applicable
Hmm. ADModify.Net has always worked well in the past. Maybe it's just not well known.
Not applicable
Total: 6 objects. 5 succeeded, 1 failed.

Copy Exchange Files
Completed
Status: Completed.
Elapsed Time: 00:34:16


Organization Preparation
Completed
Status: Completed.
Elapsed Time: 00:20:15


Bridgehead Server Role
Completed
Status: Completed.
Elapsed Time: 00:02:00


Mailbox Server Role
Completed
Status: Completed.
Elapsed Time: 00:04:16


Client Access Server Role
Completed
Status: Completed.
Elapsed Time: 00:01:32


Unified Messaging Server Role
Failed
Installing product E:servereni386SetupServerRolesUnifiedMessagingesenus32.MSI failed. Fatal error during installation. Error code is 1603.
Fatal error during installation

Elapsed Time: 00:14:22

Not applicable
How come the fix is released only for SP1?!!
Not applicable
Please read the previous comments, the SP2 fix will be available very soon.

-aseigler
Not applicable
This wont work on Exchange 2000 SP3 correct?
Not applicable
Yonkey:

Correct, there are no plans at this time to release a fix for Exchange 2000.

-aseigler
Not applicable
Alex,
PSS needs to get it's act together then. They've just told me four times that the fix is included in SP2 (including after talking to an "engineer")
How are we meant to find out that the SP2 patch is still on its way when PSS don't know and neither the KB article nor this blog entry say so?
Not applicable
Well done Alex, you are as usual an exceptional coder!
And I suspected Mr.NOMAS wrote this hot fix when I saw the KB article independant of this post. :)

Please keep the good mods coming!
Dan.
Not applicable
Okay... what I don't understand is. How am I supposed to get this Hotfix? Why isn't it just "available" like every other hotfix Microsoft has ever released. Am I supposed to pay $250 to call PSS so I can get this one hotfix... that's rediculous!! Can't someone just send it to me, I have Exchange 2003 SP1.
Not applicable
Alan,

if you call us and ask for the hotfix up front - you will not be charged for the call.

There is actually a pretty good reason for you having to call in for this: this is a hotfix and as such, it did not go through as much testing as rollups or service packs do. So - by calling in, you give us your contact information. If there is a regression or a problem found in the hotfix later, we have a way to contact you and tell you about it and what to do. So - I'd suggest you call in. :)
Not applicable
Today this list is longer, since I hadn't the time to post it last week. Sorry...

Exchange on NAS:...
Not applicable
PSS do seem to think this fix is *included* in SP2. Apparently, the keyword "kbexchange2003presp2fix" in the KB article means to them that "this fix is included in SP2".

Could the KB be updated to include a definitive statement about SP2 support?

I await the SP2 version with bated breath!
Not applicable
BEWARE of hotfix 903158 if you are using BlackBerry handheld devices in your environment.

I applied 903158 on March 24th and it broke the ability for users to send email from their handheld devices -- everything else seemed to work fine; running E3SP1 and BES 4.0.3.11.

KB 912918 did not fix the problem, had RIM and Rogers -- Service Provider on the horn for 8 hours attempting to resolve. End result, removed the hotfix.

RIM suggested upgrading to 4.0.4 or 4.1; though could not confirm if this would solve the problem.

Still waiting to hear back from RIM SE's for a workaround or fix.

BES = POS
Not applicable
The fix has now been released for SP2.  The fix is KB916783.  The article is not ready, but the content will be identical to 903158.

Thanks,

-aseigler
Not applicable
Thanks for the update. Any comments on the message by White Ninja for BES compatibility ?

Thanks !
Not applicable
I heard of one other instance of BES issues, and it was resolved with 912918.  This fix definitely falls into the category in the "Cause" section of 912918.

It doesn't explicitly say so in 912918, but I believe that in order for the changes it recommends to take affect immediately, you must restart the store.  Otherwise, you must wait for the cache to flush, similar to what is discussed in 179065 and various other articles.

-aseigler
Not applicable
Installed the SP2 version with no problem for my Blackberry users.  I had previously gone through the steps in 912918 and am running BES 4.0.4.

-jdonaldson
Not applicable
Where can you get the Post SP2 fix?  I have tried to download it from premier support but it states that it is not currently available.
Not applicable
BGT:  You have to contact support and request KB916783.  It is not available for general download.

-aseigler
Not applicable
Hi there,

Hi you guys at M$ - just so you know this patch breaks Blackberry Enterprise Server...

I'm guessing you need to finetune it some more or Blackberry have some work to do.

Oliver
Not applicable
OliverM,


Please see this:

http://blogs.technet.com/exchange/archive/2006/03/22/422799.aspx#424417

Not applicable
attempts to contact PSS for 916783 have failed spectacularly. is there a secret password i can use to get the hotfix? thanks!
Not applicable
vwebb,

What happened? There is no secret password needed for you to get this patch :). When you call in, just be clear that all you need was a hotfix. I am not sure what happened but maybe try again?
Not applicable
called into PSS support noted i needed a hotfix for exchange 03, gave the article number. the liasons(?) were adamant on a few different occasions the aritcle couldn't be found and without an article there is no hotfix. even tried sighting this blog but no one would bite.
Not applicable
When I installed the hotfix (E2003 SP1, Bes 3.6), Blackberry users could no longer send.  KB912918 doesn't help because the BES Administrator already had Send As permission. I removed BES Admin and added using the KB article, but there was no change.
Not applicable
When I installed the hotfix (E2003 SP1, Bes 3.6), Blackberry users could no longer send.  KB912918 doesn't help because the BES Administrator already had Send As permission. I removed BES Admin and added using the KB article, but there was no change.
Not applicable
Hi Oldeman,
Ensure that the BES admin account does not have a deny on send-as.  Also where is do you see that he has the send-as permission? Examine one of your users and look at the security tab and see if he has the send-as permission.  Also make sure it is set to apply to user objects.

ff
Not applicable
Yup i installed it on my 2003 SP2 server and it did indeed
break my BES server. I called MS and told them about it,
and they said "it shouldnt have", to which i replied "oh but it
did" lol. So am i to understand that upgrading BES will fix
the problem ? m currently on 4.0.3
Not applicable
I think this is a place to ask - how does 916783 relate to MS06-029/912442?  06-029 also updates store.exe, but with a version number lower than 916783?  It *seems* that 916783 includes MS06-029 as it also has a much later build date/time (despite MS06-029 being released months after 916783.)  Thanks hugely for this patch!
Not applicable
I have confirmed that MS06-029/912442 breaks BES. We're on 3.6 still, but I imagine it would break any version. Same issue as mentioned in 912918. Have others run that script with success. I ran it and the output was daunting. We have 16,000 mbxs. I'm just not sure where to start in prepping our environment for the post 7650.23 store.exe world. Seems like there should be a utility put out that just "does it". Is this script it? I haven't studied it in depth yet.
Not applicable
(Egentligen skulle jag bara posta om att jag hittat NoMAS-verktyget för publik nerladdning, men det känns...
Version history
Last update:
‎Jul 01 2019 03:12 PM
Updated by: