Lot of eventID 4999 related to ActiveSync on my Exchange Servers

[VincentBurle83]

Hello all,

I'm Vincent and I'm Exchange administrator in my International Research Organization.

Our Exchange farm is composed of 4 Exchange Server 2016 Enterprise CU 22.

I recently noticed a lot of events with ID 4999 in the Application logs on all 4 servers like below:

Log Name: Application
Source: MSExchange Common
Date: 25/02/2022 15:09:28
Event ID: 4999
Task Category: General
Level: Error
Keywords: Classic
User: N/A
Computer: XXXXXX 
Description:
Watson report about to be sent for process id: 5560, with parameters: E12, c-RTL-AMD64, 15.01.2375.018, w3wp#MSExchangeSyncAppPool, MSExchange ActiveSync, Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData, UnexpectedCondition:NotImplementedException, 76de, 15.01.2375.018.
ErrorReportingEnabled: False
exData=|exHResult=|exStacktrace=at Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData()
at Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncContentProperty.InternalCopyFrom(IProperty sourceProperty)
at Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncContent14Property.InternalCopyFrom(IProperty sourceProperty)
at Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.FlexibleSchemaStrategy.ExecuteCopyProperty(IProperty srcProperty, AirSyncProperty dstAirSyncProperty)
at Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncDataObject.CopyFrom(IProperty srcRootProperty)
at Microsoft.Exchange.AirSync.MailboxItemFetchProvider.BuildResponse(XmlNode responseNode)|exTargetSite=MSExchange ActiveSync Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData|exSource=|exMessage=|exComplete=

However, none of our users using ActiveSync with their mobile complained so far. 

Did one of you already had similar problem?

[#2]

Same ErrorID 4999 here, only with another processID (mostly 25636, but also others):

Watson report about to be sent for process id: 25636, with parameters: E12, c-RTL-AMD64, 15.01.2308.021, w3wp#MSExchangeSyncAppPool, MSExchange ActiveSync, Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData, UnexpectedCondition:NotImplementedException, 0, 15.01.2308.021.
ErrorReportingEnabled: True
exData=|exHResult=|exStacktrace=bei Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData()
bei Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncContentProperty.InternalCopyFrom(IProperty sourceProperty)
bei Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncContent14Property.InternalCopyFrom(IProperty sourceProperty)
bei Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.FlexibleSchemaStrategy.ExecuteCopyProperty(IProperty srcProperty, AirSyncProperty dstAirSyncProperty)
bei Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncDataObject.CopyFrom(IProperty srcRootProperty)
bei Microsoft.Exchange.AirSync.MailboxItemFetchProvider.BuildResponse(XmlNode responseNode)|exTargetSite=MSExchange ActiveSync Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData|exSource=|exMessage=|exComplete=

Everything started on 13.2.2022 with EventID 4999 but other message within:

Watson report about to be sent for process id: 16564, with parameters: E12IIS, c-RTL-AMD64, 15.01.2308.014, w3wp#MSExchangeOWAAppPool, M.E.C.Owa2.Server, M.E.C.O.S.C.OwaMapiNotificationManager.SubscribeToSuiteNotification, System.NotSupportedException, 80d2-dumptidset, 15.01.2308.014.
ErrorReportingEnabled: True

Over 160 views in this thread but no solution or at least a hint?

[#3]

I have exactly the same issue. Would be nice somebody will pick this topic up 😕

+ System 

  - Provider 

   [ Name]  MSExchange Common 

   - EventID 4999 

   [ Qualifiers]  16388 

    Level 2 

 Task 1 

    Keywords 0x80000000000000 

   - TimeCreated 

   [ SystemTime]  2022-03-29T12:33:41.592563000Z 

    EventRecordID 32720269 

    Channel Application 

    Computer rock.Genedata.win 

    Security 

 - EventData 

   23208 

   E12 

   c-RTL-AMD64 

   15.01.2308.021 

   w3wp#MSExchangeSyncAppPool 

   MSExchange ActiveSync 

   Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData 

   UnexpectedCondition:NotImplementedException 

   76de 

   15.01.2308.021 

   True 

   False 

   w3wp#MSExchangeSyncAppPool 

   exData=|exHResult=|exStacktrace=at Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData() at Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncContentProperty.InternalCopyFrom(IProperty sourceProperty) at Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncContent14Property.InternalCopyFrom(IProperty sourceProperty) at Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.FlexibleSchemaStrategy.ExecuteCopyProperty(IProperty srcProperty, AirSyncProperty dstAirSyncProperty) at Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncDataObject.CopyFrom(IProperty srcRootProperty) at Microsoft.Exchange.AirSync.MailboxItemFetchProvider.BuildResponse(XmlNode responseNode)|exTargetSite=MSExchange ActiveSync Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData|exSource=|exMessage=|exComplete= 

[#4]

Hi @BroBias7  and @BalticNative  ,

Which version of Exchange do you have?
On my side I verified all monitoring probes on all of my servers as explained here: https://docs.microsoft.com/en-us/exchange/management/health/troubleshooting-activesync-health-set
None of them report any error. ActiveSync is working properly and no user complained.
I think that we will have to live with it ...

[#5]

@VincentBurle83 

I'm seeing the same exact errors with Exchange 2016 CU22.  I recently upgraded to CU22 with the naive optimism that it would be resolved with that CU but no dice.  Has anyone else found a fix for these errors?  I'm not getting any complaints but if that AppPool is indeed crashing and restarting repeatedly through the day I can't imagine that's a good thing.

[#6]

Same issue here with MSExchangeSyncAppPool, crashing & recycling 100-120 times a day without a followable reason. Would be nice to get some experience exchange on troubleshooting this 😕 ...

[#7]

@VincentBurle83
Sry, have 2 similar accounts here 😄
Yes, i can confirm this as well. ActiveSync is healthy in general. However, i also discovered a bunch of 1040 events which result of conflicts between http(s)_responses for heartbeat intervalls. timeout on FW is lower than on EXCH which results in 1040 events and maybe connection overloads on the MSEchangeSyncAppPool which then results in 4999 -> recycling.

My best logical conclusion at the moment. However, i cannot adapt http_response_timeout on our fw only for a dedicated target (activesync-url) and it's not an option to change this for entire http_proxy.

Any help is welcome ^^

[#8]

@VincentBurle83 

I am also seeing this error on a two-node Exchange 2019 DAG.  Has there been any progress for anyone?

[#9]

Unfortunately, no progress 😕

I'm only pretty sure in the meantime that this relates to firewall settings on the WAF side (here: Sophos XG). I've played with several settings there, the most potential was the http session timeout, but without any success.
It doesn't matter whether i follow vendor recommendations or settings based on user experiences, the result is always the same and the 4999 errors don't stop.

From my current perspective this must be an issue at least every "exch + Sophos" user/admin should face which is why i can't understand that there is no clear solution available 😞

[#10]

@BroBias Interesting that you also have a firewall/loadbalancer in the mix as well and think it's part of the issue.  We don't have Sophos. We have Fortinet on the perimeter and then a Kemp loadbalancer internally, I think at one point I stumbled upon someone mentioning the session timeouts may be the culprit but I never found the right values to prevent the errors either.  Sure would be nice if somoene on the Microsoft side would chime in on these annoying errors.

[#11]

Wow! Now, i'm very thankful that there is somebody sharing this experience and impression/feeling of the very source ^^
Yes, I'm also 99% sure that this relates to the combination of EXCH vdir <> FW/LB <> ActiveSync requests/sessions. I can even find this reflecting in the event logs, when the errors starting as soon as the users are off to the weekend or coming back from weekend again. I also played a lot with the http session timeout setting on fw/waf side but without any success, not even little changes in the amount of logged errors.
Even following very clear recommendations for EXCH + SophosXG - no effect at all.

The main issue is, from my perspective, that these errors don't cause significant issues, at least not on my side. From time to time some few users complain about "sync hangs" on their mobiles but that's it. Exchange seems to be fine whith recycling the app pool every hour ......
However, i'm annoyed about it because it's just filling up my event logs and leaves a bad feeling while not really knowing what's going on.

[#12]

Hi @BroBias7 and @cmillerLCE ,

Sorry for the late answer but I have to admit that did not take this error in account because ActuveSync is just working fine.

On our end, we also use a load-balancer (F5 Big-IP) so it is probably something to investigate deeper ... or not.

On my side I decided to stop my investigation.

Good luck guys !

[#13]

Running Exchange 2019 three node DAG with FG Big-IP LB.
Same issue here ... no progress on it though.

[#14]

@VincentBurle83 @ScottW3532 @BroBias7 @cmillerLCE @BroBias 

We are also seeing this issue since May.

We are running an Exchange 2019 fully up to date (CU13 Jun23SU).

We do have a Sophos UTM gateway in front of it (Webserver protection).

Has anybody any update on this?

The event 4999, MSExchange Common:

Watson report about to be sent for process id: 8024, with parameters: E12, c-RTL-AMD64, 15.02.1258.016, w3wp#MSExchangeSyncAppPool, MSExchange ActiveSync, Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData, UnexpectedCondition:NotImplementedException, 0, 15.02.1258.016.
ErrorReportingEnabled: False 
exData=|exHResult=|exStacktrace=à Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData()
   à Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncContentProperty.InternalCopyFrom(IProperty sourceProperty)
   à Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncContent14Property.InternalCopyFrom(IProperty sourceProperty)
   à Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.FlexibleSchemaStrategy.ExecuteCopyProperty(IProperty srcProperty, AirSyncProperty dstAirSyncProperty)
   à Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncDataObject.CopyFrom(IProperty srcRootProperty)
   à Microsoft.Exchange.AirSync.MailboxItemFetchProvider.BuildResponse(XmlNode responseNode)|exTargetSite=MSExchange ActiveSync Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData|exSource=|exMessage=|exComplete=

 And the associated EventData:

   8024 
   E12 
   c-RTL-AMD64 
   15.02.1258.016 
   w3wp#MSExchangeSyncAppPool 
   MSExchange ActiveSync 
   Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData 
   UnexpectedCondition:NotImplementedException 
   0 
   15.02.1258.016 
   False 
   False 
   w3wp#MSExchangeSyncAppPool 
   exData=|exHResult=|exStacktrace=à Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData() à Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncContentProperty.InternalCopyFrom(IProperty sourceProperty) à Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncContent14Property.InternalCopyFrom(IProperty sourceProperty) à Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.FlexibleSchemaStrategy.ExecuteCopyProperty(IProperty srcProperty, AirSyncProperty dstAirSyncProperty) à Microsoft.Exchange.AirSync.Common.SchemaConverter.AirSync.AirSyncDataObject.CopyFrom(IProperty srcRootProperty) à Microsoft.Exchange.AirSync.MailboxItemFetchProvider.BuildResponse(XmlNode responseNode)|exTargetSite=MSExchange ActiveSync Microsoft.Exchange.AirSync.Common.SchemaConverter.Entity.EntityContentProperty.get_MIMEData|exSource=|exMessage=|exComplete= 

[#15]

We are running Exchange 2019 with 2 nodes. Recently applied CU 13 and we faced the same issue. Anyone has an update ?