Yealink HDMI ingest - lag and call failure

[andy_bentley]

We have many Yealink MVC800ii and MVC840 MTR systems running in our meeting rooms. We're seeing real issues with HDMI ingest, particularly using the HDMI cable that connects to the MTouch II panel, and less so when using the WPP20 wireless dongle - but both are not working optimally.

The usual pattern is that either presenting or sharing content (both scenarios are affected) works until around 50-60mins into a call/meeting then things start getting really laggy. Mouse moves or slide navigation are delayed and not seen on the front-of-room displays or by external participants for several seconds. The delay then starts to extend to 30+seconds, usually resulting in the call eventually dropping and the MTR needing to be restarted.

This is happening in different rooms, to different users using different models of laptop and when sharing content stored locally on their devices as well as opened directly from M365 services such as SharePoint or OneDrive.

All indications point towards a problem with the in-room kit, like a memory leak. All of our systems have fully up-to-date software and firmware and I've confirmed that they are restarting nightly via the 02:01 scheduled task. Other than Extron CTL100 workplace controllers installed in some of the affected rooms, there is no 3rd-party hardware involved in the MTR set-up, its all standard Yealink kit. We've swapped out components, cables, and even factory reset and rebuilt some systems.

Call activity and system health shown in the TAC rarely show any issues. Sometimes there is a report of 'poor' audio quality in some (not all) of the affected meetings, but generally everything appears 'healthy' or 'good'.

Has anyone experienced these symptoms (with Yealink or other manufacturers)? Or any ideas on what to try? It's now a real problem for us as we open more MTR equipped rooms.

[#2]

We are having the exact issue you described, but with the MVC640 room kits. Yealink support is being impossible to work with and demanding proof - like recorded proof of this happening. The devices are less than 6 months old and worked fine at first, but have been having multiple issues as of late.

[#3]

Nice to hear we're not alone! (although sorry you're also suffering with this issue).

It is completely subjective, but I'd have to agree that it seems like a recent update has introduced the problem. We've had our MVC800ii MTRs for around 18 months and didn't have any reports of this issue until towards the end of 2021.

I'm pinning a lot of hope on the new Teams Cast feature (we're currently testing it) as an alternative to using Yealink HDMI/USB-C ingest or the WPP dongles.

[#4]

I agree, December is the time frame for us too when the issues started. The CAST feature looks interesting! I hope that is a good solution!!

[#5]

Yealink support also just informed us of the following:

"Regarding the problem of delay in cable content sharing, this problem has been confirmed. When the customer is sharing content, the resolution of the TV is set to 4K, and the 4K resolution causes the CPU usage of the MTR application to be too high. The delay of the auxiliary stream is high or the sound and picture are out of sync. This issue is known internally by Microsoft and may be encountered by all manufacturers.

The solution to the delay problem is to adjust the resolution of the TV to 1080P. If some 4K TVs cannot be set to 1080P, you can purchase a third-party EDID Emulator accessory and connect it between the monitor and the HDMI port of the MCore to solve the problem."

[#6]

Thanks for the update. We have Extron CTL-100 workplace controllers in our rooms, I'm going to check that they are configured for 1080p EDID and will advise.

[#7]

@andy_bentley  @PatrickIPM 

Having the same problem with a customer on fifteen MVC840 Systems. But the tricky thing is that it works when internal users use HDMI ingest. No problems at all. But when an external user that is not in there tenant uses this i will start to lag directly. Works for both when they share content wirelessly, But as soon as the external uses the HDMI/USB-C cable it goes bananas and video/audio lags big time. Will also try third party EDID management, but the internal/external difference is not making any sense to me.  

[#8]

All of our rooms have 4K screens. This weekend I went around all MTR systems and set the display resolution directly on the MTR systems to 1920x1080. Most were running at 4K res so I need to check our CTL-100 programming as they should be configured to signal a 1080p EDID. We do have some rooms without CTL-100's and for some of these I installed pass-though 1080p EDID emulators. We now have 3 configurations that we will monitor:

1. Rooms with 4K screens and Extron CTL-100 workplace controllers. MTR system manually set to 1920x1080 res. EDID on CTL-100's to be checked and changed to 1080p if required.

2. Rooms with 4K screens and pass-though 1080p EDID emulators in-line between MTR system and screen. MTR system set to 1920x1080 res (which unsurprisingly Windows reports as 'Recommended')

3. Rooms with 4K screens MTR system directly connected to screens. MTR system manually set to 1920x1080 res.

One observation is that even after hard-coding the MTR systems without CTL-100 or EDID emulators to 1920x1080 (scenario 3), the screens were still reporting a 4K signal. This is unexpected. I trust the screens, they are commercial Panasonic displays.

I am also promoting the use of Teams Cast to our users as an alternative method of screen sharing.

[#9]

@andy_bentley 

Nice. Let us know how it goes!!!.

I always say that the monitor is the boss in the system when it comes to resolution. We always need to use pass through EDID emulators to be able to set the 1080p resolution, and have it to stay there. Otherwise the monitors will "override" the system and it changes back to 4K/UHD. System needs to tell the monitors what they will get. Not the other way around :). 

I know that Crestron actually and always includes a EDID emulator in all of their MTR systems they sell. Just because they know about these issues, and they tell us that they don´t support problems that occur if you don´t use it while setting up their hardware. 

But i still can´t figure out why it works for internal staff, and not for external visitors. 

[#10]

@tonyhermansen Your internal/external situation is very strange. I wouldn't expect user tenancy to affect content sharing (that happens through a combination of in-room hardware and software) to be affected by the Teams user account in any way. Any commonality to the external users? e.g. laptop make/model, or are they from one company who might be applying some funky system policies?

The 1080p EDID emulators I am using are ~£20 each from a large online retailer that has an association with a South American river.

[#11]

Yes, I think it is very strange. The customer has bought 800 Lenovo Carbon x1 Gen.9 laptops. But i have not myself checked what the external visitors have. I know that Microsoft up till a few days ago had som really big issues with not being able to call or invite external people directly from the MTR. So i had a thought if that was connected. Just because both problems was connected to external users. But we will install EDID Emulator and i think ours cost around 60Euro Each (The same one that Crestron uses for there MTR kits.) . Just to see if it makes a difference.
The only thing i can find from Microsoft or Yealink on this problem is the resolution issues. But i think that it would affect ALL users.

And i will also try to see this first hand to be able to look through the external computer and see if there is something there that causes this. I know that the CPU can rise very high when hosting a meeting and presenting at the same time. But i think that it should be able to deliver ok content at least.

[#12]

Whether the user is external or internal to your tenancy should not matter. This is a pure hardware-level connection. I would take note of the model of laptop that is connected, What you might be finding is that the guest laptop is trying to output at 4K and the ingest then has to convert to 1080p. Keep in mind the EDID emulator is telling the front of room display what resolution to set to, it has no bearing on the ingest resolution.

I would try and do some testing with different laptops and see what the HDMI output is trying to send. This may help you pinpoint what is going on.

[#13]

As Teams Cast was delayed and has only just been reactivated for us we're only now about to begin testing Cast as an alternative to HDMI ingest in our rooms

[#14]

@andy_bentley How have your three configuration testing modes gone?

[#15]

@theflyingirishman 

We hade five Yealink systems with a lot of lag problem at a customer. But then when we added a HDMI emulator to every screen we were able to eliminate the lag problem in all of the rooms. We used a emulator called : SE-HDMI-EMU-1080p

[#16]

@theflyingirishman 

Hi, Extron CTLs performing EDID emulation do help. As do simple HDMI pass-through emulators. We've proven both to help.

Just last week I got a report of screen mirroring delay again in one of our rooms, and upon checking the MTR had reverted to 4K resolution - and this is with emulators installed so I've no idea what had happened there! I took the opportunity to implement the new Resolution and Scaling options in the skypesettings.xml customisation file and did observe that the MTR system restarted in 1920x1080 resolution and this held across restarts regardless of whether EDID emulation was in-use. So this does seem like a good fix, to force the MTR itself into Full HD in software.

See the EnableResolutionAndScalingSetting, MainFoRDisplay and ExtendedFoRDisplay settings.

Remotely manage Microsoft Teams Rooms device settings - Microsoft Teams | Microsoft Learn

Regarding Teams Cast - it works, but we don't like it as it confuses our users. If there isn't an active Teams call to join, one is created meaning your experience if all you want to do is mirror your screen in a meeting room is you get it on one FoR screen only and the other shows a blank participants screen which is a pretty poor experience.

I am really hoping the new XML resolution and scaling settings see an end to our screen mirroring delays once and for all.

[#17]

Thanks, do these need to be plugged in at the screen end or the MTU end or does it not matter? Just trying to get my head around this :)

[#18]

@theflyingirishman it doesn’t matter, they just need to be inline between the screen and the MTR system

[#19]

@andy_bentley Thanks a lot for your solution. This was helping us very much. We had that problem only with two 4k Screens, if it is only one screen with 4 it is no problem, and if there is a 1080p screen and a 4k screen it was also no problem. But with the right content in the XML everything is working.

<EnableResolutionAndScalingSetting>true</EnableResolutionAndScalingSetting>
<MainFoRDisplay>
<MainFoRDisplayResolution>1920,1080</MainFoRDisplayResolution>
<MainFoRDisplayScaling>100</MainFoRDisplayScaling>
</MainFoRDisplay>
<ExtendedFoRDisplay>
<ExtendedFoRDisplayResolution>1920,1080</ExtendedFoRDisplayResolution>
<ExtendedFoRDisplayScaling>100</ExtendedFoRDisplayScaling>
</ExtendedFoRDisplay>