Music on Hold in Teams & 4 seconds until Hold is in place for caller

[DaveChomi]

There was a time I found out during my tests of our Direct Routing implementation that Microsoft made its promises happened and Music On Hold started to work on regular PSTN calls to Teams users.

It was a time when I got a report of the new "issue". Users complained it takes too long until the call is put on Hold after clicking on the button. I was just curious what happened because I remember from previous tests this was just matter of second but users reported 4 seconds. Hmmm.... 

Then I realized I can hear the music ... wohoooo ...  

So deeper deep dive showed that Microsoft is handling Hold and Music on Hold in very specific way. At least for me.

If you click Hold in Teams during the PSTN call you are basically transferring the call to another Teams "user"/object which is playing music for you. You can even hear the very short ring! 

Very confusing for the caller on PSTN side but I believe it's about way how you want to handle the Ring message on your SBC at the end. Good reason I believe not to change that behavior because of other reasons.

And that's also the explanation why it takes so long time! Imagine with Direct Routing following scenario (below is SIP trace for better understanding)

1/Teams proxy is sending REFER,

if you have your SBC setup in proper and recommended way then

2/ SBC sends invite back to Teams Proxy,

3/ then you have Ringing in place,

4/ all the SDP exchange of media sources (we are also running Mediabypass so imagine all ICE lite story happening) and boom,

5/ your call is on hold aka transferred to something else than your Teams user. 

Obviously the other way around, I mean getting the call back from Hold, is somehow reverted process but without REFER. Teams proxy basically sends new INVITE but now with Teams users media sources. Also takes a while and some users might be confused that they did something wrong or call is gone?!

So at the end it's not 4 seconds but slightly around 3 (users are exaggerating a bit, right)

I believe that this way of handling MoH is also reason why it will take some time until Microsoft will provide also MoH for Transfer function in Teams. Now if you click transfer or consult transfer the Teams Proxy sends Invite with inactive media session so might be another confusion for PSTN caller that there is just silence. 

Looking forward for Microsoft delivering these features for 100% and even more I would be happy to provide our management their own company MoH (not only on queues or AA 😉 but for users)

Hope this article helps some of you 

[#2]

@DaveChomi 

I am seeing the same issue across three seperate Direct Routing instances I have configured.
Do you have any links to other resources that discuss the problem?

Thanks

M

[#3]

@Murray Walker 

I haven't found any other.

But since I wrote this article I did some other tests and I believe that there is something also on side of the Teams app and server side communication (I mean not only SIP part of the communication while Direct Routing is involved into the call) because that delay in response of the app after pressing Hold and Resume button is also on regular Teams calls between two Teams users. Not so long like with Direct Routing calls but it is still something that could confuse users. 

[#4]

@DaveChomi 
Hello Dave, Quick question:
"I believe that this way of handling MoH is also reason why it will take some time until Microsoft will provide also MoH for Transfer function in Teams. Now if you click transfer or consult transfer the Teams Proxy sends Invite with inactive media session so might be another confusion for PSTN caller that there is just silence. " 
Does the above paragraph mean that MoH is not working with transferring PSTN calls via Direct Routing? if yes, is there a way to know when it will be released and working? 

[#5]

Hi @HendGamal 

that is exactly what I can confirm is the behavior on calls. 

If you do Hold - MoH is played to other side - you cannot transfer call as option in menu then

If you do transfer - there is silence to the other side - you can transfer to user/number

If you do Consult - there is silence to the other side - you can chat, call and then transfer

This is indeedy another very basic that I would like to already see resolved, same as custom MoH. But no indications when this will happen so far on public.

[#6]

@DaveChomi 

I have come across a similar issue. I'm afraid I'm learning but still a bit of a Noob so my terminology is not as fancy as others... Hopefully I'm able to express the issue though...

Situation: An external call is received into Teams, and then a Consult with Transfer is started to an internal Teams member. The option to call in the consult box is used but the Team member doesn't answer their phone. When the call goes to voicemail, the person transferring selects either "stop consulting" or the red hang-up button. This takes you back to the original call on hold. 

Issue: when retrieving this call off hold to speak to the original caller, there is some noise in the line but the call doesn't connect. 

Temporary work-around: Have to put the call back on hold then retrieve from hold and the caller is there. 

Question: is there a reason why this is happening, and how can I fix it? 

Thanks in advance.

[#7]

@VickiFHL 
I am sorry I cannot put more light on this issue as we do not experience such so far. 
Do you really see this issue on all users? 
What do you see on your SBC on SIP traces? That would be my source for searching an answer. 

[#8]

Hi,
I had a music on hold problem that may shed some more light on this issue.
The Microsoft support team was helpful in resolving this on their back end, as it was an issue on their back end specific to my country (in APAC region). However read on, as this may also apply to your Country/Region.

This was a Microsoft Teams direct routing solution I implemented in combination with an on premise PBX.
Issue was that when a Teams user called out to the PSTN, and placed the call on hold, there would be silence for around 10-15 seconds then the call would disconnect.
Call flow was as below
MS Teams User > SIP > Teams Voice System > SIP > SBC > SIP > on Premise PBX > H323 > Gateway > ISDN/PRI > PSTN.

After troubleshooting each component on our end we found that it was Microsoft that was sending a disconnect due what looked to be a timeout after the Microsoft Teams user put the PSTN party on hold.
We logged a support call mith microsoft support.
They explained that music on hold is handled by a bot.
The back end Microsoft phone system was summoning a music on hold bot based in North America, even though our O365/Teams tenancy was in the APAC region.
This caused a network delay resulting in a disconnect from the Microsoft side. As you can see below, the delay between the REFER and the BYE messages is 14 seconds

REFER from Microsoft at 12:23:37
BYE from Microsoft at 12:23:51

Microsoft found that they had not deployed the Music on Hold Bot component in the country that my tenency was based, (It is in APAC region).
Initially they replied saying there was no short term solution, but there were plans to do either of the following:
(1) Increase the timeout to prevent a disconnect
(2) Deploy the Music on Hold bot component in my country's O365/Teams tenancy, so that the call wouldn't have to traverse to North America from APAC region.

As a result of our support call Microsoft decided to deploy the missing component in the O365/Teams tenancy for my country.
After this compoment was deployed, the issue was resolved.
Calls are now playing music on hold with an unoticeable delay of around 1-2 seconds.
So I suggest that if you are experiencing any delay when placing calls on hold, open a case with Microsoft and see if the music on hold BOT being summoned is in the same region as your tenancy, or if it is going to another Country/Region, which may cause the same network delay that we experienced.

I've listed the SIP flow of a failed scenario and a working scenario, with some details removed for privacy.
You can see in the failed scenario that Microsoft sends a Bye message after a timeout of 14 seconds.
In the working scenario, there is another node involved on the Microsoft side, also no Bye timeout message like in the failed scenario.
Rather, there is a new INVITE sent out when I pressed the resume button on the teams client to take the PSTN called party off hold.
I hope this helps anyone with this issue!

MUSIC ON HOLD FAILED SCENARIO
BEFORE MICROSOFT DEPLOYED MUSIC ON HOLD BOT IN MY REGION

F20 | REFER | 52.114.14.70 -> Device | 12:23:37

REFER sip:{REMOVED}:5061;transport=tls SIP/2.0
FROM: {REMOVED}@sip.pstnhub.microsoft.com:5061;user=phone>;tag=28c0b1e27a2f49ad86bd6f901db1c23e
TO: <sip:{REMOVED}:5061>;user=phone;tag=17670549~318b362f-6991-45ef-a03b-77797077778f-30868965
CSEQ: 4 REFER
CALL-ID: 06d3c243ef90595ea9dee7f6836f8ee0
MAX-FORWARDS: 70
VIA: SIP/2.0/TLS 52.114.14.70:5061;branch=z9hG4bKacdf4564
CONTACT: <sip:api-du-c-ause.pstnhub.microsoft.com:443;x-i=ccdee34b-d2af-4991-9b57-07cc512b9b38;x-c=06d3c243ef90595ea9dee7f6836f8ee0/d/8/d2b5b977d51d4991b9463ba5777fa9f6>
CONTENT-LENGTH: 0
REFER-TO: <sip:sip.pstnhub.microsoft.com:5061;transport=tls;x-m=8:orgid:752b3414-9b11-4522-8b49-2d866e902a6b;x-t=479e69d1-78bc-4e1a-81dd-047fe9928e1f>
REFERRED-BY: <sip:sip.pstnhub.microsoft.com:5061;x-m=8:orgid:752b3414-9b11-4522-8b49-2d866e902a6b;x-t=479e69d1-78bc-4e1a-81dd-047fe9928e1f;x-ti=ccdee34b-d2af-4991-9b57-07cc512b9b38;x-tt=aHR0cHM6Ly9hcGktZHUtYy1hdXNlLnBzdG5odWIubWljcm9zb2Z0LmNvbS92MS9uZ2MvY2FsbG5vdGlmaWNhdGlvbj9kY2k9YjUyZmUyZmEzYmFlNDg5NGFjY2RkZjcwZGQzNzcwY2Y%3D>
USER-AGENT: Microsoft.PSTNHub.SIPProxy v.2020.6.4.2 i.ASSE.2ALLOW: INVITE,ACK,OPTIONS,CANCEL,BYE,NOTIFY

This was the SIP disconnect message from Microsoft

F39 | BYE | 52.114.14.70 -> Device | 12:23:51

BYE sip:{REMOVED}:5061;transport=tls SIP/2.0
FROM: <sip:sip.pstnhub.microsoft.com>;tag=8a3b513825004cd0a64a6e5e363c5d5a
TO: <sip:{REMOVED};user=phone>;tag=1c584506143
CSEQ: 1 BYE
CALL-ID: 7111332091762020122335@{REMOVED} 
MAX-FORWARDS: 70
VIA: SIP/2.0/TLS 52.114.14.70:5061;branch=z9hG4bK213262b6
REASON: Q.850;cause=16;text="ce6ac059-43ea-44b8-8ad6-b2cabbb0f1e3;Call ended by media agent."
CONTACT: <sip:api-du-c-ause.pstnhub.microsoft.com:443;x-i=ce6ac059-43ea-44b8-8ad6-b2cabbb0f1e3;x-c=b52fe2fa3bae4894accddf70dd3770cf/s/1/d2b5b977d51d4991b9463ba5777fa9f6>
CONTENT-LENGTH: 0
USER-AGENT: Microsoft.PSTNHub.SIPProxy v.2020.6.4.2 i.ASSE.0
ALLOW: INVITE,ACK,OPTIONS,CANCEL,BYE,NOTIFY

--------------------------------------------------------------------------------

--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
MUSIC ON HOLD WORKING SCENARIO
AFTER MICROSOFT DEPLOYED MUSIC ON HOLD BOT IN MY REGION (Placed a PSTN party on hold from Microsoft Teams and took them off hold, Music was playing, no delay)

F14 | REFER | 52.114.7.24 -> Device | 11:54:32

REFER sip:{REMOVED}:5061;transport=tls SIP/2.0
FROM: {REMOVED}<sip:{REMOVED}@sip.pstnhub.microsoft.com:5061;user=phone>;tag=c00ce9d58f3c43c8b26e31d34e10de6c
TO: <sip:{REMOVED}:5061>;user=phone;tag=20386857~318b362f-6991-45ef-a03b-77797077778f-30875372
CSEQ: 2 REFER
CALL-ID: 733faf29cc2253acb14d245441d0acc2
MAX-FORWARDS: 70
VIA: SIP/2.0/TLS 52.114.7.24:5061;branch=z9hG4bKbf649f34
CONTACT: <sip:api-du-b-uswe2.pstnhub.microsoft.com:443;x-i=d50e53f6-f50a-4629-be5c-73d96231c2ee;x-c=733faf29cc2253acb14d245441d0acc2/d/8/63764455e1f84bf5b64e95408110e975>
CONTENT-LENGTH: 0
REFER-TO: <sip:sip.pstnhub.microsoft.com:5061;transport=tls;x-m=8:orgid:752b3414-9b11-4522-8b49-2d866e902a6b;x-t=479e69d1-78bc-4e1a-81dd-047fe9928e1f>
REFERRED-BY: <sip:sip.pstnhub.microsoft.com:5061;x-m=8:orgid:752b3414-9b11-4522-8b49-2d866e902a6b;x-t=479e69d1-78bc-4e1a-81dd-047fe9928e1f;x-ti=d50e53f6-f50a-4629-be5c-73d96231c2ee;x-tt=aHR0cHM6Ly9hcGktZHUtYi11c3dlMi5wc3RuaHViLm1pY3Jvc29mdC5jb20vdjEvbmdjL2NhbGxub3RpZmljYXRpb24%2FZGNpPTdmODJhYmNiYjZkYjQyOTRhNTk3MTZjNWU5ZWVhNDlj>
USER-AGENT: Microsoft.PSTNHub.SIPProxy v.2020.7.8.1 i.ASEA.7
ALLOW: INVITE,ACK,OPTIONS,CANCEL,BYE,NOTIFY

INVITE sent from Microsoft to SBC when I pressed resume on Microsoft teams after having called party on hold for around 25 seconds.

F32 | INVITE (SDP) | 52.114.7.24 -> Device | 11:55:00

INVITE sip:{REMOVED}:5061;user=phone;transport=tls SIP/2.0
FROM: {REMOVED}<sip:{REMOVED}@sip.pstnhub.microsoft.com:5061;user=phone>;tag=87c2020910e747e9ab0a429d36fa1d2a
TO: <sip:{REMOVED}:5061;user=phone>
CSEQ: 1 INVITE
CALL-ID: 4fa906f2160c5fa693030031394afd3b
MAX-FORWARDS: 70
VIA: SIP/2.0/TLS 52.114.7.24:5061;branch=z9hG4bKc2d37118
RECORD-ROUTE: <sip:sip-du-a-as.pstnhub.microsoft.com:5061;transport=tls;lr>
CONTACT: <sip:api-du-b-uswe2.pstnhub.microsoft.com:443;x-i=fce5c483-4248-4e32-ad7a-3f87d9f31188;x-c=4fa906f2160c5fa693030031394afd3b/d/8/5567c7f5631c42af9ee7ef8995db7adc>
CONTENT-LENGTH: 1108
MIN-SE: 300
SUPPORTED: timer
USER-AGENT: Microsoft.PSTNHub.SIPProxy v.2020.7.8.1 i.ASEA.5
CONTENT-TYPE: application/sdp
ALLOW: INVITE,ACK,OPTIONS,CANCEL,BYE,NOTIFY
REQUIRE: replaces
REPLACES: {REMOVED}@{REMOVED};from-tag=7a7716055ac64a4cbefb94e30bb518f1;to-tag=1c759665711
SESSION-EXPIRES: 3600

v=0
o=- 80140 0 IN IP4 127.0.0.1
s=session
c=IN IP4 52.114.192.146
b=CT:10000000
t=0 0
m=audio 50908 RTP/SAVP 104 9 103 111 18 0 8 97 101 13 118
c=IN IP4 52.114.192.146
a=rtcp:50909
a=ice-ufrag:+D8R
a=ice-pwd:{REMOVED}
a=rtcp-mux
a=candidate:1 1 UDP 2130706431 52.114.192.146 50908 typ srflx raddr {REMOVED}  rport 50908
a=candidate:1 2 UDP 2130705918 52.114.192.146 50909 typ srflx raddr {REMOVED}  rport 50909
a=candidate:2 1 tcp-act 2121006078 52.114.192.146 49152 typ srflx raddr {REMOVED}  rport 49152
a=candidate:2 2 tcp-act 2121006078 52.114.192.146 49152 typ srflx raddr {REMOVED}  rport 49152
a=label:main-audio
a=mid:1
a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:{REMOVED}
a=sendrecv
a=rtpmap:104 SILK/16000
a=rtpmap:9 G722/8000
a=rtpmap:103 SILK/8000
a=rtpmap:111 SIREN/16000
a=fmtp:111 bitrate=16000
a=rtpmap:18 G729/8000
a=fmtp:18 annexb=no
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 RED/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=rtpmap:13 CN/8000
a=rtpmap:118 CN/16000
a=ptime:20

@DaveChomi 

@VickiFHL 

@Murray Walker 

@HendGamal 

[#9]

@AK202020 

Thank you for sharing, I would not say that there might be still missing infrastructures for some regions

😞 

But what I see on our tenant is that they improved the time for transferring the call to the bot. 

Still that ring to calling party which is put on hold and transferred to MoH bot is confusing but it is what it is.

[#10]

@DaveChomi 

A little update from our end on this issue. My IT Manager has put in a ticket about ensuring the correct zone is being used for the hold bot, as we are in Australia. They were quick to get the ticket underway but I haven't heard anything further in the meantime though. 

What HAS made a difference, though, is that on my call queues I have opted in to the Conference Call option. This has increased the speed of answering by quite a few seconds, and we are now only having to press resume once and the caller is coming back straight away!

We just made sure we were using the protocols as in this instruction: 

https://docs.microsoft.com/en-US/microsoftteams/create-a-phone-system-call-queue?WT.mc_id=TeamsAdmin...

Thanks all for your assistance, it's great that we can work together to find solutions for these issues.

[#11]

@VickiFHL 
We are still seeing on hold delays of 3-10 seconds for Teams Direct Routing calls to individual users.

I have also been talking to our SBC support about the transfer silence issue. They have advised that some Telcos actually inject their own MoH when Teams sends a re-invite after the user presses "Transfer" in the Teams client. The 2 largest Telcos in Australia (Telstra and Optus) do not inject their own MoH for re-invites they receive that have a=inactive or a=sendonly in the SDP. For these carriers and many others, you need the SBC to intercept these re-invites and inject a MoH audio stream on behalf of the Teams user.

If injecting your own MoH at the SBC you may have an issue for blind transfers. Blind transfers start with a re-invite from Teams on first "transfer" button press, but then after you enter the phone number and press "transfer" a second time to complete the transfer, Teams sends a refer with no SDP to notify the SBC that a=sendrecv now. Could be that some SBCs are able to deal with this however.

It would be good if Teams actually acted more like a PBX and did this stuff on its own.

[#12]

@Gasmanz Thanks so much for that information - I'll have a look into the MoH options as it would be great to be able to change that up. 

[#13]

@DaveChomi microsoft confirmed that if you are using the LBR configuration. Music on hold will be having the of 5 sec. 

[#14]

We are currently experiencing the music on hold still playing during the call after the call has been transferred from caller A to caller B. We are on Microsoft Calling Plan. It is ridiculus.