07-15-2019 08:08 AM
07-15-2019 08:08 AM
My organisation is planning a move to direct routing. We have an AudioCodes VE SBC. I've been working on this for awhile and got everything to a pretty decent state.
I've come across an issue which is fairly new as I remember testing this months ago, when I was still trying to figure out how to deal with REFERs coming from MS Teams on call transfers.
So the call flow is as follows:
Incoming call from SBC to MS Teams -> MS Teams user transfers the call to internal user -> Call fails as MS Teams generates a REFER back to the SBC rather than route the call internally to MS Teams user target.
The issue I'm observing aside from MS Teams routing the transfer leg of the call is that the REFER-TO user part of the URI is blank. Please see the attached screenshot.
Its as if MS Teams is treating this as an external transfer and does not know how to populate the user part of the REFER-TO header. I've tested this with both users set up with Phone system and DDIs assigned to the user and users which have no phone system add-on or assigned DDI.
Please note that if you tried to transfer the call to a PSTN number, that works fine. I'm unsure if this would be something config related within our Teams tenancy or a genuine bug. I'm more inclined that this is the former rather than the latter as it does seem like fairly basic functionality and I'm confident this used to work a few months ago.
Any advice would be appreciated.
07-15-2019 11:04 PM
We've got exactly the same behaviour when trying to transfer calls that came from PSTN network and got answered by one of our Teams users and when they try to transfer to another Teams user. REFER-TO doesn't contain any LineURI or telephone number to complete the transfer. Did you have any luck fixing this issue? Also, I'm not sure if this was working before or not. But you seem to be correct in regards to Teams treating internal transfer as external one and sending SIP REFER back to SBC without providing any additional details in regards where the call should be actually connected or transferred to. When transferring to external number this works perfectly fine, REFER-TO contains an external number and can be processed by our SBC. BTW, can you tell what SBC are you using? Is it AudioCodes or Ribbon one? Or something not currently certified by Microsoft? Thanks!
07-16-2019 03:16 AM
@Lt_Flash We're using AudioCodes, and config wise its fine. It handles REFERs just fine. The issue here is that MS Teams should never had sent this REFER out the SBC, it should have handled it within the Teams environment as its a transfer from one Teams user to another. This only thing I suspect config wise is voice route regex I'm using, as its pretty much a catch all e.g. ".*"
I've now made it more strict to capture digit patterns rather than all chars to see if the issue persists.
It makes no sense as I can call users internally, it only happens when PSTN call comes in from the direct Routing SBC, so Teams appears to be applying a different call routing logic when applying the transfer.
Out of curiosity can you let me know what your voice route pattern looks like? Was it a catch all like mine or is it more strict?
07-16-2019 04:06 AM - edited 07-16-2019 04:07 AM
Yes, I'm using the same voice pattern, ".*" to send all digits without any normalization to our SIP trunk. You're right, maybe the issue is because of that, I will update voice rules tomorrow to make it strictly country-based, like '^+44.*' to see if that helps! Good point, mate!
Also, yes, SIP REFER shouldn't be coming back to PBX when we're doing an internal transfer from one Teams User to another one, but it does for some reason, that's the problem.
I'll let you know here how it goes and thanks for your reply!
07-17-2019 10:03 PM
I've removed the ".*" rule from my installation and created all specific rules for DIDs and still no luck, I can see SIP REFER coming back to PBX when I'm trying to transfer call that came from PSTN to Teams user. No luck here, unfortunately. Did you have any luck with this? Thanks.
07-25-2019 12:47 AM
@Lt_Flash I had no luck with changing the voice route, so ended up raising it with MS support.
Apparently if you disable REFER as an allowed method within your SBC's signalling messages, then MS Teams does not use REFER for transfers and just re-INVITEs to the SBC. That way the internal transfers work perfectly fine.
07-25-2019 02:20 AM
07-25-2019 02:33 AM
No, not by rejecting the REFER with a response code. I basically did a message manipulation rule to re-build the Allow header for calls to MS Teams without including REFER. Which vendor are you using? If you have an AudioCodes as well, I can give you the rule syntax.
07-25-2019 03:04 AM
07-25-2019 08:52 AM
@Sims_Krastev you seem to be correct! I've just edited all messages that were coming from PSTN side of our SBC and stripped 'REFER' on SBC in 'Allow' field - and yes, Microsoft started to transfer calls correctly! Thanks a lot, mate, I've already read that disabling SIP REFER method helped some people, but I was trying to reply with 4xx or 5xx when receiving REFER from MS side, while I should have just removed that REFER method from SIP 'Allow' list! By the way, looking at the packets coming from Microsoft side I can see that they don't send REFER in allowed method list! Here's an example:
USER-AGENT: Microsoft.PSTNHub.SIPProxy v.2019.7.4.9 i.USWE2.4
Thanks again for your help and I'm glad everything is working fine for you too now!
08-01-2019 04:48 AM
I can confirm this behavior, just that REFER triggered transfer is most common way for it and only fully standardized one. If your SBC is capable to terminate REFER note that MS still clearly indicates in Refer-To where to send INVITE back:
and making sure you fill RURI with Refer-To and Contact uri-host correctly you will see MS does rest of the job nicely for REFER case too.
08-01-2019 05:46 AM
It shows where to send INVITE to, but it doesn't have a DID or username in SIP R-URI so SBC can't figure out where to dial to. SBC uses PSTN-type dialing, so it needs some LineURI or DID to send INVITE to, like +44XXXXXXXX@sip.pstnhub.microsoft.com. Otherwise it can't decide what to put in SIP INVITE method as 'To' field. I've tried to update SIP INVITE to whatever is in REFER-TO header and send it back to Microsoft, but I was getting a 400 Bad Request back.
There's no user part in this SIP RURI, only host. What should be put in To header? Contact header is obvious, it's our SBC Contact.
08-01-2019 05:59 AM
the thing is that SBC should only respect Refer-to header by means of SIP. REFER comes without user part but it's still fair enough and legal. So what my SBC does as last resort to send INVITE out is DNS query for hostname in Refer-To and fills RURI and To in same fashion:
INVITE sip:sip.pstnhub.microsoft.com:5061;x-m=8:orgid:fce2158f-0e99-4ff7-a0b3-5b6b87d16689;transport=tls SIP/2.0
Via: SIP/2.0/TLS 192.168.65.100:5061;branch=z9hG4bKvahihb0040adpr8tb320.1
CSeq: 1 INVITE
and that's just enough...
08-01-2019 06:02 AM
obviously we dont use same vendor SBC but standard wise logic should remain the same
08-01-2019 10:18 AM
Thanks for detailed description, I will try to reproduce this behaviour on my SBC, but it looks really strange to me that MS sends a SIP REFER packet back to SBC that connects calls to PSTN and uses LineURI telephone numbers for that. According to RFC it should provide a proper username or DID in such case. From what I can see now it's much simpler to just disallow REFER method and let Microsoft Teams handle internal call transfers on their side, which is more logical, rather than implementing such call forking. Anyway, your help is much appreciated and SIP INVITE packet is a perfect example on what I should try to achieve. Thanks!
08-02-2019 01:03 AM
Unfortunately, that doesn't work for me, I'm sending same INVITE packet as you do but I'm getting back a '400 Bad Request' with a REASON:
REASON: Q.850;cause=111;text="a5d458f9-14c0-4cc4-8c10-202277af11e9;Unable to parse RURI."
08-02-2019 01:07 AM - edited 08-02-2019 01:08 AM
Here's my INVITE packet, I had to remove triangle brackets to make post clear:
INVITE sip:sip.pstnhub.microsoft.com:5061;transport=tls;x-m=8:orgid:7709b36f-1f10-4b1c-8a0f-9c0e5390c86c SIP/2.0
Via: SIP/2.0/TLS X.X.X.X:5061;branch=z9hG4bKf26f.503bdc4.0;i=54849946
CSeq: 10432 INVITE
Allow: OPTIONS, REGISTER, SUBSCRIBE, NOTIFY, PUBLISH, INVITE, ACK, BYE, CANCEL, UPDATE, PRACK, REFER
Supported: 100rel, timer, replaces, norefersub
08-02-2019 01:41 AM
All right, just to clarify, I've got this sorta working, I was removing REFERRED-BY header when placing a new INVITE and that's why it couldn't connect the call, after leaving that header intact the call can be transferred. But this far for us it's much easier just to remove REFER method from the list of allowed methods, otherwise it's quite a complex setup with our SBC. Thanks everyone for replies, now we have two working methods that allow call to be transferred to internal MS Teams users!
08-02-2019 04:20 AM
yep, tend to be with you on conclusions. being forced to implement both in real life felt like sharing things people find useful :) Most important is we understand now how to implement both methods and logic MS uses to trigger each.
08-02-2019 07:39 AM - edited 08-02-2019 07:42 AM
Hi for us it's the other way around. Internal calls transfer perfectly fine and transfer to PSTN only works via the Teams Client. We do use some 3PIP Phones from AudioCodes, Yealink and Poly and if we try to transfer a call to an external Number, our SBC does not receive a refer-to number.
Now i have heard from someone, that their AudioCodes SBC receives this from Number even when it's coming from a 3PIP Phone.
Does anybody have an idea, why 3pip Phones or SfB Client for that matter (i know it's not supported but if 3pip should work, the SfB Client should also work) does not send a Refer to number to our SBC? Or is there anyone around who can test it and tell me if it's working with their SBC?
08-03-2019 05:52 AM
Are you typing in full E.164 external number? I have an issue with SFB where it's not sending calls to SBC unless I type in full '+44XXXXX' number even though Teams works fine with local numbers starting with '0' or any other number because we have '.*' VoiceRoute configured. Can't figure out why it's doing so, cooperation mode is set to Islands and SFB can send and receive calls but only when using full E.164 numbers. Some MS built-in normalization rules for the country are kicking in by the looks of it.
08-03-2019 05:54 AM
Also, maybe your SBC has a rule to remove Refer-To header and sometimes that rule is applied by incorrectly configured Match policy? Have you tried sniffing traffic and decoding TLS to see the actual SIP packet coming from MS before it's handled by SBC?
08-03-2019 05:57 AM
@Lt_Flash for me it doesn't make a difference if the number starts with +4144 or 044. from the looks of it, the normalization/translation of the number works fine with the SfB Client too. Also works when just dialing 044 (national format) instead of transferring.
Does it work for you when you enter full E.164 with the SfB Client or 3PIP Phone?
08-03-2019 06:02 AM
@Lt_Flash I don't think that the SBC removes the refer, because when I do it via the Teams Client, everything works fine. If I disable Referred-By on the SBC transfers stop working, even internal Teams Transfers. I'm not sure who to do TLS sniffing and decoding. What do I need to do? Run wireshark while i transfer and filter for TLS/TCP packets?
08-03-2019 06:06 AM
It's not that easy, you will need your SSL certificate and private key in order to decrypt the traffic, you need to google about that. But maybe you just disable REFER support like we did? That's much simpler and works much better and requires less rules on SBC to make calls work. Just strip 'REFER' from any 'Allow:' header coming from SBC on requests and responses.
08-20-2019 07:44 AM
It turned out, that there was in fact something wrong with how Microsoft / 3PIP SfB Phones sent the refer to our SBC. (Our SBC did not receive a refer-to number) Microsoft has now fixed the issue and we can correctly transfer calls to external numbers via the Teams Desktop Client and 3PIP Modes with users in TeamsOnly mode.
08-21-2019 07:30 PM
Good to know it's all fixed! Yes, the biggest problem when dealing with Microsoft is that you never know if that's an issue with your phone or software or with their end. I was speaking to their official support as we are an official partner and all I got in the end was 'Please post into our techcommunity'.
09-10-2019 09:14 AM
Can you post how to do message manipulation to re-build the allow header for this on AudioCodes?
09-11-2019 12:14 AM
Sure - here it is:
Action Subject - Header.Allow
Action Type - Modify
Action Value - 'PUBLISH,MESSAGE,UPDATE,PRACK,SUBSCRIBE,INFO,NOTIFY,REGISTER,OPTIONS,BYE,INVITE,ACK,CANCEL'
You can strip out any method you like or match what Microsoft send in their messages if you'd like. Its working fine for me with these.
09-11-2019 12:21 AM
Yep, works pretty much same way for me, even though right now I'm not using AudioCodes any more. And I'm analyzing each Allow string to strip only REFER from it instead of replacing Allow with generic one that allows everything.
09-12-2019 06:05 AM
Just from curiosity, do you use mediabypass already?
I have encountered interesting issue which is causing on PSTN calls to Teams user over SBC and transferred to my internal PBX extension or external number simply No Audio.
Here is the call flow - PSTN - myPBX - SBC - Teams user REFER- SBC - myPBX - phone
Once I manipulated my SBC that Microsoft takes care about sending re-Invite instead of REFER it sends new Invite as you observe BUT obviously as media source it uses the public IP of my SBC in the initial offer (it simply takes the info from original session leg).
My SBC is behind NAT. Properly configured. And it does not recognize that this new Invite is dedicated to media channel reserved already on the SBC :) so it attempts to establish new channel between private IP and public IP of its own interface -sending the media out to its own public IP - which is then visible on our firewall as traffic that is dropped. Even if I allow this traffic between private IP to public IP of the same interface it would cause flow which does not make sense at all.
If mediabypass is not used then it is fine, also for Teams caller transferred to myPBX. Just in case it is PSTN caller transfered back to SBC. (same problem occures also once the Teams user set Forwarding to his mobile for example...call connected but no audio)
09-12-2019 06:33 AM
@DaveChomi - Not I'm not using media bypass. If I have to be honest, I fail to see the benefits of media bypass. High bandwidth data line prices have dramatically decreased over the last couple of years, so by using media bypass you just save Microsoft a tone of bandwidth and infrastructure within their DCs, which I'm more than sure they have the money to pay for and maintain anyway.
Media bypass would probably work in your model for a simple Teams -> SBC -> PSTN or other way around model, when you start hair-pinning both legs of the call (existing and transfer) through existing infrastructure like your SBC and PBX you're just asking for trouble. I'd just disable media bypass if I were you, unless you're really struggling for bandwidth.
09-12-2019 01:27 PM
09-16-2019 08:17 AM - edited 09-16-2019 08:17 AM
@DaveChomi That's very interesting I guess it really depends on what your deployment looks like. We do more of a multi-tenanted carrier deployment in which RTP would never leave the customer LAN for internal calls and for external calls it would always go from the Customer LAN <-> MS <-> us. If you've deployed an enterprise model, where your SBC is deployed within your LAN there really is no point in flinging media packets all the way to MS just to receive them back to your LAN and then send them out again via your SBC. I can see how that can have an impact on quality.
Can you share the syslog for a call like that from the AudioCodes SBC? I'd be very interested to see what its doing hone its trying to negotiate the call transfer leg.
09-20-2019 02:48 AM
I can provide later the syslog, just do not want to publish all our IP addresses etc. on forum
Currently if I set our SBC to handle all REFERs locally it simply takes REFER from Teams, SBC accept and sends new Invite to Teams. Teams takes that and create new SIP call where as source IP for media presents public IP of our SBC. This call will come as new SIP call to our SBC and SBC simply tries to connect media between private IP and public IP of our WAN interface. And this is failing basically because SBC sends the traffic to its own public IP through WAN interface out of the SBC. There is then firewall which is dropping that from obvious reason.
09-20-2019 02:51 AM
Screenshots are basically refering to call from +420702XXXXXX to Teams user on number +420544137XXX which was transffered to +420723XXXXXX.
New call is visible on SBC.
And on that second call the SBC refers to Teams hey the media are on my public IP and Teams refers hey and I have media on your public IP. :)
10-20-2019 11:27 PM
Hi, We are having a similar transfer issue and I see you have the Audiocodes syntax to remove this in the ALLOW header. Any chance you could send this across so I can try this?
'No, not by rejecting the REFER with a response code. I basically did a message manipulation rule to re-build the Allow header for calls to MS Teams without including REFER. Which vendor are you using? If you have an AudioCodes as well, I can give you the rule syntax.'
10-29-2019 02:03 AM
Have you setup on the opposite trunk (IP Profile for provider SIP trunk or your internal PBX not Teams side) Handle Locally of the REFER messages?
This will do the same job basically and you do not need this special manipulation which I personally consider as too much because it does not give you any chance then manipulate REFER messages.
So for IP profile of the SIP trunk towards your provider/on-prem PBX
I would setup these parameters on Audiocodes SBC
|Remote REFER Mode||Handle Locally|
|Remote Replaces Mode||Handle Locally|
|Play RBT To Transferee||Yes|
|Remote 3xx Mode||Handle Locally|
10-29-2019 02:08 AM
Hi, in meantime I got this resolved.
It's basically correct behavior that Microsoft sends public IP of my SBC as source of media on new call leg for that transferred call, while mediabypass is enabled. It just really require that you will setup enterprise firewall to communicate from NATed private IP to its own public IP and loopback this communication. This is what basically SBC does, Sends the media stream from its own private IP to its own public IP and expects the media back :) So it is very ugly hairpin on firewall even not documented in firewall prerequisites. I would say SBC should have some logic to handle this by itself by as long as this is the only way we need to make security guys believe this is absolutely normal request :)