Skype for Business hybrid - Issues between on-premise and Teams only users

Copper Contributor

Hi everyone,

 

I successfully configured Skype for Business hybrid between my on-premises Skype for Business environment and Skype for Business Online/Microsoft Teams and I migrated my first user into Teams with "Move-CsUser" command. However we have sereval issues when a purely on-premise user wants to cooperate with a migrated Teams Only user:

  • Calling from on-premise to Teams is working fine but when a Teams user calls a Skype for Business user, the connection will be unstable. The audio call builds up without any problem, the problems are starting when some "action" is starting (for example when a party turns on his/her camera, the call will be disconnected within 30 seconds).

  • Presence is on-way only so Teams users see the presence of SfB users but SfB users see Teams users as Offline.

I captured a SIP error when a call was disconnected, let me show you:

TL_INFO(TF_PROTOCOL) [2]1808.1034::10/08/2022-14:24:58.702.00000286 (SIPStack,SIPAdminLog::ProtocolRecord::Flush:ProtocolRecord.cpp(264))[1aabcb79-c13b-5ab0-861a-bfff80e0d52c] $$begin_record
 Instance-Id: 337
 Direction: incoming;source="external edge";destination="internal edge"
 Peer: sipfed.online.lync.com:5061;ip-address=52.112.192.139
 Message-Type: response
 Start-Line: SIP/2.0 504 Server time-out
 From: "" <sip:email address removed for privacy reasons>;epid=8a491aa5aa;tag=1a882c7292
 To: "on_prem_user"<sip:email address removed for privacy reasons>;epid=8A684AB9BC;tag=2650fd94b
 Call-ID: 413c925c-4b95-42c7-8e4a-3d191b3f6e92
 CSeq: 1 BYE
 Via: SIP/2.0/TLS 10.11.19.10:49950;branch=z9hG4bKCD1C9668.444BB43E0A1595DA;branched=FALSE;ms-internal-info="caH1kdKhQQbHPP4FE5G3t11vljWVJOIhFm0rSK52Yx8CU-tEtEI7jUKQAA";received=XXX.XXX.XXX.XXX;ms-received-port=49950;ms-received-cid=5DCA1300
 Via: SIP/2.0/TLS 10.11.11.40:61791;branch=z9hG4bK7D980A6C.576D89AA0A1B75DA;branched=FALSE;ms-received-port=61791;ms-received-cid=600
 Via: SIP/2.0/TLS 10.11.11.41:49940;branch=z9hG4bK65240C76.EB7556FD0A1595DA;branched=FALSE;ms-received-port=49940;ms-received-cid=11DC00
 Via: SIP/2.0/TLS 192.168.125.139:49803;received=XXX.XXX.XXX.XXX;ms-received-port=58708;ms-received-cid=900
 Content-Length: 0
 ms-diagnostics: 1010;reason="Certificate trust with another server could not be established";expected-fqdn="sfbedge.mydomain.com";certName="a-lgw-uswe-02.lgw.skype.com";cause="Possible server configuration issue";info="The peer certificate does not contain a matching FQDN";source="sipfed.online.lync.com";source-server="RD38563D7FA25E"
 ms-split-domain-info: ms-traffic-type=SplitIntra
 ms-telemetry-id: 1AABCB79-C13B-5AB0-861A-BFFF80E0D52C
 Server: RTC/7.0

How could I resolve the issue? I checked as many thing as I can: firewall (all neccessary ports are open inbound and ourbound), certificates, etc.

 

Thank you in advance,

Balint

52 Replies

@mbalint987 hello, we are having the exact same issue as you described, I can’t work out what’s causing it, please tell me you managed to fix and and if you did could you tell me how?

@Alias17273Unfortunately I had to give up. :\ I didn't find any useful information about this issue and because this is only a lab environment, I don't have to resolve this issue. It would be fine but it seems I cannot resolve this on my own.

@mbalint987 ah fair enough, lucky for you :) we have this in a production environment :( 

 

I'm going to raise a ticket with MS and if I fix it I’ll make sure I’ll reply here to let you know the fix

Thank you so much it would be great! My company has a premier support from Microsoft but the lab environment is not supported by this contract. :( If you can write me the solution that you got from support I would be happy!

@Alias17273 

 

Have you found a solution yet? we have the same Problem in production since today. same error in the ocslog.

 

another symptom is we can call from onprem to teams by sip without issues - but if we do it by the number the reverselookup fails.

 

rgds

@mbalint987 

 ms-diagnostics: 1010;reason="Certificate trust with another server could not be established";expected-fqdn="sfbedge.mydomain.com";certName="a-lgw-uswe-02.lgw.skype.com";cause="Possible server configuration issue";info="The peer certificate does not contain a matching FQDN";source="sipfed.online.lync.com";source-server="RD38563D7FA25E"
 

 

This indicates an issue with your certificates and the 30 seconds dropped calls usually indicates an issue with A/V certificate. So check your A/V certificate, remember a new certificate should be assigned using -roll as the already issued tokens are valid for up to 8 hours. 

 

Also check if your firewall does any deep inspection on the A/V connections, this could also be an issue with the same symptoms.

 

/Kenneth ML

@Kenneth Meyer-Lassen 

 

There were no such changes on our site. What also makes me curious:

 

ms-diagnostics: 1010;reason="Certificate trust with another server could not be established";expected-fqdn="external.edgefqdn.com";certName="c-lgw-euno-02.lgw.skype.com";cause="Possible server configuration issue";info="The peer certificate does not contain a matching FQDN";source="sipfed.online.lync.com";source-server="RD38563D80164B"

 

 

expected-fqdn=is our external edge name but certName="c-lgw-euno-02.lgw.skype.com" is something on the MS side. 

 

there we no changes on our site - except for 10/22 Windows Updates and the October SfB19 Update. The Updates were done on Wednesday last week - but the issue appeared today.

It is a bit strange, I agree, but I know there usually is an explaination.

The diagnostic string tells us following:
source="sipfed.online.lync.com" = This message is from external party
expected-fqdn="external.edgefqdn.com" = This is your Edge server external AV interface. Right??

Did you verify the AV cert or do you assume it is fine??
/Kenneth ML
Yes - i checked the certificate - ist still valid till June 2023. Also tried to remove and to reapply it - still the same issue.

"expected-fqdn" equals to the EDGE "Access Edge service" FQDN - not the AV Edge Service.
I have also tried to connect to all three URLs (Edge Service, Web Conf and A/V) with openssl s_client - correct certificate is presented and valid till 06/23. Its a concolidated EDGE with 3 public IPs.

I have also run https://testconnectivity.microsoft.com/tests/OnPremisesSfB/input with A/V Test enabled - everything seems fine. The SSL/TLS for the same url as in "expected-fqdn" is validated fine for TLS 1.0, TLS 1.1 and TLS 1.2.

@Kenneth Meyer-Lassen In the meantime, I tried to continue the troubleshooting, and I found out the following:

 

It seems that the Skype for Business client disconnects the call because we don't get any answer for our OK mesage.

 

Let me show you these screenshots:

 

254003-1.png254004-2.png

 

Do you have any ideas about why don't we get ACK for the 200 OK? I checked the network with no luck and I don't have any more ideas.

i can confirm the missing ACK:

 

https://data.dieseltechnic.com/public/file/M2sRV8-YY06s6YGtMDoIsw/2022-11-01%2023_08_38-Window.png 

 

  

whats your CU on SFB19? Could 

KB4470124

Version:

2046.409

 

Date Published:

10/4/2022

 

Be the cause of the issue?

@mbalint987

@EduNic1800 Let me paste the output of the relevant cmdlets:

 

snip.PNG

 

Before I ran into this issue we used an around 1.5 years old version of SfB 2019 but after that I installed the lastest updates on the Standard Edition and Edge.

Seems you dont have the October Update(7.0.2046.409) installed, so i dont think it is caused by this CU.

Its the same for us. Federation was fine for like 12Month and suddenly its showing this behavior - except for us it is in production which breaks communication between 100Teams and 550 Onprem Users.

It would be interesting to know if @Alias17273 have found a solution.
Interesting. I haven't seen this behaviour previously and I don't see it at a customer site (yet).

It could still indicate a problem with setting up the media path as the ACK should be sent when media path is established (late media).

Ye maybe the media path changed without documentation?!

I have already tried with "ANY-Port/Protocol" for incoming and outgoing for the EDGE-Part - same issue.

 

Also i dont think its media (only) because we cannot see the peresence of the Teams users from onprem. But its working if i query an onprem user from Teams.

Same issue here, presence is one-way, SfB on-premise users see Teams only users as offline users.
The strange thing with the call issue is that the audio connection will be built up without any issue, I can hear the remote party, he/she can hear me but the call will be dropped after 30 sec.
which region are you? We are Europe/West Europe and got the issue like 4 weeks after you. Maybe some rolling release/changes on the Teams Side?

As far as I can see we're in Europe/EU but I don't see any option to get the exact region. I'm afraid that this is an issue on Teams side.

I think so.

i have noticed some changes on the mediaflow in the last weeks. Like a reinvite from teams after a call was established and other ICE related changes.

Which already caused some issues with onprem devices (like dect) but never affected the "native" SfB Client.