After an in-place upgrade to Skype for Business Server 2015 Response Group Workflows Fail to work
Published May 20 2019 05:22 PM 7,760 Views
Microsoft
First published on TECHNET on Apr 10, 2017
In a peculiar case, we noticed that after an in-place upgrade from Lync Server 2013 to Skype for Business Server 2015, Response Group Workflows stopped working. The initial troubleshooting involved verifying (i) that the Response Group Service is starting, (ii) the work-flow is defined in Skype for Business Server 2015 after the upgrade, (iii) the AD Object for the workflow exists.

To troubleshot further, as a simple step, we deleted and re-created the workflow, and could reproduce the error. From the logs on client, we could see 500 Internal Server Error , so I wanted to first make sure the permissions on AD are set correctly based on Issue: Calls to certain Response groups fail with an Error "500 Internal Server Error" . Once I could verify that the issue was indeed not to do with AD permissions, I started to investigate further.

Looking through the built-in scenarios, a scenario called RGS exists, but I wasn’t sure, if the components would suffice.













Description Centralized Logging Tool Scenario Logging Tool Components
Response Group Service RGS RgsClientsLib

I decided to create a custom scenario, call it RGSCustom and perform logging. I used the below commands to create my custom scenario

$SIPStack = New-CsClsProvider -Name "SIPStack" -Type "WPP" -Level "Debug" -Flags "All"
$S4 = New-CsClsProvider -Name "S4" -Type "WPP" -Level "Debug" -Flags "All"
$RGSClientLib = New-CsClsProvider -Name "RGSClientLib" -Type "WPP" -Level "Debug" -Flags "All"
$RgsCommonLibrary = New-CsClsProvider -Name "RgsCommonLibrary" -Type "WPP" -Level "Debug" -Flags "All"
$RgsDatastores = New-CsClsProvider -Name "RgsDatastores" -Type "WPP" -Level "Debug" -Flags "All"
$RgsDeploymentApi = New-CsClsProvider -Name "RgsDeploymentApi" -Type "WPP" -Level "Debug" -Flags "ALL"
$RgsDeploymentLibrary = New-CsClsProvider -Name "RgsDeploymentLibrary" -Type "WPP" -Level "Debug" -Flags "ALL"
$RgsDiagnostics = New-CsClsProvider -Name "RgsDiagnostics" -Type "WPP" -Level "Debug" -Flags "All"
$RgsHostingFramework = New-CsClsProvider -Name "RgsHostingFramework" -Type "WPP" -Level "Debug" -Flags "ALL"
$RgsMatchMakingService = New-CsClsProvider -Name "RgsMatchMakingService" -Type "WPP" -Level "Debug" -Flags "All"
$RgsDBSyncAgent = New-CsClsProvider -Name "RgsDBSyncAgent" -Type "WPP" -Level "Debug" -Flags "ALL"


New-CsClsScenario -Identity "Global/RGSCustom" -Provider @{Add=$SIPStack,$S4,$RGSClientLib,$RgsCommonLibrary,$RgsDatastores,$RgsDeploymentApi,$RgsDeploymentLibrary, $RgsDiagnostics, $RgsHostingFramework, $RgsMatchMakingService, $RgsDBSyncAgent}


Next it was time to start logging and while we were logging to restart the Response Group Service, so that I would have any failure encountered during service state captured

Start-CsClsLogging -Scenario "RGSCustom" -Pools SFBSTD01.contoso.com
Stop-CsClsLogging -Scenario "RGSCustom"
Search-CsClsLogging -Pools SFBSTD01.contoso.con -OutputFilePath "C:\Windows\Temp\RGSCustom.txt" -StartTime (get-date).AddMinutes(-30)




As I was investigating the CLS logs, I noticed the a peculiar error Message: CALLCONTROL: Call declined because CallControl is not started

TL_INFO(TF_COMPONENT) [SFBSTD01\SFBSTD01] 208D8.4700::05/03/2016-19:46:01.484.0000A42D (RgsCommo...


sip:sritodi@contoso.com
To: sip:sampleworkflow@contoso.com
Message: CALLCONTROL: Call declined because CallControl is not started
WorkflowId: c338e8ba142042b9b30023269d29daa0
_rgs_message_end_

I decided to look through the event logs when the service was starting and noticed EVENT ID 31067

clip_image002

This sounded a little strange as I was aware that every Lync Server 2013 pool has 2 Application Contacts, one RGS Presence Watcher and another RGS Announcement Service. This event was speaking about 3 RGS Presence Watcher Contact objects.

I decided to query the number of RGS Presence Watcher contact objects

Get-CsApplicationEndpoint | Where-Object DisplayName -eq "RGS Presence Watcher" | Ft Identity, DisplayName, RegistrarPool


This could also be accomplished by running

Get-ADObject -Filter 'msRTCSIP-OwnerUrn -eq "urn:application:RGS"' -SearchBase 'CN=Configuration,DC=contoso,DC=Com' -Properties msRTCSIP-PrimaryUserAddress, displayName | ft msRTCSIP-PrimaryUserAddress, displayName –AutoSize




I quickly realized that for the pools that were upgraded from Lync Server 2013 to Skype for Business Server 2015 and where calls to response group were failing, only had a single RGS Presence Watcher contact object

Solution: We decided to republish the topology by running

Enable-CsTopology -Verbose


Once topology was republished, we could see 3 RGS Presence Watcher contact objects. A service restart to the RGS service, and calls started to work as expected.
Version history
Last update:
‎May 20 2019 05:22 PM
Updated by: