Forum Discussion
Root bot, skill bot and scaling
We had James check his data and found this. See if it helps. In the root bot:
- Double check and be 100% sure that you're using the SkillConversationIdFactory that is a part of the MS chatbot framework (NOT one that you may have created). It should have a IStorage constructor parameter that lets you pass in whatever storage you want to use to persist ids used with skills communication. You probably need to use the class that is given to you in the chatbot framework. (i.e. SkillConversationIdFactory that inherits from"Microsoft.Bot.Builder.Skills.SkillConversationIdFactoryBase")
- For the IStorage object used by SkillConversationIdFactory, If you are using some kind of in-memory only storage, (i.e. A ConcurrentDictionary or other MemoryStorage type object), that might be a problem. The code in SkillConversationIdFactory might not be persisting the conversation/skill ID lookup data (needed to talk with skills) into a place that other apps can read.
I have found some old MS examples that give a "demo" of how to use skills and shows a SkillConversationIdFactory that uses in-memory storage...which of course won't scale or work across different apps.
Our plan was to redeploy the 4.15 dialog root bot and dialog skill bot samples and scale out the instances to 2. We have been having some trouble overwriting the existing dialog root bot's web app with the 4.15 sample. I'll update again when we get this resolved and test out scaling.
We were not aware of the delivery mode option. Thank You for bringing that to our attention. A workaround is better than nothing 🙂
Yup, misery loves company 🙂
We also upgraded to 4.15.1 but it did not solve the problem (within our own application); we still see skills getting 404s when scaling above 1 root instance. We're going to try and see if server affinity or some kind of root bot shared state (i.e. db) options even available at all within the framework, but perhaps delivery mode is the only multi-root option.
Its also not yet clear whether the expectReply is built into the sender and receiver framework (i.e. handled automatically by the middleware or other libraries) or is something that will have to be manually coded to keep things synchronous.
FYI, a couple of more (semi-useful) references about delivery mode:
https://github.com/microsoft/botbuilder-dotnet/pull/5142
https://github.com/microsoft/botbuilder-dotnet/pull/5162
- By server affinity, are you referring to the ARR affinity setting in the app service? If so, we have already tried setting both the root bot and the dialog skill bot's settings to ON. It did not have any effect.
We initially thought the problem could be due to the way our bots are storing the conversation state. We were using memory as the default storage. We thought that because the scaled bot instances are actually referring to their own memory storage, it could be likely that the bot instances do not have any references to the required conversations states, causing replies to get lost. We switched to using a physical storage and store all of the bots' conversation states in CosmosDB. However, that did not have any impact as well.
Maybe we are doing it incorrectly. If you could try it out on your end, we would like to compare notes and see if we both get the same results.FYI. I made a few edits to my last reply message above..
- We always have had ARR turned on, but what technology is actually looking for and keeping track of what instance each cookie should be routed to is unclear to us at this point. Typically that is some kind of load balancer or http router...is that just part of the chatbot framework? If it is, it does not seem to be pinning requests correctly (or that cookie is not being passed around correctly).
- We use Cosmos to save conversational state, but whatever lookup operation is producing that 404, the data it wants seems to be pinned in memory for a single root instance. Like you, we were hoping there is something that can be shared between root bots to effectively make that data it is looking for available to all instances. Have not found it yet.
Our next steps are basically more investigation and trial/error....- I just wanted to update and affirm the following
-updating to 4.15 did not resolve the scaling out issue
-deliveryMode works as a workaround
