How to maintain bot state when using multiple gunicron workers?

smartSense 6

I have developed a chatbot using azure bot framework SDK, it is using python aiohttp and asyncio to handle requests.
It is deployed on the azure app service and is running through gunicron, but its ability to handle multiple users is low it can only handle 3-4 users per second, I like to increase its capacity.

To do so I did some research and found that with multiple gunicron workers we can handle more users. But when I am using 3 workers instead of 1 on one CPU core, the bot's state is not maintained and it again asks the same question which user has already answered. I am using waterfall dialog for my bot.

I tried to have a JSON file that maintains the user state in local, but that is also not working.

Please help me with the issue as there are at least 1000 users per second is forecasted.

NOTE:- Scaling vertically does not resolve the issue and it does not increase capacity to handle even 100 users per sec. I am a beginner at bot framework and azure.

Any help would be appreciated.

Thank you

romungi-MSFT 42,311 Reputation points Microsoft Employee

2022-02-01T12:53:39.773+00:00

@smartSense For the app service used by your bot did you try horizontal scaling based on a metric?
You can navigate to metrics tab and check all the metrics that could be causing a constraint for maintaining high users/second. Based on this metric if you add a scale out rule at about 70-80% of the value.
If this is a permanent requirement irrespective of the usage then you can choose manual scale and directly mention higher number of instances and check if this works.

With regards to making changes to the settings of gunicorn I would recommend to post more details of the changes on the bot sdk repo by raising an issue to get advise from the development team on bot framework SDK.