Forum Discussion
Need help architecting an Azure Container Apps solution
Hello, everyone. I'm pretty new to this so I'd like some guidance. I looked at the ACA docs and I wasn't really able to find what I was looking for. I'm building this project solo and attempting to build this relatively 'simple' multi-container setup for the first time. Any help/insight/advice would be appreciated.
I need help architecting a system between 3 containers/services: 'connections', 'channels', and 'tasks'
'connections' container: Users can connect via a WebSocket directly to a 'connections' container so that they can send/receive messages. The user may also choose to deploy a workload configuration as well.
'channels' container: This acts as a communication intermediary between 'connections' and 'tasks' services. It maps the users (via userId) to their various tasks (via taskId). A userId may be associated with many taskIds. If a user disconnects from 'connections', they may choose to reconnect and would still be able to view their ongoing tasks within the 'channels' container. A requirement for this system is bidirectional communication as the task should send progression updates to the user and the user should send confirmation response to the task.
'tasks' container: Awaits deployed workloads initiated by the user. when the user deploys a task, the 'tasks' service will pick it up and process it. 'tasks' is set to autoscaling because tasks may take quite a long time and therefore I may need to manage multiple.
User <-- WebSocket --> 'connections'
'connections' <-- ??? --> 'channels'
'channels' <-- ??? --> 'tasks'
I have a some requirements/questions on how to best go about this.
Requirement 1: Bidirectional Communication. 'tasks' needs to send progression updates to the user. The user will also send confirmation/rejection response to tasks as well. channels has to orchestrate this bidirectional communication.
For 'task' --> user updates, I thought of using redis pubsub architecture where the user connection will subscribe to a redis channel on channels - and the task can publish messages to that channel. Is this a good solution?
For user --> 'tasks', maybe just direct messaging? However, this leads me to the next requirement...
Requirement 2: Tasks Autoscaling. 'tasks' will be autoscaling due to the long nature of processing lengthy tasks. Because of autoscaling, there may be many simultaneous spawned tasks replicas. When a user sends a confirmation response to the task, how will 'channels' know which task container to send the data to? I think channels will have to store a mapping of something like this:
mapping = {
userId: {
taskId1: taskContainer1,
taskid2: taskContainer2
}
}
Does this look appropriate? In addition, how can I get access to the container id from within the container if my task is running in a node environment?
Requirement 3: TypeScript. Currently, connections and tasks are written in TypeScript. I would also like channels to be written in TypeScript too, that way I can share types/interfaces between them.
Thanks for making it this far!
1 Reply
I would like to suggest Azure Web PubSub: https://learn.microsoft.com/en-us/azure/azure-web-pubsub/ it is practically your "connections"+"channel" containers because you need a central point to manage the bi-directionality of the communication.
For the "task" container, I would like to suggest you change Azure Container Apps to Azure Function container, but I don't know if TypeScript will fit this service.
If you want to be compliant with Azure Redis Pub/Sub, the Azure Function has a trigger for it but it is still in Preview ( Link ), and in this case, you lose the bi-directional communication acquired with the Azure Web Pub/Sub.