Forum Discussion

AlexanderLe1615's avatar
AlexanderLe1615
Copper Contributor
Nov 21, 2023

Need help architecting an Azure Container Apps solution

Hello, everyone. I'm pretty new to this so I'd like some guidance. I looked at the ACA docs and I wasn't really able to find what I was looking for. I'm building this project solo and attempting to build this relatively 'simple' multi-container setup for the first time. Any help/insight/advice would be appreciated.

 

I need help architecting a system between 3 containers/services: 'connections', 'channels', and 'tasks'

 

Containers/Services
 
  1. 'connections' container: Users can connect via a WebSocket directly to a 'connections' container so that they can send/receive messages. The user may also choose to deploy a workload configuration as well.

  2. 'channels' container: This acts as a communication intermediary between 'connections' and 'tasks' services. It maps the users (via userId) to their various tasks (via taskId). A userId may be associated with many taskIds. If a user disconnects from 'connections', they may choose to reconnect and would still be able to view their ongoing tasks within the 'channels' container. A requirement for this system is bidirectional communication as the task should send progression updates to the user and the user should send confirmation response to the task.

  3. 'tasks' container: Awaits deployed workloads initiated by the user. when the user deploys a task, the 'tasks' service will pick it up and process it. 'tasks' is set to autoscaling because tasks may take quite a long time and therefore I may need to manage multiple.

Logical Flow

User <-- WebSocket --> 'connections'

'connections' <-- ??? --> 'channels'

'channels' <-- ??? --> 'tasks'

 

I have a some requirements/questions on how to best go about this.

Requirement 1: Bidirectional Communication. 'tasks' needs to send progression updates to the user. The user will also send confirmation/rejection response to tasks as well. channels has to orchestrate this bidirectional communication.

For 'task' --> user updates, I thought of using redis pubsub architecture where the user connection will subscribe to a redis channel on channels - and the task can publish messages to that channel. Is this a good solution?

For user --> 'tasks', maybe just direct messaging? However, this leads me to the next requirement...

Requirement 2: Tasks Autoscaling. 'tasks' will be autoscaling due to the long nature of processing lengthy tasks. Because of autoscaling, there may be many simultaneous spawned tasks replicas. When a user sends a confirmation response to the task, how will 'channels' know which task container to send the data to? I think channels will have to store a mapping of something like this:

 

mapping = {

  userId: {

    taskId1: taskContainer1,

    taskid2: taskContainer2

  }

}

Does this look appropriate? In addition, how can I get access to the container id from within the container if my task is running in a node environment?

Requirement 3: TypeScript. Currently, connections and tasks are written in TypeScript. I would also like channels to be written in TypeScript too, that way I can share types/interfaces between them.

Thanks for making it this far!

1 Reply

  • AlexanderLe1615 

     

    I would like to suggest Azure Web PubSub: https://learn.microsoft.com/en-us/azure/azure-web-pubsub/ it is practically your "connections"+"channel" containers because you need a central point to manage the bi-directionality of the communication.

     

    For the "task" container, I would like to suggest you change Azure Container Apps to Azure Function container, but I don't know if TypeScript will fit this service.

     

    If you want to be compliant with Azure Redis Pub/Sub, the Azure Function has a trigger for it but it is still in Preview ( Link ), and in this case, you lose the bi-directional communication acquired with the Azure Web Pub/Sub.