There is a WebJob (many instances may be) and Azure queue as an input, InputQueue. There is another temporary Azure queue, named BatchQueue. In InputQueue we have 1 object per message. WebJob must put objects from InputQueue to BatchQueue, one by one. Then WebJob must process BatchQueue. WebJob can check BatchQueue every time when WebJob puts new object in BatchQueue.
Processing of the BatchQueue is: get ALL objects from queue and pack it in single file and then cleanup BatchQueue.
I need to start BatchQueue processing when:
- BatchQueue length > MaxLength OR
- Time interval between last processing and Now() is more then TimeInterval
I think to save text file with time of last processing in the blob storage and rewrite it during every processing. It seems to be a bit tricky and strange.
I'm trying to find some another "out of the box" tool in the Azure for it... Could somebody point to some safe way? I mean when we have multiple instances of our WebJob.
