Forum Discussion
Solved
Running server-less in azure?
Hello, i have developed a continuous running script/app that integrates two systems, pulling data out of one and posting into another. It is currently written in PowerShell, it has no UI, it just ne...
Interesting scenario, thanks for sharing AndrewX. Does your IDP ever run out of information? As in does the return url ever return an empty result? Is there a concern that if you change to every minute, more than one function could post the same information to the elasticsearch?
Depending on where you see this solution going, you might want to look at pushing this information into the Event Hub. Take this as a basis. Your situation is simpler now but the nice thing is once you are in the cloud you have more flexibility as to how to handle the information.
My suggestion though is to start with what you have and see if you can get it to run in Azure Functions. You will need something to start your function: http call, file drop or timer for example. Then once it is running you could try to just run continuously. My concern is after an unpredictable amount of time the function will stop without a clear reason why so you will need some mechanism to start it again. That is why I like the timer idea.
This is an interesting one though, and I for one would appreciate if you did a post on how you solve this. Cheers - Jeff
Hello chilberto
Thank you for this reply, this is very helpful. A little more context around the app, is that it is used to stream authentication logs from our IDP into Elasticsearch (equivalent of streaming the Azure Signin Logs from the Azure APIs to Elasticsearch). Each page of data it collects, has a "next url" link, to retrieve the next page of results. Because i am not using specific dates, any simply following the next url, the next url will ALWAYS exist and is infinite.
I could technically modify the script to pull a page of results and store the next url and stop. Then have a schedule trigger of "every minute" to pull the next stream of results starting from the "nexturl". This way it would act more like a start/stop on a recurring trigger. This might fit into the Azure Functions framework better.
Do you see any issue with that?
Interesting scenario, thanks for sharing AndrewX. Does your IDP ever run out of information? As in does the return url ever return an empty result? Is there a concern that if you change to every minute, more than one function could post the same information to the elasticsearch?
Depending on where you see this solution going, you might want to look at pushing this information into the Event Hub. Take this as a basis. Your situation is simpler now but the nice thing is once you are in the cloud you have more flexibility as to how to handle the information.
My suggestion though is to start with what you have and see if you can get it to run in Azure Functions. You will need something to start your function: http call, file drop or timer for example. Then once it is running you could try to just run continuously. My concern is after an unpredictable amount of time the function will stop without a clear reason why so you will need some mechanism to start it again. That is why I like the timer idea.
This is an interesting one though, and I for one would appreciate if you did a post on how you solve this. Cheers - Jeff
- Sometimes the IDP does run out of information, in which case, it simply returns 0 results and the next url doesn't change, so it simply makes the same request in the next iteration and will do over and over again until there is new data and a new next url. The org has enough activity and users, that i have only seen this happen occasionally.
Thanks for all your help.