How to scale an Azure Function to handle 10,000+ requests per second

%3CLINGO-SUB%20id%3D%22lingo-sub-3246888%22%20slang%3D%22en-US%22%3EHow%20to%20scale%20an%20Azure%20Function%20to%20handle%2010%2C000%2B%20requests%20per%20second%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3246888%22%20slang%3D%22en-US%22%3E%3CP%3EHello%20all%2C%3CBR%20%2F%3E%3CBR%20%2F%3EFirstly%20I'm%20sorry%20if%20this%20question%20is%20too%20broad%20or%20not%20suitable.%3CBR%20%2F%3E%3CBR%20%2F%3EI'm%20looking%20for%20advise%20on%20how%20to%20scale%20out%20a%20quite%20simple%20operation%20to%20a%20huge%20amount%20of%20requests%20per%20second.%20It's%20a%20simple%20operation%2C%20a%20GET%20endpoint%20in%20API%20Manager%2C%20which%20calls%20a%20function%20app%2C%20which%20gets%20a%20simple%20item%20from%20CosmosDB.%20But%20I've%20never%20done%20it%20at%20such%20a%20huge%20scale%20before.%20I%20know%20that%20if%20I%20test%20this%20function%20now%20with%2010%2C000%20requests%20it's%20probably%20going%20to%20blow%20up.%20There%20may%20be%20a%20situation%20where%20when%20we%20release%20the%20project%2C%20there'll%20be%20over%20100%2C000%20requests%20a%20second.%3CBR%20%2F%3E%3CBR%20%2F%3ESo%20from%20my%20basic%20knowledge%20of%20cloud%20computing%2C%20I'm%20looking%20for%20a%20load%20balancer%20that%20when%20the%20amount%20of%20request%20gets%20larger%2C%20it'll%20spin%20up%20more%20virtual%20machines%20to%20handle%20the%20incoming%20traffic.%20I%20had%20a%20read%20of%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fazure.microsoft.com%2Fen-gb%2Fblog%2Fprocessing-100-000-events-per-second-on-azure-functions%2F%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3Ehttps%3A%2F%2Fazure.microsoft.com%2Fen-gb%2Fblog%2Fprocessing-100-000-events-per-second-on-azure-functions%2F%3C%2FA%3E%26nbsp%3Band%20it%20seems%20good%2C%20a%20bit%20over%20my%20head%20though%20lol.%3CBR%20%2F%3E%3CBR%20%2F%3EI%20was%20given%20the%20option%20of%20doing%20this%20any%20way%20I%20wanted%20but%20I%20wanted%20to%20stick%20with%20.NET%20and%20Azure%20as%20it's%20been%20working%20out%20so%20well%20for%20me.%20I'm%20sure%20this%20is%20possible%2C%20I%20just%20don't%20see%20all%20the%20moving%20parts%20yet.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3247366%22%20slang%3D%22en-US%22%3ERe%3A%20How%20to%20scale%20an%20Azure%20Function%20to%20handle%2010%2C000%2B%20requests%20per%20second%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3247366%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F737066%22%20target%3D%22_blank%22%3E%40Jiadong_Chen%3C%2FA%3E%26nbsp%3BThank%20you!%20Those%20articles%20were%20very%20enlightening%20%3A)%3C%2Fimg%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3246919%22%20slang%3D%22en-US%22%3ERe%3A%20How%20to%20scale%20an%20Azure%20Function%20to%20handle%2010%2C000%2B%20requests%20per%20second%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3246919%22%20slang%3D%22en-US%22%3EHere%20is%20a%20blog%20post%20should%20be%20helpful.%3CBR%20%2F%3E%3CA%20href%3D%22https%3A%2F%2Fmadeofstrings.com%2F2019%2F01%2F09%2Fscaling-azure-functions-to-make-500000-requests-to-weather-com-in-under-3-minutes%2F%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fmadeofstrings.com%2F2019%2F01%2F09%2Fscaling-azure-functions-to-make-500000-requests-to-weather-com-in-under-3-minutes%2F%3C%2FA%3E%3CBR%20%2F%3Eand%20Azure%20Load%20Testing%20may%20be%20helpful%20as%20well.%3CBR%20%2F%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-au%2Fazure%2Fload-testing%2Foverview-what-is-azure-load-testing%3FWT.mc_id%3DDT-MVP-5001664%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-au%2Fazure%2Fload-testing%2Foverview-what-is-azure-load-testing%3FWT.mc_id%3DDT-MVP-5001664%3C%2FA%3E%3C%2FLINGO-BODY%3E
New Contributor

Hello all,

Firstly I'm sorry if this question is too broad or not suitable.

I'm looking for advise on how to scale out a quite simple operation to a huge amount of requests per second. It's a simple operation, a GET endpoint in API Manager, which calls a function app, which gets a simple item from CosmosDB. But I've never done it at such a huge scale before. I know that if I test this function now with 10,000 requests it's probably going to blow up. There may be a situation where when we release the project, there'll be over 100,000 requests a second.

So from my basic knowledge of cloud computing, I'm looking for a load balancer that when the amount of request gets larger, it'll spin up more virtual machines to handle the incoming traffic. I had a read of https://azure.microsoft.com/en-gb/blog/processing-100-000-events-per-second-on-azure-functions/ and it seems good, a bit over my head though lol.

I was given the option of doing this any way I wanted but I wanted to stick with .NET and Azure as it's been working out so well for me. I'm sure this is possible, I just don't see all the moving parts yet.

2 Replies

@Jiadong_Chen Thank you! Those articles were very enlightening :)