Forum Discussion
Solved
Logic Apps: how to pull large volumes of data from a REST API
Hi everyone, I'm pulling from a REST API that only allows 1000 rows at a time. This works for a single pull. My goal is to pull all 50k rows and dump it to a log analytics table. Can you please s...
SocInABox It depends if the REST API you are calling allows you to skip a certain number of items. If it does, you will need to keep track of which group of 1000 you are calling. Then, in a loop, call the REST API making sure to skip the correct amount. Repeat until you get less than 1000 items returned. Pseudo code would be
Initialize variable, SKIPAMOUNT, to zero Do until amountReturned<1000 Call REST API passing value from SKIPAMOUNT as the amount to skip Add 1000 to SKIPAMOUNT Process returned values
SocInABox It depends if the REST API you are calling allows you to skip a certain number of items. If it does, you will need to keep track of which group of 1000 you are calling. Then, in a loop, call the REST API making sure to skip the correct amount. Repeat until you get less than 1000 items returned. Pseudo code would be
Initialize variable, SKIPAMOUNT, to zero
Do until amountReturned<1000
Call REST API passing value from SKIPAMOUNT as the amount to skip
Add 1000 to SKIPAMOUNT
Process returned values
Thank you Gary! We will try this and let you all know how it went.
Like you said not all APIs are created equal so this is very helpful.
Like you said not all APIs are created equal so this is very helpful.