Logic App Http pagination deeper look, build custom paging wrapper without loop

Published Nov 04 2021 03:40 AM 713 Views

Introduction 

In this article I will talk how the Http action is doing the pagination on the json response.

there is some requirement to the http end point to be considered as valid pagination 

the response needs to be json

the response needs to have array named value and attribute named nextLink

Like the below diagram

Mohammed_Barqawi_0-1635756565970.png

The http action will check if the nextLink is exist and then make Get request to it 

It will stopped when there is no nextLink attribute in the response 

 

Note

The http method for the nextLink  will be always Get regardless of the http action.

 

Building the Wrapper for ResourceGraph Rest 

The problem 

You want to call https://management.azure.com/providers/Microsoft.ResourceGraph/resources

As in the documentation Resources - Resources - REST API (Azure Azure Resource Graph) | Microsoft Docs

 

It has unique way to handle paging, first the json structure is not as recommended by the standard http pagination also there is something called $skipToken that also need to be passed in the next call.

 

The solution

I have built wrapper logic app that will action as proxy and convert the ResourceGraph response to paging enabled response 

 

Mohammed_Barqawi_1-1635756719608.png

In your business logic app instead of calling the ResourceGraph Endpoint you will call the wrapper logic app 

 

Mohammed_Barqawi_0-1636001275717.png

 

By adopting the wrapper concept, you will avoid implementing the until loop pattern since you will utilize the built-in pagination inside the Http action  

 

How the wrapper is working 




Mohammed_Barqawi_0-1635847776684.png

 

 



Mohammed_Barqawi_3-1635756850874.png

 



1

Initialize the next Link variable by getting the value from triggerOutputs()['queries']?['nextLink']

Because it will come in the query string 

2

Initialize the request body option

This variable either has the Skiptoken or empty 

Sample value when it is not empty 

  "options": {

 "$skipToken" :"abc="   

  },


 

 

Mohammed_Barqawi_4-1635756927394.png

 

3

Creat the Http request that conset of the option (if it exist ) and the query 

4

Sent the request to the http end poit for the ResourceGraph 

 

Mohammed_Barqawi_5-1635756968811.png

 

5

Map the response to the standard pagination format 

And add the next Link attribute in case this is not the last request 


Note that I have a variable that hold the wrapper logic app url so I can build the next link 

 

 

How to use the solution 


Click on the button below to deploy the Logic app to your subscription  

 

Sample logic app templates in GitHub

Deploy to Azure: 

Mohammed_Barqawi_6-1635757172439.png

 

 

Cange the parameters

URL and Query

%3CLINGO-SUB%20id%3D%22lingo-sub-2907605%22%20slang%3D%22en-US%22%3ELogic%20App%20Http%20pagination%20deeper%20look%2C%20build%20custom%20paging%20wrapper%20without%20loop%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2907605%22%20slang%3D%22en-US%22%3E%3CH1%20id%3D%22toc-hId-1497236172%22%20id%3D%22toc-hId-1499047807%22%3E%3CSPAN%20style%3D%22font-weight%3A%20400%3B%22%3EIntroduction%26nbsp%3B%3C%2FSPAN%3E%3C%2FH1%3E%0A%3CP%3EIn%20this%20article%20I%20will%20talk%20how%20the%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fconnectors%2Fconnectors-native-http%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EHttp%20action%3C%2FA%3E%20is%20doing%20the%20%3CSTRONG%3Epagination%3C%2FSTRONG%3E%20on%20the%20json%20response.%3C%2FP%3E%0A%3CP%3Ethere%20is%20some%20requirement%20to%20the%20http%20end%20point%20to%20be%20considered%20as%20valid%20pagination%26nbsp%3B%3C%2FP%3E%0A%3CP%3Ethe%20response%20needs%20to%20be%20json%3C%2FP%3E%0A%3CP%3Ethe%20response%20needs%20to%20have%20%3CI%3Earray%20%3C%2FI%3Enamed%20%3CSTRONG%3Evalue%3C%2FSTRONG%3E%20and%20%3CI%3Eattribute%20%3C%2FI%3Enamed%20%3CSTRONG%3EnextLink%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3ELike%20the%20below%20diagram%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Mohammed_Barqawi_0-1635756565970.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F322150i737730B8E3E90F32%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Mohammed_Barqawi_0-1635756565970.png%22%20alt%3D%22Mohammed_Barqawi_0-1635756565970.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3EThe%20http%20action%20will%20check%20if%20the%20nextLink%20is%20exist%20and%20then%20make%20%3CSTRONG%3EGet%3C%2FSTRONG%3E%20request%20to%20it%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIt%20will%20stopped%20when%20there%20is%20no%20nextLink%20%3CEM%3Eattribute%3C%2FEM%3E%20in%20the%20response%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3ENote%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3EThe%20http%20method%20for%20the%20nextLink%26nbsp%3B%20will%20be%20always%20%3CSTRONG%3EGet%3C%2FSTRONG%3E%20regardless%20of%20the%20http%20action.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH1%20id%3D%22toc-hId--310218291%22%20id%3D%22toc-hId--308406656%22%3EBuilding%20the%20Wrapper%20for%20ResourceGraph%20Rest%26nbsp%3B%3C%2FH1%3E%0A%3CH2%20id%3D%22toc-hId-380343183%22%20id%3D%22toc-hId-382154818%22%3EThe%20problem%26nbsp%3B%3C%2FH2%3E%0A%3CP%3EYou%20want%20to%20call%20%3CA%20href%3D%22https%3A%2F%2Fmanagement.azure.com%2Fproviders%2FMicrosoft.ResourceGraph%2Fresources%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fmanagement.azure.com%2Fproviders%2FMicrosoft.ResourceGraph%2Fresources%3C%2FA%3E%3C%2FP%3E%0A%3CP%3EAs%20in%20the%20documentation%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fazureresourcegraph%2Fresourcegraph(2021-03-01)%2Fresources%2Fresources%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EResources%20-%20Resources%20-%20REST%20API%20(Azure%20Azure%20Resource%20Graph)%20%7C%20Microsoft%20Docs%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIt%20has%20unique%20way%20to%20handle%20paging%2C%20first%20the%20json%20structure%20is%20not%20as%20recommended%20by%20the%20standard%20http%20pagination%20also%20there%20is%20something%20called%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fazureresourcegraph%2Fresourcegraph(2021-03-01)%2Fresources%2Fresources%23queryrequestoptions%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3E%24skipToken%3C%2FA%3E%20that%20also%20need%20to%20be%20passed%20in%20the%20next%20call.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--1427111280%22%20id%3D%22toc-hId--1425299645%22%3E%3CSPAN%20style%3D%22font-weight%3A%20400%3B%22%3EThe%20solution%3C%2FSPAN%3E%3C%2FH2%3E%0A%3CP%3EI%20have%20built%20wrapper%20logic%20app%20that%20will%20action%20as%20proxy%20and%20convert%20the%20ResourceGraph%20response%20to%20paging%20enabled%20response%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22Mohammed_Barqawi_1-1635756719608.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F322151i170CF4E8FC3A2531%2Fimage-size%2Fmedium%3Fv%3Dv2%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22Mohammed_Barqawi_1-1635756719608.png%22%20alt%3D%22Mohammed_Barqawi_1-1635756719608.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3EIn%20your%20business%20logic%20app%20instead%20of%20calling%20the%20ResourceGraph%20Endpoint%20you%20will%20call%20the%20wrapper%20logic%20app%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22Mohammed_Barqawi_0-1636001275717.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F324114i59E109BAA7B9D612%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Mohammed_Barqawi_0-1636001275717.png%22%20alt%3D%22Mohammed_Barqawi_0-1636001275717.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EBy%20adopting%20the%20wrapper%20concept%2C%20you%20will%20avoid%20implementing%20the%20until%20loop%20pattern%20since%20you%20will%20utilize%20the%20built-in%20pagination%20inside%20the%20Http%20action%26nbsp%3B%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-1060401553%22%20id%3D%22toc-hId-1062213188%22%3EHow%20the%20wrapper%20is%20working%26nbsp%3B%3C%2FH2%3E%0A%3CP%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22Mohammed_Barqawi_0-1635847776684.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F323054i0633F8AC90924898%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Mohammed_Barqawi_0-1635847776684.png%22%20alt%3D%22Mohammed_Barqawi_0-1635847776684.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22Mohammed_Barqawi_3-1635756850874.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F322153iEB2A69ACDFDF0F30%2Fimage-size%2Fmedium%3Fv%3Dv2%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22Mohammed_Barqawi_3-1635756850874.png%22%20alt%3D%22Mohammed_Barqawi_3-1635756850874.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CTABLE%3E%0A%3CTBODY%3E%0A%3CTR%3E%0A%3CTD%3E%3CP%3E%3CSPAN%20style%3D%22font-weight%3A%20400%3B%22%3E1%3C%2FSPAN%3E%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%3E%3CP%3EInitialize%20the%20next%20Link%20variable%20by%20getting%20the%20value%20from%20triggerOutputs()%5B'queries'%5D%3F%5B'nextLink'%5D%3C%2FP%3E%0A%3CP%3EBecause%20it%20will%20come%20in%20the%20query%20string%26nbsp%3B%3C%2FP%3E%0A%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3CTR%3E%0A%3CTD%3E%3CP%3E%3CSPAN%20style%3D%22font-weight%3A%20400%3B%22%3E2%3C%2FSPAN%3E%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%3E%3CP%3EInitialize%20the%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fazureresourcegraph%2Fresourcegraph(2021-03-01)%2Fresources%2Fresources%23queryrequestoptions%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Erequest%20body%20option%3C%2FA%3E%3C%2FP%3E%0A%3CP%3EThis%20variable%20either%20has%20the%20Skiptoken%20or%20empty%26nbsp%3B%3C%2FP%3ESample%20value%20when%20it%20is%20not%20empty%26nbsp%3B%3CBR%20%2F%3E%3CP%3E%26nbsp%3B%26nbsp%3B%22options%22%3A%20%7B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%22%24skipToken%22%20%3A%22abc%3D%22%26nbsp%3B%26nbsp%3B%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%26nbsp%3B%7D%2C%3C%2FP%3E%3CBR%20%2F%3E%3CP%3E%3CSPAN%20style%3D%22font-weight%3A%20400%3B%22%3E%26nbsp%3B%3C%2FSPAN%3E%3C%2FP%3E%0A%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3C%2FTBODY%3E%0A%3C%2FTABLE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22Mohammed_Barqawi_4-1635756927394.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F322154i604538D3BA28612F%2Fimage-size%2Fmedium%3Fv%3Dv2%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22Mohammed_Barqawi_4-1635756927394.png%22%20alt%3D%22Mohammed_Barqawi_4-1635756927394.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CTABLE%3E%0A%3CTBODY%3E%0A%3CTR%3E%0A%3CTD%3E%3CP%3E3%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%3E%3CP%3ECreat%20the%20Http%20request%20that%20conset%20of%20the%20option%20(if%20it%20exist%20)%20and%20the%20query%26nbsp%3B%3C%2FP%3E%0A%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3CTR%3E%0A%3CTD%3E%3CP%3E4%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%3E%3CP%3ESent%20the%20request%20to%20the%20http%20end%20poit%20for%20the%20ResourceGraph%26nbsp%3B%3C%2FP%3E%0A%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3C%2FTBODY%3E%0A%3C%2FTABLE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22Mohammed_Barqawi_5-1635756968811.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F322155i8B88B1A5F2F4A3EC%2Fimage-size%2Fmedium%3Fv%3Dv2%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22Mohammed_Barqawi_5-1635756968811.png%22%20alt%3D%22Mohammed_Barqawi_5-1635756968811.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CTABLE%3E%0A%3CTBODY%3E%0A%3CTR%3E%0A%3CTD%3E%3CP%3E5%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%3E%3CP%3EMap%20the%20response%20to%20the%20standard%20pagination%20format%26nbsp%3B%3C%2FP%3E%0A%3CP%3EAnd%20add%20the%20next%20Link%20attribute%20in%20case%20this%20is%20not%20the%20last%20request%26nbsp%3B%3C%2FP%3E%3CBR%20%2F%3E%3CP%3ENote%20that%20I%20have%20a%20variable%20that%20hold%20the%20wrapper%20logic%20app%20url%20so%20I%20can%20build%20the%20next%20link%26nbsp%3B%3C%2FP%3E%0A%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3C%2FTBODY%3E%0A%3C%2FTABLE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--747052910%22%20id%3D%22toc-hId--745241275%22%3EHow%20to%20use%20the%20solution%26nbsp%3B%3C%2FH2%3E%0A%3CP%3E%3CBR%20%2F%3EClick%20on%20the%20button%20below%20to%20deploy%20the%20Logic%20app%20to%20your%20subscription%26nbsp%3B%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ESample%20logic%20app%20templates%20in%20%3CA%20href%3D%22https%3A%2F%2Fgithub.com%2Fmbarqawi%2Flogicappfiles%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EGitHub%3C%2FA%3E%3C%2FP%3E%0A%3CP%3EDeploy%20to%20Azure%3A%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fportal.azure.com%2F%23create%2FMicrosoft.Template%2Furi%2Fhttps%253A%252F%252Fraw.githubusercontent.com%252Fmbarqawi%252Flogicappfiles%252Fmain%252FLogicAppPaginationWrapper.json%22%20target%3D%22_self%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Mohammed_Barqawi_6-1635757172439.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F322156i953CE350D4C05082%2Fimage-size%2Fmedium%3Fv%3Dv2%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22Mohammed_Barqawi_6-1635757172439.png%22%20alt%3D%22Mohammed_Barqawi_6-1635757172439.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ECange%20the%20parameters%3C%2FP%3E%0A%3CP%3EURL%20and%20Query%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-2907605%22%20slang%3D%22en-US%22%3E%3CP%3ELogic%20app%20Http%20Pagination%20deeper%20look%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Mohammed_Barqawi_0-1635687445942.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F321941iFF9FA00CBE137700%2Fimage-size%2Fmedium%3Fv%3Dv2%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22Mohammed_Barqawi_0-1635687445942.png%22%20alt%3D%22Mohammed_Barqawi_0-1635687445942.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2907605%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3ELogic%20Apps%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Co-Authors
Version history
Last update:
‎Nov 03 2021 09:50 PM
Updated by: