Copy activity Rest API Pagination

Copper Contributor

I am ingesting data from Rest API source using Copy activity and need help in setting up Pagination.


Update - Increment should be 1 instead of 100 and now it works but now I want to see if I can do it only offset url and remove end condition. I want range condition to be dynamic as "RANGE:1:($.count/pagesize):1"  Is this possible?


Rest API source returns following response. 


status : 200

data: []

count: 156


It will always returns total count of records and based on it I need to read through different pages. By default only 100 records are returned each page. 

I tried this setup as mentioned in


It only got through first 100 records. I see the same behaviour when doing a Preview wherein second call is probably going through but response is empty. I feel probably the offset is not passed as expected which is causing the issue. Through postman I see responses for 2 pages and 3 rd page data array is empty.


0 Replies