Pipeline REST filter query to Xero
Hi there, I am trying to import data from Xero using a pipeline and do so fine when I do not filter the REST request. But when I try to add a standard filter to the REST call, I receive the below error from Data Factory. Message=Column 'Date' contains an invalid value '/Date(1669680000000+0000)/'. Cannot convert '/Date(1669680000000+0000)/' to type 'DateTime' Here is my REST query / parameters: https://api.xero.com/api.xro/2.0/Invoices?where=Date >= DateTime(2022,11,11) Now I know this is an issue with Data Factory as if I replicate the REST call with POSTMAN or the Xero API Explorer it works perfectly and returns items. So, my question is, what exactly is Data Factory doing with the request to receive this error? As the call is correct (validated by Xero) but Data factory is somehow breaking the request. Thanks, Tucker
Pagination issues during rest api calls
Hi, I have a pipeline which basically firsts gets a access token by making a http request and once I have the token I'm doing a get request to an API with the access token as a header. As the REST API o/p is limited I need to make multiple requests for pagination so I did follow the steps mentioned here :https://docs.microsoft.com/en-us/azure/data-factory/connector-rest#pagination-support There are few problems with the NextPageUrl which i receive from the REST get request, The NextPageUrl starts with http and based on my experience if the Linked_service Baseurl does not start with HTTPS it does not accept the headers passed so I need to change the url from this "http://xyz.com/api/v1/?page=1" to "https://xyz.com/api/v1/?page=1" so i tried the replace expression but It says I cant write it this way "@replace('$.ConnectResponse.Metadata.Paging.NextPageURL', 'http', 'https')" or"@replace($.ConnectResponse.Metadata.Paging.NextPageURL, 'http', 'https')" Please advise how can I change the string? Secondly I though okay let me pass the page number manually but how do I add a number to the current page number and stop once it reaches the max? from my API O/p I get example TotalPages=3 and CurrentPage=1 number so I though I could pass the queryparameter in pagination like below key = QueryParameters.page and value =@add($.ConnectResponse.Metadata.Paging.NextPageURL, 1) ----- I've tried with quotes and without quotes --- and it does work as expected. Please advise how can I fix this? Lastly Is this how i pass a header authorization token in pagination? key = Headers['Authorization'] value =Bearer @{activity('Login').output.access_token} I mean this is how I pass the header parameters for the rest source so I'm assuming I just need to pass it the same way correct me if I'm wrong. Thank you in advance.