Scheduled trigger is running 1second early
A scheduled trigger configured to execute every 10 minutes exhibits an issue where one of the six runs in every hour executes 1 second earlier than the expected schedule. This inconsistency impacts the timing accuracy of the trigger. My query is why is it running 1 second early? Due to this we are having issues skipping of few Jobs scheduled during that time.ADF copy data with pagination headers not allowing dynamic filename for sink.
ADF copy data with pagination headers not allowing dynamic filename for sink. In my ADF pipeline there is a single copy data activity with pagination headers to use pagination by using cursor parameter from body. But the source is working for api request but the sink is not allowing dynamic filenames and only storing single file with all data instead of multiple files based on cursor
How to create a spread/range of consecutive dates in Azure data factory in derived transformation
I want to create a date spread from 01-01-2005 till 23-01-2015, is it possible to populate such a dates range in the Azure data factory (specifically in mapping data flows). If yes then which function should one use to apply the same. Thank you!
filter out certain pattern date files
Hi, I want to filter out certain pattern date files in a particular folder and copy those files into data lake storage. filenames in source folder: EDL_infy_20201201_wex.csv EDL_infy_20201202_wex.csv EDL_infy_20201203_wex.csv EDL_infy_20201204_wex.csv EDL_infy_20201205_wex.csv EDL_infy_20201206_wex.csv EDL_infy_20201207_wex.csv EDL_infy_20201208_wex.csv EDL_infy_20201209_wex.csv EDL_infy_20201210_wex.csv EDL_infy_20201211_wex.csv EDL_infy_20201212_wex.csv EDL_infy_20201213_wex.csv EDL_infy_20201214_wex.csv EDL_infy_20201215_wex.csv EDL_infy_20201216_wex.csv and so on.. requirement is copy date pattern files from 20201201 to 20201210 into target folder. I used getmetadata activity and lookup and foreach activity i tried but not full fill the solution..can you please help me out. Thanks, SubbaUsing ADF Data Flow Derived Column transform against nested Delta structures
I'm trying to use a derived column transform within an ADF Data Flow where I've ingested a Delta table with nested structures. I'm struggling with the syntax needed to flatten out these structures and no column info is displayed despite me being able to preview the data. Such a structure would be: { "ContactId":"1002657", "Name":{ "FirstName":"Donna", "FullName":"Donna Brittain", "LastName":"Brittain" } } Data Preview working OK: The structure of my Delta table: The error I'm getting trying to reference a nested column: How can I reference a nested column such as Name.FirstName to flatten it out to FirstName and why is it not showing up in any of the mappings?
Manual Backup Cosmos DB
Hi, Tried to export data in CosmosDB but it was not successful. According to https://docs.microsoft.com/en-us/azure/cosmos-db/storage-explorer, by using this tool I can export the data inside the cosmosdb, but no option to export. Tried to do the instructions here https://azure.microsoft.com/en-us/updates/documentdb-data-migration-tool/ and https://docs.microsoft.com/en-us/azure/cosmos-db/import-data#JSON, but error is being encountered. Can you help me how to do this in Data Factory or any steps just to manual backup cosmos DB? Thank you.
