azure etl
47 TopicsUser Properties of Activities in ADF: How to add dynamic content in it?
On ADF, I am using a for each loop in which I am using an Execute Pipeline Activity which is getting executed for different iterations as per the values of the items provided to the For-Each Loop. I am stuck on a scenario which requires me to add the Dynamic Content Expression in the User Properties of individual activities of ADF. Specific to my case, I want to add the Dynamic Content Expression in the User Properties of Execute Pipeline Activity so that I get to individual runs of these activities on Azure Monitor with a specific label attached to it through its User Properties. The necessity to add the Dynamic Content Expression in the User Properties is due to the reason that each execution in respective iterations of these activities corresponds to a particular Step from a set of Steps configured for the Data Load Job as a whole, which has been orchestrated through ADF. To identify the association with the respective Job-Step, I require to add Dynamic Content Expression in its User Properties. Any sort of response regarding this is highly appreciated. Thank You!69Views1like1CommentAuto Create SQL table from Imported CSV in ADF?
Hi All Wondering if its possible to automatically create a sql table from an imported CSV in Data Factory? To make things a little more complicated not all csv's will have the same headings and differing number of headings. If so, can anyone point me in the direction of a how to guide? Case I've been ingesting csv's using a HTTP connector in ADF then storing CSV data into a SQL table (manually created) and then transforming and cleaning said data into a Datastore SQL table that is also manually created. I know I'm a little slow to the party but I've been looking at using parameters and was wondering if I pulled the csv's into blob storage and then transformed from there. Then I'd only need to create one SQL table and if I could automate the entire process in ADF that would save future me a lot of time. I have another issue but I'll post that separate as its a slightly different topic. Thanks for Reading Hope someone can point me in the right direction. JohnSolved6KViews1like1Comment