Nov 22 2022
Hi there! N00b in ADF here but I'm trying to get data from a website. The connection works and I'm getting the table correctly.
However, due to the structure of the table the last row is a total row I need to get rid of. I tried using Dataflow and even Power Query but I can't connect the dataflow to the source data (the web table)How do I do this in ADF? Do I have to pump the data first to a sink (eg. SQL) and THEN transform the data and again dump in SQL?Bit confused here
Dec 20 2022
@Yggdrasill, using a Data Flow, can you add a Surrogate Key transformation and then a Filter to remove the row with the max key value? Similar to the design proposed here.