Simple modification of a web table

New Contributor

Hi there! 

N00b in ADF here but I'm trying to get data from a website. The connection works and I'm getting the table correctly.

Yggdrasill_0-1669119965661.png

 



However, due to the structure of the table the last row is a total row I need to get rid of. 

I tried using Dataflow and even Power Query but I can't connect the dataflow to the source data (the web table)

How do I do this in ADF? Do I have to pump the data first to a sink (eg. SQL) and THEN transform the data and again dump in SQL?

Bit confused here

1 Reply

@Yggdrasill, using a Data Flow, can you add a Surrogate Key transformation and then a Filter to remove the row with the max key value? Similar to the design proposed here.