Forum Discussion
ADF not able to delete rows in Delta Lake
I have created at DataFlow that deletes rows in a Delta Lake. The Pipeline executes fine, but nothing has been removed from the delta lake. What am I doing wrong?
Are you referring to the correct version of the delta table after the delete ?
The data is still present in the older version and also physically present in the storage, see documentation links below -->
Important
https://docs.delta.io/latest/delta-update.html -->
delete removes the data from the latest version of the Delta table but does not remove it from the physical storage until the old versions are explicitly vacuumed. See vacuum for details.
https://docs.delta.io/latest/delta-utility.html#-delta-vacuum
The data is still present in the older version and also physically present in the storage, see documentation links below -->
Important
https://docs.delta.io/latest/delta-update.html -->
delete removes the data from the latest version of the Delta table but does not remove it from the physical storage until the old versions are explicitly vacuumed. See vacuum for details.
https://docs.delta.io/latest/delta-utility.html#-delta-vacuum