Forum Discussion

mke40's avatar
mke40
Copper Contributor
Jun 17, 2022

Azure Data factory Copy Data is dropping records

I have set up a copy data pipeline in Azure Datafactory. The table I'm copying from contains 1500 records. The copy data step only read 872 records. I've tried to find records in the input table that seem to have bad data but nothing seems to be excessive long text or the wrong data type.  I also looked for duplicate keys but I don't find any fields in the input record layout that indicate they are a key.  Does anyone have suggestions as to what I should look for or try to change to resolve the issue? 

 

 

azure data factory

2 Replies

Sort By
  • manishnarang's avatar
    manishnarang
    Copper Contributor
    Jul 19, 2022
    Hi, did you get any resolution for this?

    Have you tried writing it to a different place? like a CSV file? it says that from source 872 rows were picked up.. which seems to be an issue on the source side itself.

    Also, did you try data migration method instead of copy data activity?
    • ericpasztor's avatar
      ericpasztor
      Icon for Microsoft rankMicrosoft
      Dec 20, 2022

      mke40, to ensure that all data is copied to the sink SQL Database, try setting the enableSkipIncompatibleRow property to True (False by default and documented here). By default, rows will be skipped when the first instance of a duplicate is detected according to the sink table definition.

Resources