Forum Discussion

JasonYeung's avatar
JasonYeung
Brass Contributor
May 26, 2023

Copying data with Data Factory get Data Type can't be inferred error

Hi,

 

I'm trying to setup a Copy Data pipeline in Azure Data Factory to copy data to an Azure Table storage. The source data is from a third party company's API, which sends data in a JSON format.

 

When I tested the pipeline, I'm getting the error "Data type of column '' can't be inferred from 1st row of data, please specify its data type in mappings of copy activity or structure" (I've removed the name of the column).

 

What I think is happening is that the JSON doesn't have a value (or is null) for the column and Data Factory can't determine what the data type is. As the API can return blank and null values in the JSON, I want to explicitly define the data type in the mappings of the pipeline. I cleared and recreated the mappings, but I could never choose the data type. It would always be "Any (Any)":

 

I was wondering if there's an explicit wat to define the data type of the fields? The pipeline would eventually work if I exclude the fields that are null / blank. But this isn't too practical because some records do not have this information, but still need to be copied.

 

Jason

 

 

1 Reply

  • ynguyenluat's avatar
    ynguyenluat
    Copper Contributor

    I  have never use table storage but try using an intermediate blob storage or datalake with delimited text file  as sink before using table storage. In mapping you will be able to define the type

Resources