Jul 08 2024 03:41 AM
I have been fighting a very odd issue in a pipeline this morning and can reproduce the issue at will.
Imagine a simple pipeline with a copy data task using a SHIR.
For the source query, if I specify 32 columns in the query - it errors with a message that it cannot find the first column in the dataset.. If I specify 31 columns, or 33 columns its fine.. It doesn't matter what order the columns are in, doesn't matter what columns I choose, or the names, or whatever, but the moment I choose 32 columns - It errors with a message that it cannot find the first column (whatever I select as the first column)
Repro - Copy data task, source query =
SELECT 1 AS col1
, 2 AS col2
, 3 AS col3
, 4 AS col4
, 5 AS col5
, 6 AS col6
, 7 AS col7
, 8 AS col8
, 9 AS col9
, 10 AS col0
, 11 AS col11
, 12 AS col12
, 13 AS col13
, 14 AS col14
, 15 AS col15
, 16 AS col16
, 17 AS col17
, 18 AS col18
, 19 AS col19
, 20 AS col20
, 21 AS col21
, 22 AS col22
, 23 AS col23
, 24 AS col24
, 25 AS col25
, 26 AS col26
, 27 AS col27
, 28 AS col28
, 29 AS col29
, 30 AS col30
, 31 AS col31
, 32 AS col32
Sink = no mapping, tablename passed in as part of foreach and auto-create table
Error:
Failure happened on 'Sink' side. ErrorCode=UserErrorInvalidColumnMappingColumnNotFound,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Column name 'col1' cannot be found in either source data or column mapping.,Source=Microsoft.DataTransfer.DataContracts,'
But in the example above, if I add a "col33", it works fine
Jul 10 2024 08:49 AM - edited Jul 10 2024 08:58 AM
Hello,
I am facing the exact same problem with the exact same diagnostic using the Azure Synapse pipelines.
As a workaround I duplicated a column to have 33 columns...
Jul 10 2024 09:08 AM
@kvppn Yes, exactly what I have done - bit of a bizarre bug in the stack! Hopefully someone from MS sees this and raises whatever support ticket to resolve
Jul 20 2024 11:31 PM