Forum Discussion

aolcot24's avatar
aolcot24
Copper Contributor
Jul 08, 2024

Odd bug when selecting 32 columns in source query of copy data task

I have been fighting a very odd issue in a pipeline this morning and can reproduce the issue at will.

Imagine a simple pipeline with a copy data task using a SHIR.

For the source query, if I specify 32 columns in the query - it errors with a message that it cannot find the first column in the dataset.. If I specify 31 columns, or 33 columns its fine.. It doesn't matter what order the columns are in, doesn't matter what columns I choose, or the names, or whatever, but the moment I choose 32 columns - It errors with a message that it cannot find the first column (whatever I select as the first column)

Repro - Copy data task, source query = 

SELECT 1 AS col1
	, 2 AS col2
	, 3 AS col3
	, 4 AS col4
	, 5 AS col5
	, 6 AS col6
	, 7 AS col7
	, 8 AS col8
	, 9 AS col9
	, 10 AS col0
	, 11 AS col11
	, 12 AS col12
	, 13 AS col13
	, 14 AS col14
	, 15 AS col15
	, 16 AS col16
	, 17 AS col17
	, 18 AS col18
	, 19 AS col19
	, 20 AS col20
	, 21 AS col21
	, 22 AS col22
	, 23 AS col23
	, 24 AS col24
	, 25 AS col25
	, 26 AS col26
	, 27 AS col27
	, 28 AS col28
	, 29 AS col29
	, 30 AS col30
	, 31 AS col31
	, 32 AS col32

Sink = no mapping, tablename passed in as part of foreach and auto-create table

Error:
Failure happened on 'Sink' side. ErrorCode=UserErrorInvalidColumnMappingColumnNotFound,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Column name 'col1' cannot be found in either source data or column mapping.,Source=Microsoft.DataTransfer.DataContracts,'

But in the example above, if I add a "col33", it works fine

  • aolcot24 kvppn This is related to a product bug on SQL server family connectors. We have identified the root cause, fixed the bug and released to production regions. If you are using Selfhosted IR, you may have to wait for July new version release to get it fixed. Thanks!

  • kvppn's avatar
    kvppn
    Copper Contributor

    Hello,

    I am facing the exact same problem with the exact same diagnostic using the Azure Synapse pipelines.
    As a workaround I duplicated a column to have 33 columns...

    • aolcot24's avatar
      aolcot24
      Copper Contributor

      kvppn Yes, exactly what I have done - bit of a bizarre bug in the stack! Hopefully someone from MS sees this and raises whatever support ticket to resolve

       

Resources