Forum Discussion
Azure Data Factory DFExecutor Error
One of the ADF jobs(source CosmosDB to target CosmosDB after transformation) keeps failing. The jobs has been 'simplified' to get the transformation to make it work - but the errors we get (when it doesn't work) are not helpful - so I'm wondering if there's a different approach I should take or different troubleshooting/tracing/debugging tools I should be employing
When we run the test data and when I sample the transformation/derived columns, everything looks fine. But when I run against the whole database (which is quite tiny) it gets stuck somewhere and I don't have any errors that I can work against/with.
We tried the debug flow and inspect feature and Data preview and also the schema drift on source and destination to avoid nulls but still get the same error. Any help is appreciated.
Operation on target JOINv1V2caseNumber failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: at Sink 'sinkToKSAllDataBackupCosmosDb': Errors encountered in bulk update API execution. Number of failures corresponding to exception of type: java.lang.RuntimeException = 15; FAILURE: java.lang.RuntimeException: Stored proc returned failure 404\n\tat com.microsoft.azure.documentdb.bulkexecutor.internal.BatchUpdater$1.call(BatchUpdater.java:199)\n\tat com.microsoft.azure.documentdb.bulkexecutor.internal.BatchUpdater$1.call(BatchUpdater.java:148)\n\tat com.microsoft.azure.documentdb.repackaged.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)\n\tat com.microsoft.azure.documentdb.repackaged.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)\n\tat com.microsoft.azure.documentdb.repackaged.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:","Details":"java.lang.Exception: Errors encountered in bulk update API execution. Number of failures corresponding to exception of type: java.lang.RuntimeException = 15; FAILURE: java.lang.RuntimeException: Stored proc returned failure 404\n\tat com.microsoft.azure.documentdb.bulkexecutor.internal.BatchUpdater$1.call(BatchUpdater.java:199)\n\tat com.microsoft.azure.documentdb.bulkexecutor.internal.BatchUpdater$1.call(BatchUpdater.java:148)\n\tat com.microsoft.azure.documentdb.repackaged.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)\n\tat com.microsoft.azure.documentdb.repackaged.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)\n\tat com.microsoft.azure.documentdb.repackaged.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecu"}