One of our ADF pipelines started to fail with the following error
ErrorCode=UserErrorFileNotFound,"Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ADLS Gen2 operation failed for: Operation returned an invalid status code "NotFound". Account: "XXXprodXXa". FileSystem: "tempdir". Path: "SQL/XXX.json". ErrorCode: "PathNotFound". Message: "The specified path does not exist.". RequestId: "0000". TimeStamp: "Thu, 26 Jan 2023 01:31:31 GMT".,Source=Microsoft.DataTransfer.ClientLibrary,""Type=Microsoft.Azure.Storage.Data.Models.ErrorSchemaException,Message=Operation returned an invalid status code "NotFound",Source=Microsoft.DataTransfer.ClientLibrary,"
the pipeline try to create a folder on Azure Data Storage (ADLS Gen2) but it failed to create the folder so the next flow cannot locate the folder. The pipeline has been working for 2 months, then it stop working yesterday afternoon. It happened intermittently with no known pattern.
Anyone experiencing this and provide any insight is much appreciated.
