I have a Copy Activity in my pipeline which is referring to a external storage account, takes a delta table from there and copies that into our storage account.
We've added the client ID's of all three SP's (DEV, QA & PROD) to the external storage account.
It's working for QA & PROD environment, but not for DEV.
In DEV we're getting the error as below.
Weird thing is that sometimes it copies a couple files (2 or 3) and copies that into our storage account, and sometimes it just fails right away.
ErrorCode=FileForbidden,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Failed to read a 'AzureDataLakeStore' file. Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Net.WebException,Message=The remote server returned an error: (403) Forbidden.,Source=System,'