Copy Activity 403 Forbidden for a Selection of Files in a Folder

Hi All,

I have a Copy Activity in my pipeline which is referring to a external storage account, takes a delta table from there and copies that into our storage account.

We've added the client ID's of all three SP's (DEV, QA & PROD) to the external storage account.

It's working for QA & PROD environment, but not for DEV.

In DEV we're getting the error as below.

Weird thing is that sometimes it copies a couple files (2 or 3) and copies that into our storage account, and sometimes it just fails right away.

Any ideas?

ErrorCode=FileForbidden,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Failed to read a 'AzureDataLakeStore' file. Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Net.WebException,Message=The remote server returned an error: (403) Forbidden.,Source=System,'