ADF Copy files from filesystem to blob

%3CLINGO-SUB%20id%3D%22lingo-sub-2715841%22%20slang%3D%22en-US%22%3EADF%20Copy%20files%20from%20filesystem%20to%20blob%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2715841%22%20slang%3D%22en-US%22%3E%3CP%3Em%20new%20to%20ADF%20and%20even%20though%20I%20have%20created%20some%20simple%20pipelines%20before%2C%20this%20one%20is%20proving%20very%20tricky..%3C%2FP%3E%3CP%3EI%20have%20a%20Fileshare%20with%20files%20and%20pictures%20from%20jobs%20with%20the%20naming%20convention%3A%20%5BA%20or%20B%20for%20before%2Fafter%5D-%5BSite%20in%20numbers%5D-%5BWork%20number%5D.%5Bjpeg%20or%20jpeg%5D%3C%2FP%3E%3CP%3EI%20want%20to%20select%20only%20the%20pictures%20from%20the%20file%20share%20and%20copy%20them%20to%20my%20blob%20storage%20and%20I%20want%20to%20create%20a%20folder%20dynamically%20in%20the%20blob%2C%20for%20example%20taking%20the%20%5Bwork%20number%5D%20of%20the%20picture%20name%2C%20creating%20a%20folder%20with%20this%20number%2C%20and%20saving%20in%20that%20folder%20any%20pictures%20with%20that%20same%20work%20number.%3C%2FP%3E%3CP%3EI%20have%20successfully%20connected%20to%20my%20file%20share%20and%20blob%20and%20I%20have%20successfully%20created%20my%20datasets%20as%20binary%20and%20moved%20pictures%20across%20by%20typing%20the%20path%20and%20the%20file%20name%20in%20the%20copy%20activity%2C%20so%20the%20connectivity%20is%20there.%3C%2FP%3E%3CP%3EThe%20issue%20is%20that%20there%20are%20roughly%201%20million%20pics%20and%20I%20want%20to%20automate%20this%20process%20with%20wildcards%2C%20but%20Im%20having%20a%20hard%20time%20with%20the%20dynamic%20expressions%20in%20ADF...%20any%20help%20with%20extracting%20and%20manipulating%20the%20name%20of%20each%20picture%20to%20achieve%20something%20like%20this%20would%20be%20appreciated!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2715841%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%20Data%20Factory%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EAzure%20ETL%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ECopy%20Activity%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Occasional Visitor

m new to ADF and even though I have created some simple pipelines before, this one is proving very tricky..

I have a Fileshare with files and pictures from jobs with the naming convention: [A or B for before/after]-[Site in numbers]-[Work number].[jpeg or jpeg]

I want to select only the pictures from the file share and copy them to my blob storage and I want to create a folder dynamically in the blob, for example taking the [work number] of the picture name, creating a folder with this number, and saving in that folder any pictures with that same work number.

I have successfully connected to my file share and blob and I have successfully created my datasets as binary and moved pictures across by typing the path and the file name in the copy activity, so the connectivity is there.

The issue is that there are roughly 1 million pics and I want to automate this process with wildcards, but Im having a hard time with the dynamic expressions in ADF... any help with extracting and manipulating the name of each picture to achieve something like this would be appreciated!

0 Replies