Move Files in an Azure File Share After ETL

%3CLINGO-SUB%20id%3D%22lingo-sub-2062220%22%20slang%3D%22en-US%22%3EMove%20Files%20in%20an%20Azure%20File%20Share%20After%20ETL%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2062220%22%20slang%3D%22en-US%22%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20just%20began%20using%20Azure%20Data%20Factory.%26nbsp%3B%20I%20created%20a%20pipeline%20that%20imports%20data%20from%20files%20located%20in%20our%20Azure%20Storage%20Account%20into%20our%20Azure%20SQL%20Database.%26nbsp%3B%20Now%20I%20want%20to%20create%20a%20new%20folder%20in%20our%20storage%20account%20named%20with%20the%20date%20of%20the%20file%20and%20move%20the%20files%20to%20the%20new%20folder.%26nbsp%3B%20For%20example%3A%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EFrom%3A%3C%2FP%3E%3CP%3E%2FProcessing%3C%2FP%3E%3CP%3ETo%3A%3C%2FP%3E%3CP%3E%2FSourceDataFileArchive%2F2020-01-14%2F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20was%20able%20to%20do%20this%20with%20tasks%20in%20SSIS%2C%20but%20I%20don't%20see%20how%20to%20do%20this%20in%20Data%20Factory.%26nbsp%3B%20Would%20I%20need%20to%20create%20an%20Azure%20Function%20to%20do%20this%3F%26nbsp%3B%20Or%20something%20else%3F%26nbsp%3B%20Does%20someone%20have%20a%20link%20to%20an%20example%20solution%20for%20this%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EAny%20help%20would%20be%20greatly%20appreciated.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks%2C%3C%2FP%3E%3CP%3EJill%3C%2FP%3E%3C%2FLINGO-BODY%3E
Occasional Visitor

 

I just began using Azure Data Factory.  I created a pipeline that imports data from files located in our Azure Storage Account into our Azure SQL Database.  Now I want to create a new folder in our storage account named with the date of the file and move the files to the new folder.  For example:

 

From:

/Processing

To:

/SourceDataFileArchive/2020-01-14/

 

I was able to do this with tasks in SSIS, but I don't see how to do this in Data Factory.  Would I need to create an Azure Function to do this?  Or something else?  Does someone have a link to an example solution for this?

 

Any help would be greatly appreciated.

 

Thanks,

Jill

2 Replies

Hey @Jill_Claus,

If your filename has a date that you want to use or date from file metadata, use GetMetadata activity before your Copy activity and use the output of metadata activity(key from output array of metadata activity) in your sink dataset folder properties using the dynamic content editor.

Let me know if that works for you or you need more help.

Thanks,
Sunaina

Hola,

puedes utilizar actividad Data Factory "Copiar Datos". En receptor utilizar conjunto de datos Azure Blob Storage con parámetros. Mira en fotos:

iryna_YF_0-1614798840618.png

Script dinámico : @concat('Probar','/',variables('NombreFichero'),'/',string(formatDateTime(utcnow(),'yyyy-MM-dd')))

 

iryna_YF_1-1614798881054.png

iryna_YF_2-1614798897901.png

Resultado de 1 ejecución: se crea carpeta con nombre fichero ( o otro que pones en script) dentro de esta carpeta se crea carpeta con fecha y dentro de esta carpeta fichero (en el nombre del fichero también puedes agregar fecha):

iryna_YF_3-1614799082909.png

Un saludo!