Forum Discussion

Jill_Claus's avatar
Jill_Claus
Copper Contributor
Jan 14, 2021

Move Files in an Azure File Share After ETL

 

I just began using Azure Data Factory.  I created a pipeline that imports data from files located in our Azure Storage Account into our Azure SQL Database.  Now I want to create a new folder in our storage account named with the date of the file and move the files to the new folder.  For example:

 

From:

/Processing

To:

/SourceDataFileArchive/2020-01-14/

 

I was able to do this with tasks in SSIS, but I don't see how to do this in Data Factory.  Would I need to create an Azure Function to do this?  Or something else?  Does someone have a link to an example solution for this?

 

Any help would be greatly appreciated.

 

Thanks,

Jill

  • SLalwani's avatar
    SLalwani
    Copper Contributor
    Feb 01, 2021

    Hey Jill_Claus,

    If your filename has a date that you want to use or date from file metadata, use GetMetadata activity before your Copy activity and use the output of metadata activity(key from output array of metadata activity) in your sink dataset folder properties using the dynamic content editor.

    Let me know if that works for you or you need more help.

    Thanks,
    Sunaina

  • iryna_YF's avatar
    iryna_YF
    Copper Contributor
    Mar 03, 2021

    Hola,

    puedes utilizar actividad Data Factory "Copiar Datos". En receptor utilizar conjunto de datos Azure Blob Storage con parámetros. Mira en fotos:

    Script dinámico : @concat('Probar','/',variables('NombreFichero'),'/',string(formatDateTime(utcnow(),'yyyy-MM-dd')))

     

    Resultado de 1 ejecución: se crea carpeta con nombre fichero ( o otro que pones en script) dentro de esta carpeta se crea carpeta con fecha y dentro de esta carpeta fichero (en el nombre del fichero también puedes agregar fecha):

    Un saludo!

     

     

     

Resources