SOLVED

Using Flow with SharePoint Library - file copy executed before file upload is complete

%3CLINGO-SUB%20id%3D%22lingo-sub-319700%22%20slang%3D%22en-US%22%3EUsing%20Flow%20with%20SharePoint%20Library%20-%20file%20copy%20executed%20before%20file%20upload%20is%20complete%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-319700%22%20slang%3D%22en-US%22%3E%3CP%3ETrying%20to%20use%20Flow%20to%20move%20video%20files%20submitted%20to%20a%20%22dead%20drop%22%20folder%20into%20a%20controlled-access%20folder%20within%20a%20SharePoint%20Online%20document%20collection.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EEssentially%2C%20a%20user%20drags%20a%20video%20file%20onto%20the%20dead%20drop%20(displayed%20in%20a%20browser)%20to%20initiate%20the%20upload%20and%20it%20gets%20whisked%20away%20into%20a%20locked%20down%20folder%20afterward.%20There%20is%20also%20an%20email%20alert%20that%20seems%20to%20be%20working%20fine.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThe%20trigger%20%3CSTRONG%3EWhen%20a%20file%20is%20created%20(properties%20only)%3C%2FSTRONG%3E%20fires%20when%20the%20upload%20starts%2C%20so%20the%20file%20isn't%20ready%20to%20be%20copied%20when%20the%20Flow%20starts.%20If%20I%20insert%20a%20static%20%3CSTRONG%3EDelay%3C%2FSTRONG%3E%20that's%20long%20enough%20for%20the%20upload%20to%20complete%2C%20all%20is%20well.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThe%20problem%20is%20my%20files%20are%20potentially%20a%20few%20GB%20in%20size%20and%20the%20upload%20time%20will%20be%20highly%20variable.%20If%20the%20%3CSTRONG%3EDelay%3C%2FSTRONG%3E%20is%20too%20short%2C%20the%20remainder%20of%20the%20actions%20execute%20on%20the%20partially-uploaded%20file%2C%20with%20a%20%3CSTRONG%3EBadRequest%3C%2FSTRONG%3E%20error.%3C%2FP%3E%3CUL%3E%3CLI%3EIf%20I%20try%20a%20simple%20%3CSTRONG%3EMove%20file%3C%2FSTRONG%3E%20action%20in%20Flow%2C%20the%20action%20fails%20(status%20404%2C%20%22The%20file%20%3CEM%3E%2Fpath%2Fto%2Ffile.example%3C%2FEM%3E%20does%20not%20exist.%22)%20and%20the%20upload%20aborts%20as%20well.%3C%2FLI%3E%3CLI%3EIf%20I%20try%20a%20%3CSTRONG%3ECopy%20file%3C%2FSTRONG%3E%20action%20(to%20be%20followed%20by%20a%20%3CSTRONG%3EDelete%20file%3C%2FSTRONG%3E%20action)%20with%20%3CSTRONG%3EAsynchronous%20Pattern%3C%2FSTRONG%3E%20active%20and%20the%20%3CSTRONG%3ETimeout%3C%2FSTRONG%3E%20set%20to%2012h%2C%20the%20copy%20fails%20(status%20400%20%22The%20system%20cannot%20find%20the%20file%20specified.%22)%3C%2FLI%3E%3C%2FUL%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI've%20tried%20testing%20the%20%3CSTRONG%3EModified%3C%2FSTRONG%3E%20property%20of%20the%20target%20file%20in%20a%20%3CSTRONG%3EDo%20until%3C%2FSTRONG%3E%20loop%2C%20in%20the%20hope%20that%20the%20modification%20date%20and%20time%20are%20updated%20as%20the%20upload%20progresses%20and%20can%20thus%20be%20used%20as%20a%20progress%20counter%20-%20that's%20not%20working.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EFor%20the%20record%2C%20I've%20tried%20these%20other%20things%20with%20comparable%20results%3A%3C%2FP%3E%3CUL%3E%3CLI%3EUsing%20the%20%3CSTRONG%3EWhen%20a%20file%20is%20created%20in%20a%20folder%3C%2FSTRONG%3E%20trigger%3C%2FLI%3E%3CLI%3ESetting%20%3CSTRONG%3EConfigure%20run%20after%3C%2FSTRONG%3E%20option%20in%20subsequent%20actions%20to%20ensure%20that%20the%20previous%20%3CSTRONG%3ECopy%20File%3C%2FSTRONG%3E%20or%20%3CSTRONG%3EMove%20file%3C%2FSTRONG%3E%20action%20was%20completed%20before%20moving%20ahead%20with%20the%20next%20action.%3C%2FLI%3E%3C%2FUL%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ESo%20is%20there%20any%20way%20to%20test%20to%20see%20if%20the%20uploaded%20file%20is%20%22all%20there%22%20after%20my%20%3CSTRONG%3EWhen%20a%20file%20is%20created%20(properties%20only)%3C%2FSTRONG%3E%20trigger%20fires%3F%20Is%20there%20a%20property%20that%20gets%20applied%20only%20after%20the%20upload%20is%20complete%20that%20I%20can%20wait%20for%20or%20a%20better%20trigger%20to%20use%20to%20start%20the%20Flow%3F%20Is%20there%20just%20some%20better%20way%20to%20re-route%20large%20uploaded%20files%20into%20a%20locked-down%20folder%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-319700%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EDocument%20Library%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ESharePoint%20Online%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-321351%22%20slang%3D%22en-US%22%3ERe%3A%20Using%20Flow%20with%20SharePoint%20Library%20-%20file%20copy%20executed%20before%20file%20upload%20is%20complete%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-321351%22%20slang%3D%22en-US%22%3E%3CP%3EThat%20works%20perfectly.%26nbsp%3B%20Thanks%20very%20much!!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-321035%22%20slang%3D%22en-US%22%3ERe%3A%20Using%20Flow%20with%20SharePoint%20Library%20-%20file%20copy%20executed%20before%20file%20upload%20is%20complete%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-321035%22%20slang%3D%22en-US%22%3E%3CP%3EHi%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F267008%22%20target%3D%22_blank%22%3E%40DPapandria%3C%2FA%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThe%20key%20indicator%20as%20to%20whether%20your%20file%20has%20uploaded%20or%20not%20is%20the%20%3CSTRONG%3Efile%20size%3C%2FSTRONG%3E.%20Until%20the%20upload%20is%20complete%2C%20the%20file%20size%20will%20be%200.%20As%20soon%20as%20it's%20complete%20you'll%20see%20the%20actual%20size.%20I%20tested%20this%20on%20a%20400MB%20file.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ESo%20I%20have%20shaped%20a%20Flow%20to%20look%20like%20the%20following%3A%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3COL%3E%3CLI%3ETrigger%20-%20When%20a%20file%20is%20created%20(properties%20only)%3C%2FLI%3E%3CLI%3EAction%20-%20Initialise%20variable%3A%20Created%20a%20boolean%20called%20FileUploaded%20and%20a%20default%20value%20of%20false%3C%2FLI%3E%3CLI%3EControl%20-%20Do%20until%3A%20FileUploaded%20%3D%20true%3COL%3E%3CLI%3EAction%20-%20Get%20file%20metadata%20(this%20has%20to%20be%20done%20each%20time%20to%20ensure%20that%20we're%20looking%20at%20a%20fresh%20set%20of%20data%3C%2FLI%3E%3CLI%3ECondition%20-%26nbsp%3BIf%20Size%20(from%20get%20file%20metadata)%20is%20not%20equal%20to%200%3COL%3E%3CLI%3EIf%20Yes%3A%20Set%20Variable%20%22FileUploaded%22%20to%20true%3C%2FLI%3E%3CLI%3EIf%20No%3A%20Delay%20for%2010%20seconds%3C%2FLI%3E%3C%2FOL%3E%3C%2FLI%3E%3C%2FOL%3E%3C%2FLI%3E%3CLI%3EAction%3A%20Move%20File%3C%2FLI%3E%3C%2FOL%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ESee%20my%20attached%20screenshot%20for%20an%20example.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThe%20thing%20to%20remember%20is%20that%20the%20loop%20will%20only%20run%20through%2060%20iterations%2C%20so%20ensure%20that%20you%20select%20your%20delay%20to%20give%20your%20file%20enough%20time%20to%20upload.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EHope%20this%20helps.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1480471%22%20slang%3D%22en-US%22%3ERe%3A%20Using%20Flow%20with%20SharePoint%20Library%20-%20file%20copy%20executed%20before%20file%20upload%20is%20complete%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1480471%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F177418%22%20target%3D%22_blank%22%3E%40Matt%20Weston%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20am%20trying%20to%20replicate%20this%20flow%2C%20the%20section%20i%20am%20stuck%20on%20is%20the%20%22get%20file%20Metadata%22%2C%3C%2FP%3E%3CP%3EThe%20is%20asking%20for%20a%20file%20identifier%20i%20have%20tried%20ID%20and%20name%20but%20this%20errors%20stating%20the%20file%20does%20not%20exist.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ECan%20you%20let%20me%20know%20the%20values%20the%20get%20file%20metadata%20is%20expecting%20please%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
Highlighted
New Contributor

Trying to use Flow to move video files submitted to a "dead drop" folder into a controlled-access folder within a SharePoint Online document collection.

 

Essentially, a user drags a video file onto the dead drop (displayed in a browser) to initiate the upload and it gets whisked away into a locked down folder afterward. There is also an email alert that seems to be working fine.

 

The trigger When a file is created (properties only) fires when the upload starts, so the file isn't ready to be copied when the Flow starts. If I insert a static Delay that's long enough for the upload to complete, all is well.

 

The problem is my files are potentially a few GB in size and the upload time will be highly variable. If the Delay is too short, the remainder of the actions execute on the partially-uploaded file, with a BadRequest error.

  • If I try a simple Move file action in Flow, the action fails (status 404, "The file /path/to/file.example does not exist.") and the upload aborts as well.
  • If I try a Copy file action (to be followed by a Delete file action) with Asynchronous Pattern active and the Timeout set to 12h, the copy fails (status 400 "The system cannot find the file specified.")

 

I've tried testing the Modified property of the target file in a Do until loop, in the hope that the modification date and time are updated as the upload progresses and can thus be used as a progress counter - that's not working.

 

For the record, I've tried these other things with comparable results:

  • Using the When a file is created in a folder trigger
  • Setting Configure run after option in subsequent actions to ensure that the previous Copy File or Move file action was completed before moving ahead with the next action.

 

So is there any way to test to see if the uploaded file is "all there" after my When a file is created (properties only) trigger fires? Is there a property that gets applied only after the upload is complete that I can wait for or a better trigger to use to start the Flow? Is there just some better way to re-route large uploaded files into a locked-down folder?

3 Replies
Highlighted
Best Response confirmed by DPapandria (New Contributor)
Solution

Hi @DPapandria

 

The key indicator as to whether your file has uploaded or not is the file size. Until the upload is complete, the file size will be 0. As soon as it's complete you'll see the actual size. I tested this on a 400MB file.

 

So I have shaped a Flow to look like the following:

 

  1. Trigger - When a file is created (properties only)
  2. Action - Initialise variable: Created a boolean called FileUploaded and a default value of false
  3. Control - Do until: FileUploaded = true
    1. Action - Get file metadata (this has to be done each time to ensure that we're looking at a fresh set of data
    2. Condition - If Size (from get file metadata) is not equal to 0
      1. If Yes: Set Variable "FileUploaded" to true
      2. If No: Delay for 10 seconds
  4. Action: Move File

 

See my attached screenshot for an example.

 

The thing to remember is that the loop will only run through 60 iterations, so ensure that you select your delay to give your file enough time to upload.

 

Hope this helps.

Highlighted

That works perfectly.  Thanks very much!!

Highlighted

@Matt Weston 

I am trying to replicate this flow, the section i am stuck on is the "get file Metadata",

The is asking for a file identifier i have tried ID and name but this errors stating the file does not exist.

 

Can you let me know the values the get file metadata is expecting please?