SOLVED

Get & Transform Data | Get Data from a folder transforming the files before the combination

%3CLINGO-SUB%20id%3D%22lingo-sub-211663%22%20slang%3D%22en-US%22%3EGet%20%26amp%3B%20Transform%20Data%20%7C%20Get%20Data%20from%20a%20folder%20transforming%20the%20files%20before%20the%20combination%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-211663%22%20slang%3D%22en-US%22%3E%3CP%3EHello%2C%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20want%20to%20read%20the%20files%20in%20a%20folder%2C%20but%20those%20files%20needs%20some%20transformation%20before%20I%20can%20combine%20them%20with%20power%20query.%20Normally%20what%20i%20would%20do%20if%20all%20of%20them%20were%20in%20a%20table%20format%20is%20to%20combine%20them%20and%20load%20but%20since%20the%20data%20in%20each%20file%20is%20not%20fully%20structure%20if%20i%20do%20this%20i%20won%C2%B4t%20get%20correct%20data.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20have%20many%20files%20that%20before%20the%20combination%20need%20to%3A%3C%2FP%3E%3CP%3E1-Remove%20the%20first%2010%20rows%20of%20each%20since%20they%20are%20empty.%3C%2FP%3E%3CP%3E2-Unpivot%20columns.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EHow%20can%20i%20do%20this%20with%20Power%20Query%20before%20combining%20them%20in%20a%20massive%20way%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-211663%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EBI%20%26amp%3B%20Data%20Analytics%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ERequests%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-212290%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20%26amp%3B%20Transform%20Data%20%7C%20Get%20Data%20from%20a%20folder%20transforming%20the%20files%20before%20the%20combination%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-212290%22%20slang%3D%22en-US%22%3EThank%20you%20for%20your%20support!%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-212094%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20%26amp%3B%20Transform%20Data%20%7C%20Get%20Data%20from%20a%20folder%20transforming%20the%20files%20before%20the%20combination%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-212094%22%20slang%3D%22en-US%22%3EAlso%2C%20load%20as%20Connection%20Only%20to%20start%20with.%3CBR%20%2F%3E%3CBR%20%2F%3Ethen%20After%20it's%20complete%20go%20to%20the%20Query%20Pane%20and%20change%20your%20load%20destination%20to%20Table%20or%20Data%20Model%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-212093%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20%26amp%3B%20Transform%20Data%20%7C%20Get%20Data%20from%20a%20folder%20transforming%20the%20files%20before%20the%20combination%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-212093%22%20slang%3D%22en-US%22%3E%3CP%3EHi%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F161565%22%20target%3D%22_blank%22%3E%40Carlos%20Cardenas%20Velasquez%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EStart%20with%20Combine%20and%20Edit%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22image.png%22%20style%3D%22width%3A%20804px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F37087iF1B89A1E57AC0687%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20title%3D%22image.png%22%20alt%3D%22image.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThen%20do%20your%20transformations%20(removing%20rows%20and%20unpivotting)%20in%20the%20Transform%20Sample%20File%20step%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22image.png%22%20style%3D%22width%3A%20875px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F37088i7ECB2604D3C3D942%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20title%3D%22image.png%22%20alt%3D%22image.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThose%20transformations%20will%20then%20be%20applied%20to%20all%20the%20files%20as%20they%20are%20pulled%20in.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1809259%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20%26amp%3B%20Transform%20Data%20%7C%20Get%20Data%20from%20a%20folder%20transforming%20the%20files%20before%20the%20combination%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1809259%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F9832%22%20target%3D%22_blank%22%3E%40Wyn%20Hopkins%3C%2FA%3E%26nbsp%3BI%20have%20done%20this%20multiple%20times%2C%20however%2C%20the%20buttons%20at%20the%20bottom%2C%20combine%20and%20load%2C%20are%20no%20longer%20available.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20am%20using%20Excel%202016%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1811888%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20%26amp%3B%20Transform%20Data%20%7C%20Get%20Data%20from%20a%20folder%20transforming%20the%20files%20before%20the%20combination%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1811888%22%20slang%3D%22en-US%22%3E%3CP%3Ehi%20%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F842526%22%20target%3D%22_blank%22%3E%40chucknjoy%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EId%20suggest%20starting%20a%20new%20post%20and%20adding%20a%20screenshot%20to%20help%20us%20debug%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ECheers%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWyn%3C%2FP%3E%3C%2FLINGO-BODY%3E
Highlighted
New Contributor

Hello,

 

I want to read the files in a folder, but those files needs some transformation before I can combine them with power query. Normally what i would do if all of them were in a table format is to combine them and load but since the data in each file is not fully structure if i do this i won´t get correct data.

 

I have many files that before the combination need to:

1-Remove the first 10 rows of each since they are empty.

2-Unpivot columns.

 

How can i do this with Power Query before combining them in a massive way?

 

5 Replies
Highlighted
Best Response confirmed by Carlos Cardenas Velasquez (New Contributor)
Solution

Hi @Carlos Cardenas Velasquez

 

Start with Combine and Edit

 

image.png

 

Then do your transformations (removing rows and unpivotting) in the Transform Sample File step

 

image.png

 

 

Those transformations will then be applied to all the files as they are pulled in.

 

 

Highlighted
Also, load as Connection Only to start with.

then After it's complete go to the Query Pane and change your load destination to Table or Data Model
Highlighted
Highlighted

@Wyn Hopkins I have done this multiple times, however, the buttons at the bottom, combine and load, are no longer available.

 

I am using Excel 2016

Highlighted

hi @chucknjoy 

 

Id suggest starting a new post and adding a screenshot to help us debug

 

Cheers

 

Wyn