Stuck in PQ and not sure how to achieve next steps

[Jpalaci1]

Hi, I need some help with a Power Query query. I’ve got stuck in the middle somewhere and have been Googling and searching and hit a roadblock. I know the exact output I want but not sure how to get there via PQ.

Steps performed so far:

1. Take the range and create a table (the format of how the data spits it out seems to warrant this to get correct range of data)
2. From Table/Range
3. Check data types automatically changed by PQ and all right
4. Column 1.1 Transform>Split Column by delimiter>split by comma
5. Select Total (last column)>Home>Remove Column
6. Select column 1.2>Split Column by delimiter>custom> ( > left-most delimiter (column 1.2 now becomes column 1.2.1)
7. Select new column and delete (it deletes the (USD) because I don’t need it)
8. Select column 1.2.1>Transform>Fill>Fill down
9. Select column 1.1>filter>filter out nulls
10. Select column 1.1>Add Column>Duplicate Column>reorder column in third position (before Account Name)

I’m not sure how to take column 1.1 and do the two following:

1. Fill down the employee number (the six digit value) over the values there and stopping at the next employee number
2. Filter out TOTALS (USD) and all its values (rows 26-36 in my PQ Table2). I think it might be just easier to just define the table without them to avoid dealing with the issue.

Then do the same thing with column 1.1 – Copy as I did with 1.1 but get rid of the employee numbers and only have the account numbers

I assume once I fix both 1.1 and 1.1. – Copy I can filter out Account Name columns of nulls and just have account names.

I can just change the names for each month to it’s numerical equivalent as well as name the columns to employee #, employee name, and account #

And try as I might, I’m not sure transpose or make into a list for each month. Stacked onto of each other.

Thank you for any help!

[#2]

@Jpalaci1 Check out this M code (paste in the advanced editor), you'll be able to retrace my steps.

let
    Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type any}, {"Account Name", type text}, {"January", Int64.Type}, {"February", Int64.Type}, {"March", Int64.Type}, {"April", Int64.Type}, {"May", Int64.Type}, {"June", Int64.Type}, {"July", Int64.Type}, {"August", Int64.Type}, {"September", Int64.Type}, {"October", Int64.Type}, {"November", Int64.Type}, {"December", Int64.Type}, {"Total", Int64.Type}}),
    #"Split Column by Delimiter" = Table.SplitColumn(Table.TransformColumnTypes(#"Changed Type", {{"Column1", type text}}, "en-US"), "Column1", Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv), {"Column1.1", "Column1.2"}),
    #"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Column1.1", type text}, {"Column1.2", type text}}),
    #"Removed Columns" = Table.RemoveColumns(#"Changed Type1",{"Total"}),
    #"Split Column by Delimiter1" = Table.SplitColumn(#"Removed Columns", "Column1.2", Splitter.SplitTextByDelimiter("(", QuoteStyle.Csv), {"Column1.2.1", "Column1.2.2"}),
    #"Changed Type2" = Table.TransformColumnTypes(#"Split Column by Delimiter1",{{"Column1.2.1", type text}, {"Column1.2.2", type text}}),
    #"Removed Columns1" = Table.RemoveColumns(#"Changed Type2",{"Column1.2.2"}),
    #"Filled Down" = Table.FillDown(#"Removed Columns1",{"Column1.2.1"}),
    #"Filtered Rows" = Table.SelectRows(#"Filled Down", each ([Column1.1] <> null)),
    #"Duplicated Column" = Table.DuplicateColumn(#"Filtered Rows", "Column1.1", "Column1.1 - Copy"),
    #"Reordered Columns" = Table.ReorderColumns(#"Duplicated Column",{"Column1.1", "Column1.2.1", "Column1.1 - Copy", "Account Name", "January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"}),
    #"Added Conditional Column" = Table.AddColumn(#"Reordered Columns", "Custom", each if [Account Name] = null then [Column1.1] else null),
    #"Filled Down1" = Table.FillDown(#"Added Conditional Column",{"Custom"}),
    #"Filtered Rows1" = Table.SelectRows(#"Filled Down1", each [Custom] <> "TOTALS (USD)"),
    #"Reordered Columns1" = Table.ReorderColumns(#"Filtered Rows1",{"Custom", "Column1.1", "Column1.2.1", "Column1.1 - Copy", "Account Name", "January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"}),
    #"Removed Columns2" = Table.RemoveColumns(#"Reordered Columns1",{"Column1.1"}),
    #"Filtered Rows2" = Table.SelectRows(#"Removed Columns2", each ([Account Name] <> null))
in
    #"Filtered Rows2"

[#3]

@Jpalaci1 

One more variant

let
    Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
    #"Kept First Rows" = Table.FirstN(
        Source,
        List.PositionOf(Source[Column1],"TOTALS (USD)")
    ),
    #"Removed Columns" = Table.RemoveColumns(#"Kept First Rows",{"Total"}),
    #"Filtered Rows" = Table.SelectRows(#"Removed Columns", each ([Column1] <> null)),
    #"Replaced Value1" = Table.ReplaceValue(
        #"Filtered Rows",
        each [Column1],
        each
            if Value.Type([Column1]) = type text
            then [Column1]
            else null,
            Replacer.ReplaceValue,{"Column1"}
    ),
    #"Split Column by Delimiter" = Table.SplitColumn(
        Table.TransformColumnTypes(
            #"Replaced Value1",
            {{"Column1", type text}}, "en-US"),
            "Column1",
            Splitter.SplitTextByEachDelimiter({","}, QuoteStyle.Csv, false
        ),
        {"Employee Number", "Name"}
    ),
    #"Replaced Value" = Table.ReplaceValue(
        #"Split Column by Delimiter",
        " (USD)","",
        Replacer.ReplaceText,{"Name"}
    ),
    #"Filled Down" = Table.FillDown(
        #"Replaced Value",
        {"Employee Number", "Name"}
    )
in
    #"Filled Down"

[#4]

@Jan Karel Pieterse 

Thank you so much! this really helps!

Two questions:

1) How do I stack each column onto each other/transpose the list? For example:

Employee     Amount    Month

Employee 1       x                1

Employee 2       x                1

Employee 3       x                1

Employee 1       x                2

Employee 2       x                2

Employee 3       x                2

Employee 1       x                3

Employee 2       x                3

Employee 3       x                3

2) Are there any resources/M Code resources where there already made/examples of Excel funtions? For example M Code equivalent of:

=IFS

=IFS

=UNIQUE(FILTER(

=LEFT(SEARCH(

=XLOOKUP(

=A1=B1 (to return Boolean T or F for data checks)

=WEEKNUM

=IF(AND(

Thank you again so much!

[#5]

Another question:

If I have questions about my M Code it's best to copy what I got so far in Advanced Editor and share that?

Thank you!

[#6]

@Jpalaci1 

@Jan Karel Pieterse could correct me, my variant

1) Table Transform - remove Employee and Account # columns, select Name, unpivot other columns, Group By Name and Month. Please see in attached.

2) I strongly recommend not to map Excel formulas into Power Query, that's ineffective approach. Think in terms of records and lists (aka rows and columns), not in terms of cells and their values. Merging, grouping, pivoting/unpivoting are usually used. As for date/text/numbers transformation check Transform tab from left to right, majority of the are here.

3) It's always better to attach the file, scripts within posts are more for other people who will check for similar solutions.

[#7]

@Sergei Baklan 

Last question. When I get my data it comes as a whole new data set. I don't just append the latest months as the new month's file can have corrections for the previous months. Should I put into a folder the raw data and when I get the latest take out last month's and replace the newest in the folder? Should I just do a new query using by pasting the M Code every month a new file comes in?

Thank you so much for the help!

[#8]

@Jpalaci1 

If all these files have the same structure, you may collect them in some folder and use From Folder connector to transform all files at once. To keep only latest figures, on final step sort result by name and month and remove duplicates.

Ideal situation if you build this reporting system and forget about it for years. For example, Flow drops attachments from emails into the folder upon receiving; power query in background performs scheduled refreshes and returns result to publish it on web or in another Excel.