Forum Discussion
Very slow Power Query refresh
Hi JCouture17
Logically, if time to refresh increases every week this means you have at least one table that grows every week... Generic recommendation: get rid of the rows & columns you don't need as early as possible (cf. https://social.technet.microsoft.com/Forums/en-US/1b104337-1910-4d84-b175-e45ce31a9e8a/listtransformtabletorecordst-each-complexfuncfield-vs-tableaddcolumnt-quottmpquot?forum=powerquery)
If you Buffer your tables presumably you already tested perf. with and w/o buffering. Otherwise be aware that buffering large tables can be counterproductive (cf. https://blog.crossjoin.co.uk/2020/05/21/monitoring-power-query-memory-usage-with-query-diagnostics-in-power-bi/)
- https://blog.crossjoin.co.uk/2020/05/31/optimising-the-performance-of-power-query-merges-in-power-bi-part-1/
- https://blog.crossjoin.co.uk/2020/06/03/optimising-the-performance-of-power-query-merges-in-power-bi-part-2/
- https://blog.crossjoin.co.uk/2020/06/07/optimising-the-performance-of-power-query-merges-in-power-bi-part-3-table-join-and-sortmerge/
- https://blog.crossjoin.co.uk/2020/06/14/optimising-the-performance-of-power-query-merges-in-power-bi-part-4-table-join-and-other-join-algorithms/
- https://www.thebiccountant.com/speedperformance-aspects/
As a side note, very tiny possible optimization. You can replace:
#"Replaced Value1" = Table.ReplaceValue(#"Replaced Value",
each [Prod Date],
each if [CQ] <> "-" then [CAPScrapWeekly.Prod Date] else [Prod Date],
Replacer.ReplaceValue,{"Prod Date"}
),
#"Changed Type4" = Table.TransformColumnTypes(#"Replaced Value1",{{"Prod Date", type date}}),
#"Removed Columns1" = Table.RemoveColumns(#"Changed Type4",{"CAPScrapWeekly.Prod Date"}),
with
#"Replaced Value1" = Table.ReplaceValue(#"Replaced Value",
each [CAPScrapWeekly.Prod Date],
each [CQ],
(a,b,c) as date => if c <> "-" then b else a,
{"Prod Date"}
),
#"Removed Columns1" = Table.RemoveColumns(#"Replaced Value1",{"CAPScrapWeekly.Prod Date"}),