Forum Discussion
Power Query - Best Practice to Reduce # of Steps?
- Mar 09, 2022
Hi DKoontz
Reordering columns n times makes no sense from an efficiency perspective. You should re-order only once i.e. at the end of your query
Re. renaming columns it depends what your query does. Assume you have a table with [Column1],[Column2]. If you rename [Column1] and later delete/remove that column, for sure it makes no sense to rename it. On the other hand, if in one step you rename [Column1] and in another step rename [Column2] (as long as you're no going to delete any of those columns later) this shouldn't make a differerence from a performance perspective (vs. renaming the 2 columns in a single step)
Regarding type changes it depends on the scenario as some transformations/operations "loose" columns types and you might want/need to re-type some columns before moving forward
A common "mistake" is to add a custom column without typing it then to add a change type step: If you add a column theoritically you know its data type so type it when adding that columnCouple of other points:
- Other common "mistake": a step that changes the type of [Column1] followed by a delete/remove [Column1] when [Column1] isn't involved in other calc. in the meantime
- As a general rule: get rid of the columns and rows you don't need as soon as possible
- The number of steps of a query doesn't necessarily correlates with its efficiency
- There are non-obvious things that only MSFT people know. E.g look at Curt H. reply to https://social.technet.microsoft.com/Forums/en-US/30da8f81-a3a9-4924-b54c-3007f508b24f/tableaddcolumn-tableremovecolums-vs-tabletransformcolumns-from-performance-point-of-view?forum=powerquery
Hi DKoontz
Reordering columns n times makes no sense from an efficiency perspective. You should re-order only once i.e. at the end of your query
Re. renaming columns it depends what your query does. Assume you have a table with [Column1],[Column2]. If you rename [Column1] and later delete/remove that column, for sure it makes no sense to rename it. On the other hand, if in one step you rename [Column1] and in another step rename [Column2] (as long as you're no going to delete any of those columns later) this shouldn't make a differerence from a performance perspective (vs. renaming the 2 columns in a single step)
Regarding type changes it depends on the scenario as some transformations/operations "loose" columns types and you might want/need to re-type some columns before moving forward
A common "mistake" is to add a custom column without typing it then to add a change type step: If you add a column theoritically you know its data type so type it when adding that column
Couple of other points:
- Other common "mistake": a step that changes the type of [Column1] followed by a delete/remove [Column1] when [Column1] isn't involved in other calc. in the meantime
- As a general rule: get rid of the columns and rows you don't need as soon as possible
- The number of steps of a query doesn't necessarily correlates with its efficiency
- There are non-obvious things that only MSFT people know. E.g look at Curt H. reply to https://social.technet.microsoft.com/Forums/en-US/30da8f81-a3a9-4924-b54c-3007f508b24f/tableaddcolumn-tableremovecolums-vs-tabletransformcolumns-from-performance-point-of-view?forum=powerquery
Thank you for the in-depth response, I really appreciate it! That is a good tip on typing custom columns, I've already seen myself do that and have gone back through and fixed those errors.
Definitely need to start thinking about speed but I was also worried about having unnecessary steps being more complicated for anyone else to go back into and look at. Seems like so much of PQ is future proofing, and that takes a whole different mindset than I am used to.
Thanks again!