Forum Discussion
Power Query Connecting to Datasource
It takes 2 minutes to connect to two Power Query tables in a 724 KB Excel file.
The source data comes from two Excel files: the first Excel file is 10.3 MB and has five tables used in subsequent queries; the second is 388 KB of raw data.
The two "final" tables are fed by 9 "connection only" or feeder queries with five different merges.
One of the final tables is 1,860 rows and the other is 730 rows.
There is nothing especially fancy or complicated in any of the Queries except for "pivots."
None of the "connection only" queries have any "Refresh control," but the two final tables are set to "Refresh...when opening..." and "Enable Fast Data Load."
I have a simple UDF (.xlam) "add-in" with about 10 lines of VBA code that sets the X- and Y- axis minimums and maximums for graphs.
It takes just about as long to connect without the UDF as it does with the UDF.
I've gone through many of the online videos and articles to help speed up a query.
I'm on Build 26100.1150 ge_release, running a Dell Latitude 7340 64-bit processor with 16 GB of memory.
I'd love the help and advice of this community. Thanks in advance.
- Also I'm running through Ethernet with 300 MB/s up and 100 MB/s down
- Since this post I've also:
(1) Unlinked and Re-Linked OneDrive NO EFFECT
(2) Reset OneDrive NO EFFECT
(3) Put all files on Local C: Drive NO EFFECT
While on Local C: Drive:
(4) Removed the largest table (46,500 rows X 8 columns) that was also "unpivoted" NO EFFECT
(5) Removed all other non-essential tables NO EFFECTQuery itself could be slow.
Excel table is internally quired two times, first one to analyse and define types, second one to rederive the data. There is the trick with M-scripting for workaround, but to use if that's really critical.
Sorting and FillUp/Down are quite expensive operations.
How do you do merging - Table.NestedJoin or Table.Join and how.
Other things could affect. Etc., etc. Chris Webb has series of posts related to PQ performance, one of them Chris Webb's BI Blog: Optimising The Performance Of Power Query Merges In Power BI, Part 5: Cross Joins . Perhaps here you could find something useful.
Such things as performance is too abstract to discuss without concrete file(s).
- Thank you Sergei. Much appreciated. These references were helpful. So were these:
https://learn.microsoft.com/en-us/power-bi/guidance/power-query-referenced-queries
https://learn.microsoft.com/en-us/power-query/new-dataflow-from-template
- Next, I reset to default all the Refresh settings, in both Get Data > Query Options and for each individual query. NO EFFECT
