SOLVED

Power Query Faulting

%3CLINGO-SUB%20id%3D%22lingo-sub-3093073%22%20slang%3D%22en-US%22%3EPower%20Query%20Faulting%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3093073%22%20slang%3D%22en-US%22%3E%3CP%3EHello.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20was%20a%20fan%20of%20Microsoft%20Access%20until%20I%20discovered%20Power%20Query.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EDespite%20my%20enthusiasm%20for%20PQ%2C%20I%20am%20find%20some%20issues%20that%20I%20cant%20seem%20to%20figure%20out.%3C%2FP%3E%3CP%3E1.%26nbsp%3B%20A%20switchboard%20I%20am%20trying%20to%20implement%20does%20not%20seem%20to%20be%20updating%20quickly%20enough%20and%20potentially%20not%20at%20all.%26nbsp%3B%20All%20data%20is%20saved%20locally%20on%20my%20hard%20drive%20so%20it%20is%20not%20a%20connectivity%20issue.%3C%2FP%3E%3CP%3E2.%26nbsp%3B%20I%20discovered%20with%20a%20data%20set%20that%20I%20am%20pulling%20down%20from%20an%20web%20based%20reporting%20tool%2C%20that%20any%20negative%20dollar%20values%20are%20excluded%20and%20removed%20before%20any%20manipulation%20of%20data.%26nbsp%3B%20This%20is%20causing%20inaccurate%20aggregation%20of%20%24%20and%20misrepresenting%20what%20I%20am%20trying%20to%20present.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EAny%20help%20and%20direction%20would%20be%20greatly%20appreciated.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-3093073%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EBI%20%26amp%3B%20Data%20Analysis%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EExcel%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EOffice%20365%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EPower%20BI%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3093493%22%20slang%3D%22en-US%22%3ERe%3A%20Power%20Query%20Faulting%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3093493%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F1292397%22%20target%3D%22_blank%22%3E%40Luciano360%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E1)%20Power%20Query%20performance%20depends%20on%20many%20things%2C%20that's%20not%20only%20which%20data%20source%20you%20use.%20As%20an%20example%20you%20may%20check%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fblog.crossjoin.co.uk%2F2020%2F05%2F31%2Foptimising-the-performance-of-power-query-merges-in-power-bi-part-1%2F%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3EChris%20Webb's%20BI%20Blog%3A%20Optimising%20The%20Performance%20Of%20Power%20Query%20Merges%20In%20Power%20BI%2C%20Part%201%3A%20Removing%20Columns%20Chris%20Webb's%20BI%20Blog%20(crossjoin.co.uk)%3C%2FA%3E%2C%20here%20are%20a%20lot%20more%20related%20posts.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E2)%20Could%20you%20provide%20some%20sample%3F%20How%20exactly%20do%20you%20aggregate%2C%20field%20types%2C%20etc.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3094734%22%20slang%3D%22en-US%22%3ERe%3A%20Power%20Query%20Faulting%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3094734%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F521%22%20target%3D%22_blank%22%3E%40Sergei%20Baklan%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThank%20you%20for%20your%20prompt%20attention...%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EConcerning%20the%20issue%20of%20removing%20negative%20amounts%20I%20I%20would%20like%20to%20share%20the%20work%2C%20but%20I%20am%20not%20sure%20how%20to%20do%20so%20on%20this%20forum.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EWhile%20%22playing%22%20with%20this%20data%2C%20I%20noticed%20that%20when%20the%20data%20is%20reading%20to%20an%20outside%20source%20the%20problem%20arises%2C%20however%2C%20if%20the%20raw%20data%20is%20resident%20in%20the%20same%20file%2C%20the%20queries%20run%20as%20they%20should.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThe%20preferred%20method%20for%20managing%20the%20data%20would%20be%20the%20raw%20data%20as%20a%20seterate%20file%20since%20it%20would%20get%20updated%20routinely%20and%20I%20would%20like%20to%20keep%20all%20the%20query%20data%20in%20tact%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3094803%22%20slang%3D%22en-US%22%3ERe%3A%20Power%20Query%20Faulting%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3094803%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F1292397%22%20target%3D%22_blank%22%3E%40Luciano360%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIf%20you%20may%20share%20the%20file%20that%20will%20be%20great.%20It%20could%20be%20an%20option%20to%20attach%20the%20file%20to%20the%20post%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22image.png%22%20style%3D%22width%3A%20783px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F343990iB32E6D6D56C48728%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22image.png%22%20alt%3D%22image.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3EIf%20it%20is%20not%20available%20for%20please%20send%20me%20with%20Private%20Message%20here%2C%20I%20will%20re-share.%3C%2FP%3E%0A%3CP%3EIn%20any%20case%20please%20remove%20any%20sensitive%20information%20if%20exists.%3C%2FP%3E%3C%2FLINGO-BODY%3E
New Contributor

Hello.

 

I was a fan of Microsoft Access until I discovered Power Query.

 

Despite my enthusiasm for PQ, I am find some issues that I cant seem to figure out.

1.  A switchboard I am trying to implement does not seem to be updating quickly enough and potentially not at all.  All data is saved locally on my hard drive so it is not a connectivity issue.

2.  I discovered with a data set that I am pulling down from an web based reporting tool, that any negative dollar values are excluded and removed before any manipulation of data.  This is causing inaccurate aggregation of $ and misrepresenting what I am trying to present.

 

Any help and direction would be greatly appreciated.

5 Replies

@Luciano360 

1) Power Query performance depends on many things, that's not only which data source you use. As an example you may check Chris Webb's BI Blog: Optimising The Performance Of Power Query Merges In Power BI, Part 1: Removing..., here are a lot more related posts.

 

2) Could you provide some sample? How exactly do you aggregate, field types, etc.

@Sergei Baklan 

 

Thank you for your prompt attention... 

 

Concerning the issue of removing negative amounts I I would like to share the work, but I am not sure how to do so on this forum.

 

While "playing" with this data, I noticed that when the data is reading to an outside source the problem arises, however, if the raw data is resident in the same file, the queries run as they should.

 

The preferred method for managing the data would be the raw data as a seterate file since it would get updated routinely and I would like to keep all the query data in tact

@Luciano360 

If you may share the file that will be great. It could be an option to attach the file to the post

image.png

If it is not available for please send me with Private Message here, I will re-share.

In any case please remove any sensitive information if exists.

best response confirmed by Luciano360 (New Contributor)
Solution

@Luciano360 

Here is the summary for other people regarding negative numbers.

- we query external binary file (.xlxb) and Power Query ignores minus for negative numbers

- actually, even if we use the same From File menu, behind for binary file Power Query use another connector. Same as for Access Database (you may try From Access Database and navigate on Excel binary file, result will be the same as for From File).

- this connector returns masked by applied format numbers, not actual ones. Other words close to what in Excel is returned by =TEXT( number, format string ), but not exactly the same

- In particular file to numbers was applied custom number format [=0]#,##0;#,##0

- it looks like Power Query doesn't recognize correctly leading [=0] and ignores minus returning the number

- if apply simple #,##0 format or like, Power Query works correctly with negative numbers

 

In general where possible better to work with .xlsx files. From performance point of view even better with .csv

Thanks You!

I changed the source file to .xlsx and it seems to have resolved the issue!

Many thanks!