SOLVED

Pattern matching for User information rows to uniquify users

%3CLINGO-SUB%20id%3D%22lingo-sub-2138426%22%20slang%3D%22en-US%22%3EPattern%20matching%20for%20User%20information%20rows%20to%20uniquify%20users%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2138426%22%20slang%3D%22en-US%22%3E%3CP%3EHello%20there%2C%3C%2FP%3E%3CP%3Enew%20to%20this%20so%20bear%20with%20me.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20got%20a%20big%20Excel%20sheet%20with%20about%205000%20row%20entries%20and%2010%20columns%20of%20user%20data.%3C%2FP%3E%3CP%3EThings%20like%2C%20Name%2C%20Company%20Name%2C%20ID%2C%20Zip-Code%2C%20you%20name%20it%2C%20casual%20stuff.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ESince%20this%20is%20user%20entry%20based%20there%20might%20be%3A%3C%2FP%3E%3CP%3E1.%20Spelling%20mistakes%20of%20the%20Company%20name%3C%2FP%3E%3CP%3E2.%20Different%20people%20use%20different%20shortcuts%20for%20company%20names%3C%2FP%3E%3CP%3E3.%20Some%20company%20names%20are%20used%20differently%20(%20Dodger%20Highball%20Stadium%20for%201%20person%20and%20the%20other%20enters%20Highball%20Stad%20Dod)%20for%20example.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ESince%20i%20use%20this%20data%20in%20PowerBI%20i%20need%20the%20example%20from%203.%20to%20match%20each%20others%20name.%26nbsp%3B%3C%2FP%3E%3CP%3ESo%20both%20are%20called%20the%20actual%20name%2C%20Dodger%20Highball%20Stadium%20to%20use%20the%20data%20further.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EIs%20there%20a%20way%20with%20pattern%20matching%20or%20something%20like%20this%20to%20try%20to%20get%20things%20to%20work%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EMuch%20appreciated%20to%20any%20information%20which%20may%20lead%20me%20to%20a%20solution%20%3CLI-EMOJI%20id%3D%22lia_grinning-face-with-big-eyes%22%20title%3D%22%3Agrinning_face_with_big_eyes%3A%22%3E%3C%2FLI-EMOJI%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks%20in%20advance!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2138426%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EExcel%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2138596%22%20slang%3D%22en-US%22%3ERe%3A%20Pattern%20matching%20for%20User%20information%20rows%20to%20uniquify%20users%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2138596%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F967860%22%20target%3D%22_blank%22%3E%40DanielWWW%3C%2FA%3E%26nbsp%3BDid%20some%20testing%20with%20Fuzzy%20Matching%20in%20Power%20Query.%20In%20order%20to%20match%20%22Dodger%20Highball%20Stadium%22%20with%20%22Highball%20Stad%20Dod%22%20you%20need%20to%20set%20the%20similarity%20threshold%20to%200.18.%20This%20means%20that%20you%20are%20going%20end%20up%20with%20a%20lot%20of%20other%20matches%20as%20well.%20entries%20like%20%22Stadium%20Rd.%22%20or%20%22Dodge%20Motor%20Company%22%20or%20%22Highba%20Inc.)%20will%20also%20be%20matched.%20Anything%20with%20large%20parts%20of%20either%20of%20the%20three%20words%20seems%20to%20be%20matched.%20Thus%20not%20very%20useful.%26nbsp%3B%3C%2FP%3E%3CP%3ESo%2C%20you%20could%20perhaps%20use%20fuzzy%20matching%20with%20a%20higher%20threshold%20that%20will%20allow%20for%20small%20spelling%20errors.%20But%20to%20capture%20the%20more%20extreme%20variations%2C%20I%20fear%20you%20are%20going%20to%20need%20human%20brain%20power%20and%20some%20manual%20work%20to%20set%20up%20a%20transformation%20table%20with%20all%20possible%20references%20to%20one%20and%20the%20same%20location.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EAttaching%20my%20test%20file.%20Perhaps%20useful.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2138708%22%20slang%3D%22en-US%22%3ERe%3A%20Pattern%20matching%20for%20User%20information%20rows%20to%20uniquify%20users%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2138708%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F403176%22%20target%3D%22_blank%22%3E%40Riny_van_Eekelen%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThank%20you%20Riny%20for%20the%20fast%20answer%20already.%3C%2FP%3E%3CP%3EI%20will%20try%20it%20out%20and%20test%20a%20little%20bit%20and%20will%20let%20you%20know.%3C%2FP%3E%3CP%3E%3CLI-EMOJI%20id%3D%22lia_grinning-face-with-big-eyes%22%20title%3D%22%3Agrinning_face_with_big_eyes%3A%22%3E%3C%2FLI-EMOJI%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2138741%22%20slang%3D%22en-US%22%3ERe%3A%20Pattern%20matching%20for%20User%20information%20rows%20to%20uniquify%20users%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2138741%22%20slang%3D%22en-US%22%3E%3CP%3EHi%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F967860%22%20target%3D%22_blank%22%3E%40DanielWWW%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EYou%20can%20use%20the%20fuzzy%20match%20in%20Power%20Query%20for%20this%20issue%2C%20what%20I%20would%20suggest%20you%20use%20the%20FUZZY%20match%20with%20transformation%20table%20where%20you%20can%20input%20the%20value%20FROM%20and%20TO%20handle%20this%20scenario.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%3ERegards%2C%20Faraz%20Shaikh%20%7C%20MCT%2C%20MIE%2C%20MOS%20Master%2C%20Excel%20Expert%3C%2FSPAN%3E%3C%2FP%3E%3CDIV%20class%3D%22lia-message-body%20lia-component-message-view-widget-body%20lia-component-body-signature-highlight-escalation%20lia-component-message-view-widget-body-signature-highlight-escalation%22%3E%3CDIV%20class%3D%22lia-message-body-content%22%3E%3CP%3E%3CEM%3E%3CFONT%20color%3D%22%23808080%22%3EIf%20you%20find%20the%20above%20solution%20resolved%20your%20query%20don't%20forget%20mark%20as%26nbsp%3B%3CSPAN%3EOfficial%2FBest%20Answer%20%26amp%3B%20like%20it%20to%20help%20the%20other%20members%20find%20it%20more.%3C%2FSPAN%3E%3C%2FFONT%3E%3C%2FEM%3E%3C%2FP%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E
New Contributor

Hello there,

new to this so bear with me.

 

I got a big Excel sheet with about 5000 row entries and 10 columns of user data.

Things like, Name, Company Name, ID, Zip-Code, you name it, casual stuff.

 

Since this is user entry based there might be:

1. Spelling mistakes of the Company name

2. Different people use different shortcuts for company names

3. Some company names are used differently ( Dodger Highball Stadium for 1 person and the other enters Highball Stad Dod) for example.

 

Since i use this data in PowerBI i need the example from 3. to match each others name. 

So both are called the actual name, Dodger Highball Stadium to use the data further.

 

Is there a way with pattern matching or something like this to try to get things to work?

 

Much appreciated to any information which may lead me to a solution  

Thanks in advance!

3 Replies
Best Response confirmed by DanielWWW (New Contributor)
Solution

@DanielWWW Did some testing with Fuzzy Matching in Power Query. In order to match "Dodger Highball Stadium" with "Highball Stad Dod" you need to set the similarity threshold to 0.18. This means that you are going to end up with a lot of other matches as well. Entries like "Stadium Rd." or "Dodge Motor Company" or "Highba Inc.) will also be matched. Anything with large parts of either of the three words seems to be matched. Thus not very useful. 

So, you could perhaps use fuzzy matching with a higher threshold that will allow for small spelling errors. But to capture the more extreme variations, I fear you are going to need human brain power and some manual work to set up a transformation table with all possible references to one and the same location.

 

Attaching my test file. Perhaps useful.

@Riny_van_Eekelen 

Thank you Riny for the fast answer already.

I will try it out and test a little bit and will let you know.

Hi @DanielWWW 

 

You can use the fuzzy match in Power Query for this issue, what I would suggest you use the FUZZY match with transformation table where you can input the value FROM and TO handle this scenario.

 

Regards, Faraz Shaikh | MCT, MIE, MOS Master, Excel Expert

If you find the above solution resolved your query don't forget mark as Official/Best Answer & like it to help the other members find it more.