SOLVED

Microsoft Forms: Ranking question type calculation methodology

Brass Contributor

What methodology is used in determining the results of a Ranking question type? I believe I have ruled out Ranked Choice Voting (RCV) but would like to know for sure the methodology.
Thanks!
~Kimberly

9 Replies
best response confirmed by Kimberly_Huffman (Brass Contributor)
Solution

@Kimberly_Huffman i dont know the official name, but i like to think of it as points.  Eg you have 5 items to rank. And 3 people do the ranking:

 

A, B, C, D, E

A, C, D, E, B

C, A, B, D, E

 

Position 1 gets 5 points, position 5 gets 1 point.  So add it up

A=14 points

B = 8

C = 12

D = 7

E = 4

 

So the ordering in the results page should show A as the winner, C as 2nd place, etc etc

While we order by total points, the bars are colored appropriately so you can see the breakdown within that score

 

Hope this makes sense!

 

@Jon_Kay Yes that seems to be the case. :smile:

 

I was able to find a RCV tabulator.  So for my purposes, I'm using the MS Forms Ranking question type to collect votes. When the voting is over, I will open the results in Excel, convert the ranked candidate cells to columns, save, and then use the RCV tabulator.  It's going to be interesting to see if the results will be the same. 

 

Thanks!

@Kimberly_Huffman can you share the RCV tabulator you used? How did it turn out?

@mrdlo Sure :) I found a lot of great information and the tabulator at Ranked Choice Voting Resource Center.  We used the Single-Winner RCV. I downloaded and installed the RCV Universal Tabulator.  It worked great. The tabulator is actually stored in GitHub along with documentation.  Good Luck!

~Kimberly

@Kimberly_Huffman how did the results compare to the Forms ranking methodology?

@Kendra_Perry In the screenshot, you can see how the Forms ranked compared to using the RCV Universal Tabulator. I collected the votes using Forms, opened the results in Excel, and uploaded those results to the RCV tabulator. Hope this helps.Forms vs RCV.png

Thanks for sharing! Very interesting.

@Jon_Kay what happens if there are 2 candidates with same number of points ? How does the logic for deciding if a candidate’s rank should be higher or lower than the other with same number of points decided ?

@Jon_Kay @Kimberly_Huffman 

I know this is old, but I'm sharing for the Interwebs searches in the future.

 

My ideal way of ranking needs to include the ability to NOT rank items you're not familiar with. (For example, ranking movies, you'd skip the movies you never saw, but you don't want that to affect your final score.)

To address this, you'd need a ranking system that allows a N/A option.
Then, voters rank.

Then, you convert each voters' rankings into a win-loss set of "games".
For example, if someone ranks items ABCD as 1) A, 2) D, 3) C (with B not ranked), you'd list out the "games" as A beats D, A beats C, D beats C, with no "games" for B whatsoever.
Finally, you can use a sports-ranking method to take those games and produce a set of rankings. I like the Colley Matrix, but there are others out there.

Like I said, it's far more complicated, but it would allow you to properly rank items that have less general experience. I'll probably try to code this at some point in the future.

1 best response

Accepted Solutions
best response confirmed by Kimberly_Huffman (Brass Contributor)
Solution

@Kimberly_Huffman i dont know the official name, but i like to think of it as points.  Eg you have 5 items to rank. And 3 people do the ranking:

 

A, B, C, D, E

A, C, D, E, B

C, A, B, D, E

 

Position 1 gets 5 points, position 5 gets 1 point.  So add it up

A=14 points

B = 8

C = 12

D = 7

E = 4

 

So the ordering in the results page should show A as the winner, C as 2nd place, etc etc

While we order by total points, the bars are colored appropriately so you can see the breakdown within that score

 

Hope this makes sense!

 

View solution in original post