Here we present comparisons between our predictions and the results of the 2014 Brownlow.
Recall we made two sets of predictions, the first where we allocated 3,2,1 votes to the three best players in each match, and the second where each player was allocated how many votes they were ‘expected’ to get on average. The two figures below give the predictions and how they compared to the results for the first and second sets of predictions respectively.
Allocating 3,2,1.
(click to enlarge)
Allocating expected votes
(click to enlarge)
The figure below gives the Kendall rank correlation coefficients between the results and both sets of predictions considering the first 5 to 30 positions.
(click to enlarge)
However, I believe the Kendall rank correlation is not ideal for measuring the performance of our predictions for two reasons:
1. If we look at say the top 5 ranks of our predictions and results, it’s is possible to get a very low correlation between the two even if both lists contain the same set of players.
2. We care far more about the top positions being correct or more accurate than lower positions.
This last figure gives the NDCG (Normalized Discounted Cumulative Gain) which aims to measure the quality of ranked predictions. There are some assumptions made here that may not be reasonable, such as the the relative importance weights for each position, but it’s something I thought may be worth experimenting with.

(click to enlarge)