Which ATP Tournaments Are the Most Competitive?
This post has been updated since its original publication. Edits are shown in strikethroughs and italics.
Last summerish, I published two posts about tournament competitiveness, one at the end of grass season and one at the end of the clay season. I never got around to doing the hard courts, because I was focused on creating this site and moving things over from the FBITennis blog. If you did not see them before, there’s no need to follow those links because pretty much everything in those posts will be here too.
As a point of reference, here is one of the charts from the original clay court post, just so you can see the data I included:
For each tournament, I calculated the average ELO of the players in the main draw (using my surface-specific ELO blend), the difference between the highest ELO player and the lowest, and the standard deviation of all the ELOs to get an idea how the talent level is distributed in the draw. Then using my tournament forecasts (which include a matchstats component and ELO), I determined how many players had a 20% chance or better to make the quarterfinals, and so on, as shown in the chart.
Bringing It Forward
As I was listening to yesterday’s Tennis Abstract podcast, host-for-48-minutes Carl Bialik mentioned that he would be interested in seeing how competitive the tournaments are with respect to forecasted results. In other words, a tournament with a bunch of 60-40 matches would be considered more competitive than a tournament with the same number of 70-30 matches (all other factors being equal). So, I decided to dust off the info above and add a component to address Carl’s comment.
But first, to clean up something that bothered me in the original tables. The percentage of that last field, i.e., less than a 1% chance of making the semifinals, was a little confusing. All the other categories are greater than (though really greater than or equal to), and if you want a competitive tournament, you want those numbers to be high. Then the last number switches it around, and you want the number to be low. Too confusing. In the new data, that last element is flipped around so it represents the percentage of the field that has at least a 1% chance of making the semifinals.
With that change, let’s call that data the pre-tournament information. It is based on the players who started in the first round, and their forecasted results, calculated before the tournament started.
To address Carl’s comment, I have added four other data points to take into account how things move along in the tournament: the average winning percentage of the underdogs in
all the matches, the quarterfinals matches, the semifinals matches and the finals, and the average ELOs of the semifinalists (excluding any players who withdrew from the semis).
Now, rather than present this information in a series of tables (really pictures of tables) by surface, I thought it would be more user-friendly to present it in one giant, sortable table, to let you do your own thinking about what it means. It will not be as pretty/colorful, but should be more useful. So that it remains available long after this post has faded into the archive, I have posted the table to a separate page here.
Roughing Up A Tournament Competitiveness Score
In case you don’t want to do your own thinking on this, but want to look at the table anyway, I also decided to reduce all the elements into a single Tournament Competitiveness Score (TCS), calculated as follows:
- Standardize each of the stats within a category (e.g., Avg ELO, Champ >5%, etc.).
- Weight each of the standardized scores as follows:
- Avg ELO and Stdev ELO get a weight of one, as I think these are the most important in determining competitiveness.
- Max ELO -Min ELO is included for interest, but is not weighted, because it provides little information that the standard deviation does not (i.e., two standard deviations on either side of the mean covers 95% of the cases and therefore gets you close to the max and the min).
gets a weight of 0.5. My thinking is that this particular measure may not be hugely important, as some of the conclusions we reach from it is probably in our conclusions from the standard deviation.
- The pre-tournament forecasts for getting to the QF, SF, F and winning the championship, and the broader measure of who has at least a 1% shot at the semis, all get weighted at 0.5.
- The average forecasted winning percentage of the underdogs for all players in the draw gets
a smallno weight at just 0.1. I think this information is interesting, and I wanted to include it in the table. However, as a component in the Tournament Competitiveness Score, all the first round matches it includes are essentially covered by the pre-tournament measures, and the outer round matches it includes are part of other post-tournament measures.
- The average forecasted winning percentage of the underdogs for the quarterfinals, semifinals and finals get weighted 0.5 each, to match the pre-tournament forecast weights.
- The average ELO of the semifinalists gets weighted 1.5x to provide the “right” balance against the pre-tournament factors and also because the players who make it through to the semis significantly affect our perception of how competitive a tournament is.
- Sum all of that.
(A) the pre-tournament weighted standardized scores and multiply by 65%, and (B) sum the post-tournament weighted standardized scores and multiply by 35%, and (C) add the two together. I made those percentages up.The sum of the weights for the pre-tournament portion is greater than the sum of the weights for the post-tournament portion intentionally. The idea is that the pre-tournament scores should carry more weight, because those are the things the tournament has at least nominal control over from a competitiveness standpoint. Also, I think some of the competitiveness impression we derive from the post-tournament scores is covered, at least in part, by the pre-tournament expectations. Sure, things often do not go as planned in a tournament, but they also go more consistently to plan that we generally think.
- Normalize the sum you get from #3 to a scale of 1-100. Previously I normalized by giving the lowest-scoring tournament a 0 and the highest scoring tournament a 100, but that does not feel right. The 2018 Australian Open does not feel like a zero, and 2018 Vienna does not feel like a “perfect” 100. TCS now is normalized to tighten the range, so nothing is a zero and nothing is perfect.
There are innumerable ways to come up with a Tournament Competitiveness Score, and I may choose to change the weights and methods as I gain experience with it, but for now, I think it is a decent
first crack at it.
Grand Slam Warning
When you see the table, one of the things that may surprise you is how poorly the Grand Slams fare. It is important to distinguish the Tournament Competitiveness Score, from any measure of how “good” the tournament is. Obviously we all love the Grand Slams, where the greatest players usually rise to the top of their games and give us some great matches. Plus, for real tennis lovers, there are so many matches at Grand Slams, not to mention tradition, prize money, five setters for me, etc. There’s more to enjoying tennis than competitiveness and a look at the quality of the entire field.
Grand Slams will not fare well with something like TCS, because the field is too deep. More players get in, therefore there is a wider range of ELOs in play, and more one-sided matchups. The one-sidedness is further enhanced by the best-of-five format. For now, I suggest you compare apples to apples and look at the Grand Slams against each other (and maybe throw in the Masters events).
I’m still noodling on whether to adjust (i.e., boost) TCS for the Grand Slams and maybe the Masters events to account for their larger fields. I tried a couple of adjustments, mostly designed to bump the Grand Slams and push Houston and Winston-Salem to the bottom (my personal taste), but so far I have not found an adjustment method I’m comfortable with.
I may blend in ELOs of the Top 8 or Top 16 seeds for all the tournaments, and there the Grand Slams will shine, bringing them more into the mix. That, however, is something a little different than measuring the overall competitiveness of the tournament, so I need to give it more thought. Instead, I am letting the last post-tournament element, the ELO average of the semifinalists, give the Grand Slams a boost, when they deserve it.