My Simple Ratings Method Revisited
In the last piece I wrote on Geegeez I attempted to demonstrate to readers how you could go about creating a rating method in order to help you to analyse a race, writes Dave Renham. This article continues on from that simple ratings method, as I have decided to dig into the past and do some testing using historical results.
To recap, I was looking for a method for rating handicap races and, as far as weighting was concerned, I decided to give each factor a similar emphasis. In order to do this I used the PRB (Percentage of Rivals Beaten) metric.
The rating method I came up with involved five factors, as follows:
- Draw – Using the Draw Analyser tool rather the draw tab in the racecard in order to define a more precise date range, I set what I felt were relevant parameters. These parameters had to ideally a) match the race in question; and b) give me a big enough sample size.
My ideal date range is a recent one such as 2016 to 2023, for handicaps only and, in terms of runners, covering a spread of plus or minus two runners compared with the field size of the race I was rating. So, for example, if it was a 10-runner race, I would set 8 to 12 runners on the Draw Analyser. The Draw Analyser gives PRB figures for individual stalls as well as grouping them into thirds. I used the thirds method for the ratings, grouping low draws together, middle draws together and high draws together.
- Most Recent form – for this factor I used last time out (LTO) finishing position. For the relevant PRB figures I looked at two full years of handicap race data (2021 and 2022) to give what should be incredibly accurate readings. This amounted to several thousand races. The PRB figures had a range from 0.60 for winners last time out to 0.41 for horses that finished 9th or worse LTO.
- Recent Market data – for this factor I used LTO price – so the Industry Starting Price the horse returned in its most recent race. Once again I used 2021 and 2022 handicap races to create these PRB figures. The PRB figures had a range from 0.60 for horses priced 6/4 or shorter LTO to 0.36 for horses that priced 40/1 or bigger LTO.
- Long term form – for long term form I used career placed percentages in handicap races. Again the data for the PRB figures was taken from the two years of 2021-22 handicap results. The PRB had a range from 0.58 for career placed percentages of 51% or more, down to 0.44 for those who hit 20% or less.
- Fitness – for this final factor I used days since last run to create the PRB figures. To give the most accurate scores I used the same data set (’21-’22 handicap races) as I had done for the previous factors. The PRB figures ranged from 0.61 for horses that returned to the track within three days to 0.43 for horses off the track for 71 or more days.
So, essentially when rating each race, I took the relevant five PRB figures for each horse and added them up. The horse with the highest total became top rated, the second highest total became second rated and so on.
After writing the original article my plan was to rate a few races and see how the figures worked out. This is something I am still in the process of doing and will feed back my findings in a future article.
[I initially had no intention of back testing results because I thought it would take far too long. However, using a bit of excel, a fair amount of copying / pasting, and a few shortcuts I thought of as I was going along, I managed to get a year’s worth of ratings data in a few hours. The only ‘problem’ is that to do this I had to ignore the draw factor. The main reason for this was that it would take me far too long to gather the draw data (probably several months). But there were other reasons as well, one being that a good proportion of course and distances do not have a significant draw bias so trawling through masses of these types of race would not really improve the ratings or make them more accurate.]
So my ratings would be created using the four other PRB figures based on LTO position, LTO price, career handicap placed percentages and days since last run.
When you create ratings or systems and then back test them on past results, it is important to ensure that you use a different data set. This is a common mistake people make – one I made the first time I tried to create systems back in the early 90s. Hence, having used a data set of 2021 to 2022 to create all the PRB figures, I needed to choose a different year for the testing phase. I chose 2018.
A year of handicaps gives me plenty of data to work with. I did however want to narrow that down by looking only at 3yo+ and 4yo+ handicaps, as this would avoid handicaps with younger, less exposed runners. My assumption is that these ratings will work far better in races that involve older horses. I also chose to try and eliminate any draw factors by choosing handicaps races of 1 mile 1 furlong or more. Without the draw in the ratings, it made no sense to test shorter distance handicaps where draw bias can be extremely relevant and potential skew some ratings results (without the draw PRB being considered). Finally, I looked at 8- or 9-runner 3yo+/4yo+ handicap races for the basis of this article.
Just to reiterate I am back testing my ratings on:
- Year - 2018 (UK racing)
- Age group - 3yo+ / 4yo+ handicaps
- Distance - 1m1f or longer
- Runners – 8 or 9
Before sharing the results of my testing, let me discuss briefly what I am hoping to find. For the ratings to have the potential to be useful/effective, more often than not, they need to show strong correlation with the actual betting market. If your top two rated horses are consistently near the head of the actual betting market this is a far more positive sign than if they are consistently near the foot of the betting market. Of course in terms of making a profit from your ratings, you are looking for them to be more accurate than the actual betting market and throw up value selections. Not easy!
OK, let’s dig into my findings:
Firstly let's see how the top rated horses matched up against the actual betting market.
This graph is extremely positive with over 41% of the top rated runners starting as the favourite. Indeed 79% of the top rated runners started in the top three in the betting. There is a definite sliding scale, too, showing the type of correlation you would be hoping for. Let me look at the second top rated runners next in the same way:
Again, this graph gives positive results. You would expect the higher bars in the graph to be on the left hand side once more, and they are. Just over 72% of the 2nd rated runners ended up in the top four of the betting.
So the top section of the ratings are looking good. How about the lowest rated? Firstly let us look at the market rank percentages for the lowest rated runners. To begin with let me look at the 8th rated runner in 8-runner handicap races:
The graph is reversed compared with the first two we saw, which is exactly what we are looking for. 44% of the lowest rated runners were at the bottom of the betting market in 8th place. Less than 17% of them ended up in the top four of the betting. Further positive news as far as the ratings are concerned.
Now a look at the 9-runner races (bottom rated):
A similar lay out to the 8-runner races with nearly 72% of 9th rated runners ending up 7th, 8th or 9th in the actual betting market.
I have to say that I am extremely pleased with the correlation to date between my ratings and the betting market. For something that is relatively simple (just four parameters), it is mirroring the betting market well.
So these ratings, on the evidence we have so far (based on 324 races), definitely show some potential. Time to see how the ratings fared in terms of winning – their win strike rate. Clearly I was hoping that the top rated runners would comfortably out-perform lower rated ones. Here are the findings:
More positive news with the top two rated runners both securing strike rates in excess of 20%. Also a clear break between the top four rated and those rated fifth to ninth. The 7th rated is very slightly out of kilter, but this can happen – the important fact is the trend from top rated to bottom rated is downwards.
What I now want to look at is how the ratings would have performed if betting on them. I am assuming that we are backing at £1 level stakes to Betfair Starting Price (BSP). Here are the findings:
The results for the top rated runner are a little disappointing, losses of around 17p in the £. Horses ranked 7th have made surprisingly high profits, but most of the big priced BSP winners happened to pop up in this specific ranking position. I doubt these figures would be replicated again – this is just the type of outlier you can get when analysing BSP profit/loss.
When taking the top four rated as a whole, they have outperformed horses rated fifth to ninth as the table below shows:
Considering how big priced runners on Betfair can skew the figures, these grouped results are very heartening.
I must admit I am pleasantly surprised with these initial findings. I am intrigued to see how the ratings work with shorter distance races where I can include the fifth parameter – draw bias.
My next port of call was looking at ratings position versus betting market position. I wanted to compare the performance of horses that are rated better than their odds position, compared to those who are not.
Just to clarify, some examples of horses that are rated better than their odds position would be as follows (I appreciate for many I am just stating the obvious, but just in case there is any confusion in my English/grammar):
And here are examples where they are not (these include identical positions in the rank of the ratings compared to the actual market rank):
My hope is that I see better returns for horses that are rated better than their odds position, compared to those who are not. This would suggest that the ratings can potentially pinpoint some value selections.
Here are the returns for each:
These figures suggest the rankings are doing a pretty good job – it seems there has been more value when the rating rank has been better than the market position.
Before winding this piece up, I have one more set of data to share with you. I am looking at the actual rating scores and grouping the lower rated runners as a whole, and comparing them with the higher rated runners. Now the lowest possible rating using my PRB scores is 1.64; the highest possible is 2.39. The groupings I have used are horses that were rated 1.64 to 1.84, and horses that were rated 2.18 to 2.39. These groupings from 2018 3yo+/4yo+ handicaps would have produced the following results:
The strike rates should come as no surprise based on the evidence of the ‘Ratings Win SR%’ graph shown earlier, but the differences in returns are even wider than I had expected. It is another indication that these simple ratings have some real potential.
I'm to park things here for now and start further number crunching for the follow up article. The data set of 324 races is a decent one, but before making too many bold claims, I think we need to look to how these ratings fare in other races. Research wise, I plan to analyse the 2018 data from 10- to 12-runner races next. Once that’s done, I will write it up and share my findings.
Stay tuned!
- DR
Great article Dave. Lots of scope provided by your utilisation of the PRB metric. I have misplaced your email address, can you email me? Thanks.