by In our first article in this series we learned that even if some sports models, used to make publically available predictions for sport events, have been proven to be far superior to random guesses, they still tend to be incapable of beating betting markets.
The second article touched upon how we had obtained Pinnacle’s biathlon odds for the 2020-21 season and that this means that we are now in a position to test how our model’s predictions, for athletes/teams to win/not to win, performed against Pinnacle’s opening and closing odds.
Pinnacle offered precisely 300 such markets last season. Rather than placing a table with 300 rows inside this article, we have made a Google doc and put the odds for all of these 300 markets in four different tabs. One tab for each table we are displaying in this article. The tabs have similar names to the tables so it should be easy to understand which is which.
Unfortunately we cannot provide you with editing rights to this document, as changes to the document would mean that the other readers would not see the different tabs as we intend them to be. If you should want to make changes and play with the numbers, it should be easy enough to copy it to another Google doc or Excel.
In the tabs we have inputted the percentage chance we in our previews indicated for the athletes/teams that Pinnacle had “to win/not to win” markets for. Than we converted these percentages to odds. After doing this it is easy to spot which of Pinnacle’s opening odds were longer than those implied by our model’s probability predictions.
For Pinnacle’s opening odds this happened in 151 of the 300 markets. In the table below we have summed up how betting one unit on all of the bets our model indicates as value bets would have performed. The bet size of one unit per bet makes best sense in our opinion, as the sample size of some 151 bets is much smaller than what we would have preferred it to be, and it is well known that equal stakes is what makes best sense when you are checking the performance of a tipster/model on a small sample size. This is why we will stick with even stakes for all our tests.
To identify precisely which 151 bets we have taken at what odds you need to open the Google doc and the tab furthest to the left called; “Opening odds 0% min. value”.
| Opening odds | |
| Turnover | 151 |
| Average odds | 3.06 |
| Profit/Loss | 11.47 |
| Yield | 7.60% |
| Expected profit | 20.54 |
| Expected yield | 13.60% |
As can be seen above, placing a one unit stake, per bet which our model indicated had value, would have netted a profit of 11.47 units. This amounts to a yield of 7.6 percent.
The model being profitable for this tiny sample of bets feels good, especially so as Pinnacle employs a six to seven percent margin on these markets. This means that selecting bets randomly, in the long run, should produce a loss similar to this margin.
However, the profit is substantially smaller than the model expected it to be. Based on the model’s valuation of the bets, it expected to get a profit of 20.54 units and a yield of 13.60%. This is clear cause for concern.
Furthermore, the above is not very realistic for several reasons. We would obviously not always be able to place wagers on the opening odds. This is why we will also check the model’s performance versus the closing odds later. Furthermore would it really make sense to take very marginal wagers? I would never place a bet at odds 3, if the model indicates that the correct odds is 2.99 or even 2.95. Surely it would be wiser to operate with some safety margin and only bet when the value of the bets exceeds some threshold?
Below we will present a similar table as above, but this time we have assigned a minimum threshold of five percent value. The table shows the results if we would only take bets with as a minimum five percent value. The stats the table is based on, is like the stats from any table in this article, available in this Google doc. The names of the different tabs mirrors the names of the tables they are supporting.
| Opening odds, minimum value 5% | |
| Turnover | 95 |
| Average odds | 3.79 |
| Profit/Loss | 22.21 |
| Yield | 23.37% |
| Expected profit | 19.06 |
| Expected yield | 20.06% |
Setting a threshold of minimum five percent value for the bets we would take at Pinnacle’s opening odds, decreases the number of bets to 95. However, it increases the profit to 22.21 units. This makes the yield a tasty 23.37 percent, which is just above the expected yield of 20.06 percent. Even so, the sample size of 95 bets and the average odds of almost 4, means that variance, or the luck factor to use other words, plays a huge part in the results.
Nevertheless, a yield of more than 20 percent is in my opinion very impressive.
Above we have checked the model’s performance versus the opening odds, but how does it perform against the closing odds, which we fully expect to be harder to beat? As can be seen in the table below the 160 bets recommended by the model at the closing odds would approximately have broken even, producing a tiny profit of 0.26 units and a yield of 0.16 percent. This is far worse than the expected profit/yield of 13.84 units and 8.65 percent. Getting the stakes back when going up against the closing odds is far from impressive, however it is still far from bad if compared against the performance of other models available in an open forum. Money back is not so bad when we remember that random betting is likely to lose some six to seven percent.
| Closing odds | |
| Turnover | 160 |
| Average odds | 2.04 |
| Profit/Loss | 0.26 |
| Yield | 0.16% |
| Expected profit | 13.84 |
| Expected yield | 8.65% |
We will also produce data for the more realistic scenario where we impose a minimum value threshold of five percent. Our model produced 83 bets with at least five percent expected value measured against Pinnacle’s closing odds. These bets produced a profit of 2.59 units and a yield of 3.12 percent. Far worse than the model’s expected profit of 12.11 units, which would have produced a yield of 14.59 percent.
| Closing odds, minimum value 5% | |
| Turnover | 83 |
| Average odds | 2.59 |
| Profit/Loss | 2.59 |
| Yield | 3.12% |
| Expected profit | 12.11 |
| Expected yield | 14.59% |
Based on this small sample the indications are that, as expected, the closing odds seems to be sharper than the opening and that our model all things considered is doing rather well. In my opinion the comments that our simulations is not very good seems to be unfair, even if the sample sizes are too small to say much for sure.
In our fourth and final article about the performance of our biathlon model, we will try to assess these results a bit more in-depth and look into how likely the rather good-looking results are to be caused by luck or skill.
