Investigating the efficiency of the Asian handicap football betting market with ratings and Bayesian networks

3.1The rating system

The pi-rating is a football rating system that determines team ability based on the relative discrepancies in scores between adversaries. It was first introduced in (Constantinou & Fenton, 2013) and thereafter used in (Constantinou, 2018; Hubacek et al., 2018; 2019; Van Cutsem, 2019; Wheatcroft, 2020). Modified versions of the pi-rating also formed part of the top two performing models in the international competition Machine Learning for Soccer (Constantinou, 2018; Hubacek et al., 2018). This paper makes use of the original pi-rating system (Constantinou & Fenton, 2013), with two modifications described below.

The pi-ratings assign a ‘home’ (H) and an ‘away’ (A) rating to each team, to account for team-specific home advantage and away disadvantage. Therefore, when a team X plays against team Y, the match prediction is determined by team’s X rating H versus team’s Y rating A. The ratings are revised after each match based on two learning rates: a) the learning rate λ which determines to what extent new match results override previous match results in terms of the impact in determining current team ratings, and b) the learning rate γ which determines to what extent performances at home grounds influence a team’s away rating and vice versa. Therefore, at the end of a match between teams X and Y, the new ratings at time t are revised given the most recent ratings at time t - 1 as follows:

respectively, where

where G_oH and G_oA are goals observed for home and away teams respectively, and similarly G_pH and G_pA are goals predicted for home and away teams.

While the original pi-ratings represent a diminished expectation of goal difference against the average opponent in the data, in this paper they represent the actual goal difference expectation. Specifically, the rating equation in this paper is simplified not to include the deterministic function Ψ (e) =3 × log ₁₀ (1 + e) defined in the original paper (Constantinou & Fenton, 2013), which is a function that diminishes the importance of each additional goal difference under the assumption that a win is more important than increasing goal difference. The justification for this first modification is that, in AH, we are only interested in goal differences and thus, the motivation here is to optimise for goal difference rather than the ability to win matches.

The second modification involves the learning rate λ. In this paper, λ is multiplied by k when a match involves at least one team which had previously played less than 38 matches, according to available data. This modification aims to increase the speed by which team ratings converge for new teams during their first EPL season (each team plays 38 matches in a season), and is expected to be especially impactful during the very first season in the data since, at that point, all teams are considered ‘new’ by the rating. Therefore, the revised pi-ratings exclude ψ (e), defined above, and include k, as follows:

where k = 3 for match instances in which both teams had previously played less than 38 matches; otherwise k = 1. The parameter k was optimised in terms of minimising prediction error e. A limitation here is that the k parameter was optimised given integer inputs from 1 to 10. For future work, it is recommended that the k parameter is optimised given real numbers.

3.2The BN model

The graph of a BN model can be automatically discovered from data, determined by knowledge, or a combination of the two. Learning the correct graph of a BN from data remains a major challenge in the fields of probabilistic machine learning and causal discovery. While some structure learning algorithms perform well with synthetic data, it is widely acknowledged that this level of performance does not extend to real-world data which typically incorporate noise and latent confounders.

In disciplines like bioinformatics, applying structure learning algorithms can reveal new insights that would otherwise remain unknown. However, these algorithms are less effective in areas with access to domain knowledge, such as in sports. As a result, the BN model in this paper has had its graphical structure determined by the temporal fact that possession influences the number of shots created, which in turn influence the number of shots on target, and which in turn influence the number of goals scored. Each of these factors is also dependent on the level of rating difference between the two teams, as illustrated in Fig. 1. This type of model can also be characterised as a hierarchical Bayesian model of which the network is the structural representation, and where the nodes represent variables or hyperparameters of statistical distributions.

Fig. 1

The BN model. The Rating Difference (RD) is the only observable node in the network, determined by the pi-ratings, and represents the difference between the home team’s prior home rating and the away team’s prior away rating RXHt-1-RYAt-1 .

The temporal order of events in the BN graph naturally captures the importance of each event in predicting goals scored. For example, the graph assumes that shots on target have a direct influence on goals scored, whereas possession has an indirect influence and hence, while influential, it is assumed to be less impactful than shots on target. While the temporal order defines direct and indirect influences, note that the magnitude of direct influences is still determined by data.

For each match, the prior ratings are retrieved and the difference in team ratings is used as an input into the BN model, which is a Hybrid BN model consisting of both discrete and continuous variables, designed in AgenaRisk (Agena, 2019). Specifically, the actual input is the difference between prior home and away ratings, and is passed to the BN model as an observation to node Rating Difference (RD) in the form of

To ensure that the BN model is trained accurately with respect to the rating data, the parameterisation of the CPDs is also restricted to match instances in which both teams had previously played at least 38 matches. All the residual variables in the BN model are latent. Specifically,

It should be clear by this point that for both home and away teams: a) nodes P and p (SM) are hyperparameters of Beta node S, b) nodes S and p (ST) are hyperparameters of Beta node ST, and c) nodes ST and p (G) are hyperparameters of node G; effectively creating a Beta-Binomial Hybrid BN process.

4.1Data

The rating method, the BN model, and the betting simulation are based on data collected from www.football-data.co.uk and manually recorded from www.nowgoal.com. Table 5 specifies which of the data variables are used by the rating system, the BN model, and the betting simulation. For example, the rating system only requires information about goal data and hence, it only considers variables Date, Home team, Away team, Home goals, and Away goals. Since the ratings are used as an input into the BN model, they represent an additional BN variable and at the same time make the BN model independent of variables Date, Home team and Away team.

The data are based on the English Premier League (EPL) seasons 1992/93 to 2018/19. However, AH odds data were available only from season 2006/07 onwards, whereas ball possession data which is needed by the BN model were available only from season 2009/10 onwards. As a result, the rating system is trained with up to 27 seasons of data, the BN is parameterised with up to 10 seasons of data (since it requires possession data), and the betting simulation is performed over 13 seasons (since it requires AH odds data).

4.2Model fitting

By definition, the ratings are developed in a temporal manner. That is, for a match prediction at time t the model considers team ratings at time t - 1. For any match prediction, a team’s rating will always be based on the most recent rating prior to the date of the match under prediction, and a team’s rating will always be based on past match results.

Conversely, the BN model functions as a machine learning model independent of time and is validated using leave-one-out cross validation (LOOCV). A prediction between teams that have rating difference Z, where Z is one of the 23 RDLs as defined in Table 4, is derived from all data matches with rating difference Z, excluding the match under prediction during validation.

This combination of model parameterisation and validation with a rating system and a BN model is adopted by (Constantinou, 2018). The validation approach is unconventional because the BN model assumes no temporal relationships. When applied to past matches, it generates predictions at time t based on the whole data set which may include future match results. The reason this approach works well, without overestimating the future accuracy of the model, is because it does not matter whether the data comes from past or future. This is because the model assumes that the relationship between, for example, shots on target and goals scored remains invariant over time for the average EPL team, and empirical results support this claim. These include:

The empirical support for the model extends to demonstrating that the model can produce good predictions for matches between teams X and Y even when the prediction is derived from match data that neither X nor Y participated in (Constantinou, 2018). This claim is also supported by the results presented in this paper. Specifically, during Seasons 2006/07 to 2008/09 the following teams have had their performance determined by data that did not include any of their matches: a) Sheffield United, b) Charlton, and c) Derby. The reason this occurred is because, as discussed above, the BN model was trained with data from season 2009/10 onwards, which does not include any match instances associated with these teams. Their performance in terms of possession, shots, shots on target and goals scored was derived by other similar match situations in terms of rating difference between home and away teams.

This approach has advantages and disadvantages. The disadvantage is that, for those who are interested in how such a model would have performed in the past, the results only approximate past performance under the assumption the model would have been trained with at least the same amount of data as the test model. On the other hand, the advantage is that this approach allows us to preserve the sample size of the training data throughout validation, and this enables us to validate how the resulting model would have performed over multiple seasons without modifying its parameterisation (excluding the removal of a single sample; i.e. the match under assessment during validation).

To understand why this is important, consider evaluating match instances five seasons in the past. A temporal model would require the removal of the five most recent seasons from the training data. This would have led to limited samples for some of the predetermined levels of rating difference shown in Table 4. The limited data issue can only be overcome by reducing the number of predetermined levels of rating difference (i.e., the dimensionality of the model); but doing so would produce a different model than the one described. Instead, the approach adopted by (Constantinou, 2018) enables us to address the temporal aspect of the problem through the ratings and to preserve the fitting of the BN across all seasons tested; effectively enabling us to test the current parameterised model on multiple seasons independent of time.

5.1Pi-ratings accuracy and overall model fitting

Figure 2 shows that the optimal λ and γ parameters, that minimise the goal difference error as defined in Section 3, are λ= 0.018 and γ= 0.7. Note that while the results are based on training data from seasons 1992/93 to 2018/19, the optimisation is restricted to match instances in which both teams had previously played at least 38 matches; a total of 9,073 match instances. This is to ensure that the model is optimised on matches in which both teams have had their ratings developed by at least one football season.

Fig. 2

The optimal modified pi-rating learning rates and associated prediction error e, given k = 3, are λ= 0.018 and γ= 0.7. The results are based on training data from seasons 1992/93 to 2018/19. The optimisation is restricted to match instances in which both teams had previously played at least 38 matches; a total of 9,073 match instances.

The optimal learning rates are fairly consistent with those reported in (Constantinou & Fenton, 2013) (i.e., λ= 0.035 and γ= 0.7) on the basis of minimising goal difference error over five EPL seasons, with those reported in (Van Cutsem, 2019) (i.e., where λ= 0.06 and γ= 0.6) on the basis of minimising mean squared goal difference error over eight EPL seasons, with those reported in (Constantinou, 2018), λ= 0.054 and γ= 0.79, on the basis of minimising the Rank Probability Score (RPS) error metric (Constantinou & Fenton, 2012) over multiple leagues worldwide, and with those reported in (Hubacek et al., 2018), λ= 0.06 and γ= 0.5, where pi-ratings had been used in conjunction with Gradient boosted trees parameters to minimise RPS over multiple leagues worldwide.

However, note that the optimal learning rate λ is lower in this study, and this is likely due to the modification that performs more aggressive rating revisions to the first 38 matches of each team, since it is intended to improve the speed of rating convergence. Interestingly, the overall mean goal difference error shown in Fig. 2, e = 1.2283 (or e²= 1.509), is considerably lower than those reported in (Constantinou & Fenton, 2013) and (Van Cutsem, 2019), where e²= 2.625 and e²= 2.66 respectively, and this suggests that the modifications have had a positive impact on the ratings.

Figure 3 illustrates the expected goal difference for each of the 23 rating difference levels. Level 23 represents the highest rating discrepancy in favour of the away team, where the average expectation of the match is a score difference of –1.38 (or 1.38 goals in favour of the away team), and level 1 represents the highest rating discrepancy in favour of the home team with an expected score difference of 2.18 (or 2.18 goals in favour of the home team). The graph reveals a linear relationship between rating discrepancy and score discrepancy.

Fig. 3

Sensitivity analysis between the 23 states of RDL node and the expected goal difference, with linear trendline superimposed as a dashed red line.

As shown in Table 4, the granularity of the 23 intervals was selected to ensure that for any rating difference state there are at least 50 data points for a reasonably well-informed prior of observed goal difference. As with any discretised variable, different splits produce slightly different results. In the case of the RDL distribution, any changes in discretisation will remain faithful to the linear relationship illustrated in Fig. 3; implying that we should expect minor changes to the interval averages as long as the number of splits remains invariant and data points for each interval are maintained above 50.

Any minor model amendment is naturally expected to have minor impact on the predicted probabilities, and any minor impact is expected to have some influence on the results based on small discrepancies between predicted and published market odds (i.e., small θ values as defined later in subsection 5.3, such as θ = 1 or 2). However, no changes are expected for larger discrepancies. Since the conclusions in this paper are not driven by results that are based on such small differences between predicted and observed odds, any minor modification is not expected to alter concluding remarks.

Table 6 presents the discrepancy in prediction error e for the different hyperparameter combinations of λ and γ, relative to the prediction error generated by the optimal inputs of λ= 0.018 and γ= 0.7. Recommended values for λ and γ, that could be considered as an alternative to the optimal values of λ= 0.018 and γ= 0.7, are the ones that generate up to 0.01% discrepancy in prediction error. A total of 31 different combinations, excluding the optimal combination, generate discrepancy within the 0.01% threshold. By keeping the value for λ static, the recommended hyperparameter combinations are a) λ= 0.017 and γ= 0.65–0.73, b) λ= 0.018 and γ= 0.65–0.74, c) λ= 0.019 and γ= 0.65–0.74, and d) λ= 0.02 and γ= 0.7–0.72.

Table 6

The discrepancy in prediction error e for the different combinations of λ and γ, relative to the optimal inputs of λ= 0.018 and γ= 0.7. Darker green cells represent lower discrepancy, whereas darker red cells represent higher discrepancy. Recommended inputs for λ and γ are the ones that generate up to 0.01% discrepancy

5.2.Predictive accuracy

Predictive accuracy is measured for both 1X2 and AH distributions. The Brier Score is used to measure the accuracy of the binary AH outcome, and the RPS metric (Constantinou & Fenton, 2012) is used to measure the accuracy of the multinomial 1X2 distribution. The RPS can be viewed as a Brier Score extended to multinomial ordinal distributions.

Table 7 shows that the RPS error for the 1X2 outcomes ranges from 0.184 to 0.213 with an average RPS of 0.195 across all 13 EPL seasons. This result compares well relative to previous studies that assumed pi-ratings. Specifically, in (Constantinou, 2018) the RPS ranged from 0.187 to 0.236 for 52 different leagues worldwide, with an average RPS of 0.211 at validation, an average RPS of 0.203 for EPL matches, and an average RPS of 0.208 in the competition. Similarly, the average RPS was ∼0.2 in (Hubacek et al., 2018), according to Fig. 3, and 0.206 in the competition.

These results are consistent with those reported in Section 5.1, which show that the overall error e optimises lower in this study; i.e., the ratings more accurately predict score difference. The results from profitability presented in Section 5.3 are also consistent with these findings.

5.2.1Time-series analysis and sample size requirements

The model is trained with data the covers 13 years of data, and not all the variables could be measured throughout this period. Still, the model seems to work well without evidence of bias. This subsection investigates whether the variables used in the model show any drift over time that the model might have filtered out. Moreover, because the dimensionality of the model has been adjusted relative to the available sample size, this subsection also reports the sample size required for the model priors to be well informed by data.

Figure 4 presents the results from time-series analysis in investigating potential shifts in the predictive outputs of the model over time. The analysis is performed by increasing the training data set by a single football season’s worth of data at a time, and the shift is measured in terms of changes in the expected value of the given distribution. As shown in Fig. 4, the analysis focuses on the four main variables of the BN model; namely possession, shots, shots on target, and goals scored. Moreover, the different levels of rating difference (refer to Table 4) are categorised into four groups, and are measured with reference to the 10 specified seasons.

Fig. 4

Investigating the prediction shift over time, with reference to the four main variables of the BN model. The shift is measured in terms of the expected value of the specified distribution, and by adding a football season’s worth of data, to the training data set, at a time.

The reason the first three, out of the 13, seasons are not considered here is because (as later shown in Table 8) the BN model was not trained with data samples from the first two seasons, and this also means that any results obtained during the third season rely on very low samples. As previously discussed in subsection 3.2, the reason the first two seasons are not considered by the BN model is because the BN is trained with match instances in which both teams had previously played at least 38 matches, to allow for the pi-ratings to converge to reasonably accurate estimates before considered for model training.

Table 8

The number of data samples observed in each of the 23 intervals of Rating Difference Level (RDL) after each subsequent league data set is added to the training data. Darker red cells represent sample size less than 19 (equivalent to less than half a season’s data), orange cells represent sample size less than 38 (equivalent to more than half, and less than, a season’s data), and white cells represent sufficient sample size

The results are discussed with reference to Table 8, which presents the available samples for each of the 23 levels of rating difference, after each subsequent league data set is added to the training data set that was used to learn the BN model. The results show that many of the shifts occur in the first few seasons, and this is reasonable since the first seasons are the ones which rely on fewer samples. Shifts are also observed after adding the most recent leagues, but these shifts are largely restricted to the outputs of possession and shots on target, and involve matches with a level of rating difference between 1 to 6. The most important output of the model, which is the goal difference, remains essentially unchanged over time. Therefore, while it is reasonable to assume that shifts in playing style might occur naturally from season to season, collectively these shifts appear to have no meaningful impact on the prediction of goal difference, from which the 1X2 and AH distributions are formed.

Lastly, the results in Table 8 suggest that the model described in this paper should be trained with at least five seasons of league data, to ensure that the model priors are well informed by data. Because the model is trained with publicly available data, and which only increases over time, there is no motivation to use less data than what is currently available and hence, this requirement should not be viewed as a limitation.

5.3Profitability

The assessment of profitability is based on 13 EPL seasons and considers both the average and the best available (maximum) market odds. The simulation is evaluated both in terms of maximising profit as well as the Return On Investment (ROI). A standard betting strategy is used where fixed singe-unit bets (e.g., £1) are placed on 1X2 and AH outcomes with payoff that is higher than the model’s estimated unbiased payoff by at least θ, where θ is the discrepancy between the predicted probability and the payoff probability. For example, if the model predicts 51% and the bookmakers’ payoff for that event is 50% (i.e., odds of 2), then θ = 1; i.e., the bookmakers pay 1% more than the model’s estimate. The betting simulation is performed across all payoff decision thresholds θ, for both 1X2 and AH outcomes. The results are first discussed in terms of 1X2 betting performance with reference to Tables 9, 10 and 11; then in terms of AH betting performance with reference to Tables 12, 13 and 14.

Table 9 presents the profitability generated by 1X2 bets over all possible payoff decision thresholds θ, assuming static θ across all 13 seasons, for both average and maximum market odds. Unsurprisingly, the results show that it is much easier for the model to generate profit when taking advantage of the maximum market odds. Moreover, low thresholds θ (i.e., when the predictions are roughly in agreement with market odds) are not profitable. Interestingly, ROI maximises at much higher thresholds θ compared to profit; i.e., profit maximises at 8% and 9% whereas ROI at 18% and 16%, for average and maximum market odds respectively. This is because lower thresholds θ generate a higher number of bets which can generate higher profit even if ROI is lower.

Tables 10 and 11 show how the profitability changes when we consider the threshold θ that maximises ROI (Table 10) or profit (Table 11) per season, rather than considering a static θ across all seasons (Table 9), for both average and maximum market odds. Profit, once more, tends to maximise on lower thresholds θ compared to ROI. As an example, Table A1 provides the information used during the betting simulation to assess profitability for 1X2 outcomes, based on average odds of season 2010/11 as shown in Table 10.

The results show that the optimal threshold θ varies dramatically between seasons, and there is much to be gained by identifying the optimal θ. However, the high variance of θ suggests that is not reasonable to expect that we will be able to successfully predict the optimal value of θ before a season starts. Moreover, while the results are restricted to cases with 30 or more bets in a single season, it is clear that in many cases the sample size remains insufficient for deriving reliable and robust single-season conclusions. This means that the maximised profitability presented in Tables 10 and 11 is not a realistic expectation of real-world performance; only Table 9 is. These results are important because they highlight the danger when optimising models based on the results of a single season (or few seasons), which is often the case in the literature. This outcome is also discussed in Section 6, point iv.

Interestingly, while maximising profit per season is guaranteed to also maximise profit over all seasons (Table 11), the same does not apply to ROI (see Table 10 and compare it to Table 11). That is, optimising the betting strategy for maximum ROI per season does not necessarily imply that ROI will maximise across all seasons. Table 10 shows that even though ROI is maximised for each season independently, the overall ROI across all 13 seasons is 9.03% in the case of average odds, which is notably lower compared to the respective overall ROI of 12.96% in Table 11. However, this observation does not extend to the case of maximum market odds. This outcome is also discussed in Section 6, point vi.

Tables 12, 13, and 14 repeat the analysis of Tables 9, 10, and 11, but for AH rather than 1X2 betting. Table A2 presents an example of the information used during the betting simulation to assess profitability from AH bets, and it is based on average odds of season 2010/11 as shown in Table 13. Overall, the AH bets appear to generate both lower profit as well as ROI compared to 1X2 bets. As with 1X2 bets, optimising for maximum ROI per season leads to a lower ROI across all seasons, compared to maximising profit. Specifically, Table 13 shows that when maximising ROI per season leads to an overall ROI of 5.7% for average odds, which is lower than the respective average ROI of 6.33% in Table 14 when maximising profit. Once more, this observation only applies to the case of average market odds.

An interesting observation is that AH betting generates a lower number of bets when θ is low, compared to 1X2 betting, and a higher number of bets when θ is high. This suggests that AH betting is less sensitive to the betting decision threshold θ compared to 1X2 betting, for both average and maximum market odds. Furthermore, the optimal threshold θ for AH bets does not vary as much as it did for 1X2 bets. Despite the relatively low variance of θ and the common occurrence of winning 60+ out of 100 AH bets, profitability is still inconsistent between seasons. This is because match bets do not share the same payoff.

5.3.2Betting stake adjustments

Figure 5 presents the cumulative profit generated over eight different betting scenarios that represent the combinations of the following betting options: a) optimising for maximum ROI or profit, b) optimising θ per season or across all seasons, and c) simulating 1X2 or AH bets. The results illustrate how the difference in profit and ROI evolves across the 13 seasons between 1X2 and AH bets. While AH bets generate considerably lower profit and ROI, the profitability is much less volatile than 1X2 bets and hence, it is subject to a lower risk of loss which can often be detrimental. For example, note the significant losses for the two best performing scenarios during matches 1900 to 2100, which are both based on 1X2 bets. However, the lower risk of loss also limits profits.

Fig. 5

Cumulative profit when the betting procedure is optimised for either profit or ROI, overall or per season, and based on either 1X2 or AH maximum market odds. The results are based on 13 EPL seasons; from 2006/09 to 2018/19. Optimisations for overall profit and ROI, across all 13 seasons, are restricted to θ discrepancies that generate at least 100 bets over those 13 seasons.

A fairer assessment of risk between 1X2 and AH profits would be to simply optimise stakes such that, at the end of the betting period, they both produce the same profit. Figure 6 provides these results by extending the scenarios of Fig. 5 to include an additional betting scenario in which AH stakes are increased proportional to the difference in cumulative profit between 1X2 and AH bets. For example, in Fig. 6a the new AH betting scenario assumes an increase of 5.59 times the stakes of AH bets, in order for the cumulative profit to become equal to that generated by 1X2 bets.

Fig. 6

Comparing the volatility of profits when the stakes of AH bets is increased by as much required for the cumulative profit to match that of 1X2 bets.

Overall, the graphs suggest that if we want AH bets to generate as much profit as 1X2 bets do, then profit from AH bets will likely be subject to a similar risk of loss as with 1X2 bets. Therefore, while AH is often preferred due to the lower variance of returns, this advantage is rather eliminated when we need to bet proportionally larger to match the expected profit produced by the corresponding 1X2 bets. This outcome is also discussed in Section 6, point iii.