Uncovering the sources of team synergy: Player complementarities in the production of wins

3.1Player complementarity

To demonstrate what we mean by team synergy, consider the hypothetical relationships on a baseball team displayed in Fig. 1. For each panel, we display—through several contour lines—the mix of player talents across two particular positions on a baseball team holding the number of team wins fixed (where darker lines moving towards the northeast signify increases in team performance).5 On any contour line, the slope at a particular point denotes the increase (decrease) in WAR required at one position to displace a loss (increase) in WAR at the other position in order to hold overall team performance constant. The degree to which players are substitutable or complementary determines the curvature of these contour lines.

Fig.1

Hypothetical complementarity of select player relationships by position or placement in the batting order in terms of wins-above-replacement (WAR). DH refers to designated hitter, SP references starting pitcher, 2B is shorthand for second baseman, and SS denotes the shortstop position.

We propose that the stronger complementarities are among players the greater the scope that exists for team performances to be differentiated on the dimension of synergy. In the top left panel, we display what this relationship might look like for a designated hitter (DH) and a starting pitcher (SP) on the same team. Given that these two types of players will never find themselves on the field at the same time, and also engage in extreme types of activities, it’s natural to imagine that their performances are perfectly substitutable, consistent with the linear contour lines in this panel. In other words, for a team to achieve +2 wins (medium gray line) across their DH and SP positions they could obtain any combination of +2 WARs among the two positions. One possibility could be getting a strong +2 WAR designated hitter and a replacement level pitcher, or alternatively a strong +2 WAR starting pitcher and a replacement level designated hitter, or maybe a more evenly split +1 WAR at both positions. This sort of neutral interaction among players mirrors the implicit assumption underlying the construction of WAR values for teammates and how they map in the aggregate into a team’s WAR.

Now, consider the relationship between the 3rd and 4th place hitters in a team’s lineup, as displayed in the top right panel of the figure. For the 3rd and 4th hitters in a lineup, it is much more natural to assume that their performances are intricately linked when determining team performance. As good as the 3rd hitter might be (e.g. batting.300 and consistently getting on base), without a 4th hitter to either drive him in to score runs or protect the 3rd hitter from getting intentionally walked in pivotal hitting situations (e.g. runners on base with two outs), the team’s performance is likely to suffer. The complementary nature of these two players’ performances in dictating overall team performance is captured by the strong kinks in the contours displayed in this panel. For that same team to achieve +2 wins at the most cost-effective point, it will be important to have exactly a +1 WAR 3rd and +1 WAR 4th hitter. In this case, having a stronger (i.e. +2 WAR) 3rd hitter is only valuable in so much that the team has a capable 4th hitter to complement him. These sorts of complementarities are very certainly lost in the construction of WAR at the individual level and, thus, also by aggregating to the team level.

Of course, the 3rd and 4th hitter might be an extreme case as well. In the bottom panel, we instead display what might be the appropriate degree of complementarity between the second basemen (2B) and shortstop (SS). Here, one could imagine that some amount of substitution exists between these two defensive positions: a shortstop with incredible range might be able to cover up for a slow-footed second baseman in fielding ground balls up the middle. Alternatively, it would be natural to imagine that the two positions also have a degree of complementarity in that both of these players are also necessary for a team to successfully turn a double play. Furthermore, depending on where these two positions bat in the lineup, further offensive interdependencies may also come into play. Keeping with the notion of a team trying to accrue +2 wins across the two positions, this example is a balance of the previous two. While a +1 WAR second basemen and +1 WAR shortstop will yield +2 wins, a spectrum exists of the possible combinations of WARs across the second basemen and shortstop that would still yield +2 wins for the team. Note this spectrum is not as interchangeable as the hypothetical relationship between the starting pitcher and designated hitter, but some substitutability does exist unlike the hypothetical 3rd and 4th hitter relationship. Importantly, however, even the more moderate degree of interdependency displayed in this panel would not be captured in the construction of WAR.

3.2WAR and team wins

Instead of building a structural model of how individual players contribute to the different outcomes through the course of a game and then how these outcomes translate into the likelihood of a team winning a game, we take a more aggregate (reduced form) approach. Specifically, we search for systematic correlations in the misspecifications of WAR across teammates by incorporating into our analysis how a team actually performed. If these misspecifications are systematic both across teams and individual players as teammate relationships change, our statistical model will attribute them to the player complementarities that must exist between teammates along the lines of what is described in Fig. 1.

We show here that this tends to manifest itself in the fact that simply summing the WAR values for a team across its players does not perfectly replicate its wins above those expected of a team comprised entirely of replacement-level players. To get a sense of exactly how important player interactions may be to team performance, we regressed the number of wins for each team on the sum total of its players’ WAR. Specifically, we ran linear regressions of the form

where W_nt is the number of wins of team n in season t and WAR_nt is the sum total of wins-above-replacement statistics for all players on team n in season t based on either FanGraphs or Baseball-Reference’s calculations.6.

The ɛ_nt in these regressions are what we call team productivity residuals. We refer to them as such because in many ways they represent the baseball equivalent of the famous “Solow residual” used in economics to measure the productivity of firms.7 An MLB team with a positive ɛ_nt was a team who out-performed, or won more games than what could be attributed to the sum of its individual player performances (or in economic terms, a firm that produced more output than the usage of its individual inputs would suggest). Alternatively, a team with a negative residual would be a team who despite perhaps having a number of strong individual performances (as measured by WAR) under-performed as it pertains to wins.

The results from these regressions using data from the 1998-2016 seasons, shown in Table 1, provide several insights. First, it is clear that the estimates of β are close to 1.8 This is intuitive given how both WAR metrics are constructed, but also allows us to confidently use the idea that increasing a team’s WAR should have a one-to-one relationship with their number of wins.9. Furthermore, the estimate for α in our regressions is just less than 50. This estimate, too, has a natural interpretation of being the number of wins one would expect a team full of replacement level players to accrue. At about 50, clearly a team with only replacement level players is far from an average, or 0.500 winning percentage, team. With that being said, it is consistent with the construction of these measures.

The team productivity residuals, ɛˆnt , are our estimates of the element of team performance that is unexplained by the sum of its players’ individual performances, and the variation that we may potentially attribute to a team’s synergy. Based on the R² values of these regression, this amounts to about 20% of the variation in team wins in our sample. Figure 2 further demonstrates just how important this element is by plotting a kernel density function of ɛˆnt from both regressions (solid lines). With a standard deviation of about 5 wins and a range equal to approximately 30-40 wins, it is evident that a considerable portion of the variability in team performance cannot be explained by WAR. This unexplained variation can be pivotal considering that a standard deviation increase in a 0.500 winning percentage team’s productivity residual would likely make the difference in becoming a playoff team. Our contention is that at least part of this variation is the result of attempting to construct an individualized measure of performance while abstracting from the complex set of interdependencies amongst teammates.

Fig.2

Kernel densities of team productivity residuals for various measures of wins-above-replacement (Fangraphs (fWAR) - panel A, Baseball-Reference (bWAR) - panel B) and our WAR^- metrics.

To see why, consider the following interdependencies captured in our regression. In constructing a roster, teams face a problem of maximizing wins, W, subject to a payroll constraint and MLB regulations like the luxury tax. Suppose this production function, w, takes the form

where w takes as its inputs a team’s defensive and offensive production.¹⁰10 Defensive production is captured by d (f, p), a function that describes how fielding (f) and pitching (p) resources determine defensive value; and offensive production is captured by l (b), which specifies how batters (b) determine offensive value given a lineup configuration. If the team’s defensive ability depends upon the specific mix of fielding and pitching inputs, or if teams maximize the returns to their batter’s output based on their order in the lineup, then team synergy will be evident in the degree of substitution/complementarity between inputs.

We do not observe the functional form w takes for each team, but we do have the extensive work of sabermetricians to appeal to on this matter. In fact, the construction of WAR resembles this production function in many ways, as it incorporates fielding, pitching, and hitting metrics separately by converting them to run-equivalent values for each player which are then translated to team win values by using an historical run differential-win relationship. If the technology for turning player talents into team wins is linear with respect to the sum of its players’ individual WAR, i.e. if player performances are perfect substitutes, then the relationship below across the players (i) on a team in a given season should hold and the residuals of our regression should be zero in the absence of other contextual factors contributing to the deviation of team wins and team WAR.

However, if complementarities exist between teammates causing this relationship to be nonlinear, then our regressions will be misspecified.

When we replace WAR in our regressions with our measure that adjusts for the strength of teammate interactions, WAR^-, this is exactly what we see. In this case, the higher R² values of these regressions suggest that accounting for player performance interactions reduces the unexplained variation in team wins by roughly 40%. This remains the case even after we account for sampling variability and estimation uncertainty by examining the Pseudo R² values from a k-fold cross-validation of these regressions.¹¹11 The result of this adjustment on our team productivity residuals can be seen in Fig. 2. The kernel density of team productivity residuals using our WAR^- measures (dashed lines) is much more concentrated than before with a standard deviation of about 4 wins and a range of about 25 wins.¹²12 Taken together, these results suggest that complementarities between individual players, or what we call team synergy, do indeed have scope for explaining some of the unexplained variation in team performance by WAR.

3.3Team wins and teammate interactions

Next, we focus on our methodology for decomposing team productivity residuals into player-specific productivity residuals. The basis for this decomposition is the following identity,

To arrive at this value, first we construct two weights designed to capture how a player’s placement in the batting lineup (l_it) and his defensive position (d_it) affect his expected contribution to team wins,

where S_ijt denotes the number of times player i appeared in the jth slot of the lineup and g_ijt denotes the number of times player i appeared at each of the eight non-pitching defensive positions or pitcher/DH as a share of his total appearances.¹³13 The adjustment weights, b_j and p_j, then describe the relative importance weight given to their respective variables. Lineup weights are defined following Tango et al. (2007), while defensive position weights are defined following FanGraphs’ positional adjustment methodology (FanGraphs, 2016a) for fWAR and Baseball-Reference’s positional adjustment methodology for bWAR (Baseball-Reference, 2017). We then normalize each weighting scheme to sum to 1, with the resulting values reported in Table 2.

In essence, with l_i and d_i we are skill-weighting the amount of time played at defensive positions and in the lineup in our calculations of the expected contribution to team wins. We use the Fangraphs or Baseball-Reference positional weights for this purpose, because they reflect a value judgement of the relative skill required to play each position holding fixed offensive skill. Similarly, we use the the Tango et al. (2007) lineup weights because they put a premium on time spent at the lineup positions that turn over more regularly throughout the course of a game. In this sense, they reflect the relative likelihood of getting additional plate appearances in a season, rather than any notion of proximity toward other players in the lineup.

With these weights in hand for each season, we then proceed to the construction of Wˆint for each player based on his position and his share of playing time,

where we refer to the product η_itτ_it as a player’s appearance weight in each season. FanGraphs and Baseball-Reference construct their WAR measures such that players contribute 1,000 WAR per 2,430 games league-wide (162 games for 30 teams), where, by construction, the offensive WAR contributions of pitchers sums to zero. The η in the above equation correspond to the proportion of league-wide fWAR/bWAR apportioned to position players and pitchers, respectively. This split is based on the assumption that because position players appear on both sides of the ball their contribution should be larger (FanGraphs, 2016a) as well as the relative split of salaries for free agent pitchers vs. hitters (Baseball-Reference, 2017). We maintain this assumption here, as it is also in keeping with the fact that we do not use offensive fWAR or bWAR data for pitchers in our analysis. To obtain τ, we use the sum of plate appearances (PA) and defensive outs (DOuts) for position players differentially weighted by the lineup and defensive position weights reported in Table 2. For pitchers, we use outs recorded (POuts) as we found it to be the most reliable measure for capturing the differences in pitching contributions across starters and a variety of relievers (middle relievers, closers, etc.).¹⁴14 Finally, we scale these inputs on a per-game basis, dividing plate appearances by a three appearance per-game scalar (3*162) and defensive and pitching outs by a 27 outs per-game scalar (27*162).

When aggregated across players on a given team in a given season, our player productivity residuals measure the difference between a team’s actual win count and what it would be expected to be based on the sum total of individual player performances as measured by WAR. Stacking these residuals into a player-season by team (IT × N) matrix ɛˆ , we model the interactions between player residuals as a panel spatial autoregression (SAR),

where A is an IT × IT network matrix identifying teammates in a given season. Typically, such a matrix is symmetric with 0’s on the diagonal and 1’s off the diagonal “connecting” teammates. However, in order to capture potential interdependencies in teammate relationships which correspond to their positions in the field and lineup, we replace the 1’s with weights α_ijt. For a pairing between player i and j playing for team n in season t, these connection weights are defined as follows:

This formalization of the network structure of our model captures several hypothesized features of teammate connections. First, the more one or both players in a pairing play, the more likely they will have played together and the stronger their on-field connection will be. Second, the implicit orderings of our lineup and defensive position weights shown in Table 2 capture specific on-field dynamics. If both players in a pairing tend to bat higher in the lineup, they will be more likely to affect each other’s performance based on the greater number of game situations they are expected to be a part of over the course of a season. Similarly, defensive pairings that include a catcher will be given relatively more weight, and if the other player is, for example, a middle infielder, this pairing will receive greater weight, all else equal, than one with a left fielder. Then, because pitchers receive a defensive weight equal to one, pitcher-catcher relationships will receive more weight than other position pairings, all else equal. Finally, to allow for added weight to be given to repeated “connections” across seasons in explaining player performance interactions, we sum over this value for each previous season in which the players were teammates.

Next, we assume that a factor structure exists for the panel SAR residuals, υ, such that player productivity residuals are summarized by a player-season specific component, or factors F, as well as a team-specific component, or factor loadings Λ. The F trace out a player’s career arc, potentially across several teams, and reflect whether that player finds himself among over- or under-performing teammates in each season. Identification of this latent variable is, therefore, predicated on roster turnover. Because of this, it will be more difficult in general for us to establish such a player vs. team breakdown the less roster turnover exists on a team over time. The Λ, on the other hand, reflect an organization’s average historical tendency to over- or under-perform relative to the collection of its players.

Solving for ɛˆ yields our spatial factor model with spatial weight matrix Ω = (I - ρA) ^-1,

where F is an IT × 2 matrix of our player-season factors and Λ is an 2 × N matrix of their team-specific factor loadings. To estimate this model, we use a two-step estimation procedure described in the Appendix. In the first step, an estimate of ρ is obtained by maximum likelihood conditional on a scale normalization on A. Given ρ, the factor model is then estimated by spatial principal components analysis (SPCA) to extract the latent player-season and team-specific components up to a scale normalization on Λ (Demsar et al. (2012)). In the next section, we provide further motivation for what we aim to capture in these factors in terms of team synergy.

4.1Sources of team synergy

Our methodology for measuring team synergy boils down to nothing more than a decomposition of the spatial correlation matrix of teammates’ productivity residuals into an exact linear combination of latent factors. To see this, consider that we can decompose our player productivity residuals into two parts: 1) a part that is unique to each player that we attribute to measurement error in team productivity residuals, and 2) a part that can be explained by each player’s interactions, or spill-overs, with his teammates that we attribute to team synergy, where the scalars w correspond to the entries of our spatial weight matrix Ω,

It is important to note here the role played by the measurement error term. In the absence of systematic spatial correlations in the player productivity residuals of teammates, this term will dominate our results. In this sense, it is an “out” for the model that allows it to explain the variation in player productivity residuals solely as a function of individual circumstances.¹⁵15 Based on our findings in Table 1, roughly 60% of the variance of the residual component between team wins and team WAR fits this description. The remaining 40% is what we then capture in the team synergy component.

We associate positive spill-overs with “good team synergy” and negative spill-overs with “bad team synergy.” We do not take a stance on what drives these spill-overs between teammates; and, in all likelihood, our latent factors probably capture a combination of many of the determinants of team synergy that others have already explored. However, by not restricting them ex-ante, they likely also embody elements of team synergy that could not be measured previously. The extent to which we do provide context for our factors is only to appeal to the work of other social scientists who have singled out certain psychological traits, such as “character” and being a “team player,” as being attributes of individuals in groups that excel in working together.

By allowing for two factors and restricting their loadings such that F = [ch, tp] and Λ = [l, λ], where l is a unit vector across teams, we can restrict our factor model to embody similar features.

We think of the factor ch as capturing a player’s innate character, as through this factor players demonstrate spill-overs to their teammates which do not depend on the identity of their team. In contrast, we think of the factor tp as capturing a player’s contribution that is more closely linked to the “match quality” of his current team (via λ), which we term as the team player factor.

Teams with large |λ| are then said to exhibit good organizational culture, as they either reinforce positive spill-overs (tp < 0 & λ > 0) or minimize negative spill-overs (tp > 0 & λ < 0) between teammates. Notice, however, that in our framework these factor loadings are fixed across time. As such, their estimation accounts for the vast majority of the uncertainty associated with our spatial factor model. For example, when a new season’s data is added to the model, the inference of λ applied to the previous seasons’ factor values for all current and former players on each team must be updated as a result. Looking at how estimates of λ evolve over time can then give a sense of changes in the model’s interpretation of an organization’s culture.

Figure 3 plots rankings from zero to 100 for all 30 MLB teams across the 1998-2016 seasons based on our estimated values of |λ| using both fWAR and bWAR data. To capture variation over time in these rankings the figure contains box and whisker plots for each organization summarizing the distribution of rankings obtained by estimating our spatial factor model “recursively” by adding one season at a time to the 1998 data. The dots in the figure correspond to the median ranking for each organization, while the bars give a sense of the interquartile range and broader sample variation over time. Certain organizations stand out along this dimension. For instance, the St. Louis Cardinals, Arizona Diamondbacks, and San Francisco Giants are in the top three of both rankings; while others do not fair nearly as well. Several organizations near the middle to bottom of the rankings, however, also exhibit a very large amount of variability over time, suggesting that for these organizations substantial changes in culture occurred during this time period.

Fig.3

Ranking of MLB organizations on the basis of our Organizational Culture metric for competing measures of wins-above-replacement (Fangraphs (fWAR) - panel A, Baseball-Reference (bWAR) - panel B). Organizations are ranked on a relative scale from 0 to 100, with 100 equal to the best organization over the 1998-2016 seasons. Dots denote the median ranking over this sample period for each organization, with box and whiskers summarizing sample variation.

4.2Team performance

If WAR measurements are indeed influenced by teammate interactions, then the regressions underlying our team productivity residuals are misspecified. Namely, WAR may be under- or over-counting the importance of individual contributions to team wins by ignoring the interactions between teammates. To adjust for this possible source of bias, we construct an alternative measure called WAR^- which subtracts from the WAR of each player the portion of his productivity residual that can be explained by his teammates’ residuals. In network statistics, this is often referred to as the “in-degree” for a node.

Recall that Fig. 2 demonstrated the relative importance of adjusting WAR in this way for explaining deviations of team productivity residuals from zero. We can get a sense of the impact that this adjustment has on the productivity residual for any individual team by examining the aggregation of the differences between WAR and WAR^- over teammates in each season. This is often referred to as the network’s “total-degree.” We call it “team complementarity wins-above-replacement,” or tcWAR, and scale it by -βˆ from the regressions in Table 1 so that we can relate it directly to Fig. 2.

Figure 4 scatters a team’s productivity residual in each season against its tcWAR. The figure is constructed so that the x-axis coordinate (tcWAR) is equal to the number of team wins (y-axis coordinate) explained by team synergy. Some of the best and worst teams on both ends of the synergy spectrum are noted in the figure for both fWAR and bWAR results. While not identical, the teams that are singled out by both metrics on the basis of tcWAR overlap to a large degree. For instance, the 2012 Orioles, 2008 Angels, 2007 Diamondbacks, 2006 Athletics, and 1998 Padres all show up as teams with large positive tcWAR values and the 1998 Mariners, 1999 Royals, and 2015 Reds all show up as teams with large negative tcWAR values. While the size of the team productivity residuals tends to vary across fWAR and bWAR, the number of team wins that each metric attributes to team synergy remains fairly similar. For example, of the 2012 Orioles’ nearly 15 team wins above fWAR’s expectation, tcWAR attributes roughly 4 of these to good team synergy. In contrast, of the 2012 Orioles’ nearly 8 team wins above bWAR’s expectation, tcWAR attributes roughly the same number to team synergy.

Fig.4

Scatter plot of our team complementarity WAR (tcWAR) metric against our team productivity residuals for all MLB teams over the 1998-2016 seasons. Outlying team-season values referred to in the text are labeled for competing measures of wins-above-replacement (Fangraphs (fWAR) - panel A, Baseball-Reference (bWAR) - panel B).

Examining our tcWAR estimates on an organization-by-organization basis reveals that it is fairly rare for a team to over-achieve in terms of team synergy (tcWAR > 0); and, furthermore, those teams that do demonstrate very little persistence on this dimension. To confirm this, we regressed the current season’s tcWAR on the previous season’s value for the full panel of 30 MLB teams over the 1998-2016 seasons. These regressions for fWAR and bWAR data produced remarkably low estimates of first-order autocorrelation in team synergy; and, hence, exhibited very strong mean-reverting properties.¹⁶16 In this respect, our results are consistent with the notion that team synergy may be accurately described as “catching lightning in a bottle.” It is, therefore, likely to be an aspect of team performance that must be closely monitored and constantly managed by organizations.

To identify organizations that have exceeded and fallen short of expectations on this dimension, we took the residuals from these regressions and summed them over time for each organization into a metric that we call “team synergy wins-above-expected.” Figure 5 presents the results of this exercise, ranking organizations from over- to under-achievers during our sample period. As with our Organizational Culture rankings, here, too, there exists some variability across fWAR and bWAR in interpreting team synergy. In some instances, the differences can be quite pronounced; as they are for the San Franciso Giants who top the bWAR rankings with nearly 10 wins above expected, but fall in the middle of the pack in the fWAR rankings with about 1 win above expected. A few organizations, however, stand out in both rankings, such as the Oakland A’s, Chicago White Sox, New York Yankees, Los Angeles Dodgers, and St. Louis Cardinals.

Fig.5

Cumulative team complementarity wins-above-expected for each MLB organization over the 1998-2016 seasons based on residuals from a regression of current season tcWAR on its previous season’s value. Panels of the figure display results for competing measures of wins-above-replacement (Fangraphs (fWAR) - panel A, Baseball-Reference (bWAR) - panel B).

The apparent lack of persistence in team synergy raises the question of what value tcWAR holds for a team or analyst. Therefore, to demonstrate its value we next show that the highly mean-reverting properties of team synergy are something that can be exploited to improve upon PECOTA’s pre-season team win projections. First, though, we consider the possibility that the information on team synergy found in tcWAR is already captured in PECOTA’s player-based projections. Table 3 contains the coefficients obtained by regressing PECOTA projections for the 2008-2016 seasons on the previous season’s projection, recursive estimates of the previous season’s tcWAR computed using only data through the previous season, and the combined previous season’s value of WAR for the current season’s roster. Interestingly, we find that at least part of what we measure in tcWAR does seem to be reflected in the PECOTA projections on either an fWAR or bWAR basis according to these regressions. This would seem to set a high bar then for tcWAR to provide any value-added over PECOTA. We show, however, that a very simple forecasting model incorporating the previous season’s team wins, the PECOTA projection, and a projected value of tcWAR based on the first-order autoregression described above can do just that.

To see why this is the case, Table 3 also contains the results for these regressions using the full sample of data. Even after accounting for the PECOTA projection, the regressions calculated on either an fWAR or bWAR basis still load significantly onto tcWAR, suggesting there is information in our metric that is not captured by the PECOTA forecast. Using out-of-sample one-step ahead projections from this regression, we find that it would have been possible to improve upon PECOTA pre-season projections by a statistically significant margin of roughly 1 win based on a Diebold and Mariano (1995) test of equal mean absolute error across models. While the magnitude of this improvement may seem small, for our purposes it is sufficient to demonstrate that tcWAR contains information that is not already summarized in PECOTA. We leave it to future work to determine whether or not this result can be improved upon, perhaps through a reconfiguration of PECOTA’s projection system to also account for the player complementarity effects that we discuss next.

4.3Player evaluation

We can also refine WAR as a measure of player performance by taking into account how much a player affects his teammates’ performances. Here, we add to WAR^- the contribution of each player to all of his teammates’ productivity residuals, or what is referred to in network statistics as the “out-degree” of a node. We call this measure WAR⁺.

Figure 6 scatters WAR^- and WAR⁺ versus WAR on an fWAR and bWAR basis. Interestingly, WAR^- and WAR on an individual player-season basis are very highly correlated, with the plotted points clustered fairly closely around the 45 degree line. Thus, it is the aggregation of somewhat small differences at the player level that leads to the drastic reduction in the unexplained variance of team performance in Table 1 and Fig. 2. For WAR⁺, on the other hand, the differences are much more pronounced. In particular, our analysis suggests that WAR overestimates the relative performance of low impact (WAR < 1), and underestimates the relative performance of high impact (WAR > 4) players on team performance.¹⁷17

Fig.6

Scatter plot of competing measures of wins-above-replacement (Fangraphs (fWAR) - panel A, Baseball-Reference (bWAR - panel B) against our WAR^- and WAR⁺ metrics for MLB players spanning the 1998-2016 seasons. Solid lines in each graph correspond with 45 degree lines, while the vertical lines denote threshold values from Fangraphs for Scrub/Role (WAR = 1) and Good/Star (fWAR = 4) players.

The difference between WAR⁺ and WAR can be used to evaluate players on the basis of their contribution to team performance through their impact on their teammates. In network statistics, this is what is called the “net-degree” for each node.

In keeping with our terminology above, we instead refer to it as “player complementarity wins-above-replacement,” or pcWAR. In Fig. 7, we plot the pcWAR for all player-season combinations in our dataset relative to a player’s WAR on an fWAR and bWAR basis. Notice that summing a player’s pcWAR and WAR reproduces our WAR⁺ metric, such that adding the x-axis and y-axis coordinates for each player-season in this figure provides a sense of his true value to his team.

Fig.7

Scatter plot of competing measures of wins-above-replacement (Fangraphs (fWAR) - panel A, Baseball-Reference (bWAR) - panel B) against our player complementarity WAR (pcWAR) metric for MLB players spanning the 1998-2016 seasons. Vertical lines denote threshold values from Fangraphs for Scrub/Role (WAR = 1) and Good/Star (WAR = 4) players.

The conventional wisdom that good players make their teammates better is confirmed by our analysis of pcWAR, as Fig. 7 demonstrates a strong positive correlation exists between pcWAR and WAR for all player-season combinations in our sample. The vertical lines in the figure correspond to thresholds for WAR used by FanGraphs to distinguish Good from Star players (WAR = 4) and Scrub from Role players (WAR = 1). Star players tend to add anywhere from about 0 to 1.5 wins to their team through their indirect impact on the performance of their teammates, whereas Scrub players tend to add from about 0 to 0.5 losses to their team. In between, there exists considerable variation with players contributing from -0.5 to 0.5 wins.

The impact of Star players on their teammates likely comes through so strongly in our analysis because they are among the most talented; and, therefore, have skill sets that are just naturally likely to be more complementary to others on the team in a variety of ways. We show below, however, that even still there exists a considerable amount of diversity across these players in how much this is the case. Part of the reason for this likely reflects the team-related aspects of complementarity, i.e. the player is just a bad fit for the team as a whole, but part also boils down to the player’s ability (or willingness) to adapt to his teammates.

Figure 8 ranks all active players through the 2016 season on the basis of their career average pcWAR values.¹⁸18 The left-hand panel of the figure shows the top 25% of players on this dimension, while the right-hand panel shows the bottom 25%. Many of the top players in the game dominate our leaderboard, with Mike Trout the undisputed champion in this regard, averaging over one-half win of additional value over the course of his career. While our estimates for pcWAR may seem small at first glance in terms of win value, they are of a non-trivial economic value. With a team win valued at roughly $6 million in MLB, the value of team synergy alone for some of the game’s best players is just as high according to our pcWAR metric as what WAR would assign to a typical role player on the team (Cameron, 2014). In fact, even a player whose WAR was 0 and pcWAR was as low as 0.1 would still be worth paying the MLB minimum salary.

Fig.8

Ranking of active MLB players through the 2016 season on their career average pcWAR values broken down into contributions from player- and team-specific components and constructed from competing measures of wins-above-replacement (Fangraphs (fWAR) - panel A, Baseball-Reference (bWAR - panel B). Left-hand side of each panel displays the top 25% of rankings, while the right-hand side shows the bottom 25% of rankings.

The figure also breaks down pcWAR into separate components due solely to characteristics of the player (e.g. the contributions from the “character” factor of our model) versus other contextual factors related to the team (e.g. the contributions from the “team player” factor of our model). The former component of pcWAR is potentially of value for teams looking to alter their synergy profile through trades or free agency, as it strips out any previous organizational effects. For example, a common criticism of general managers of the consideration of leadership qualities in the evaluation of another team’s player is that it is difficult to ascertain how much of a player’s past performance is the result of his previous team environment versus some innate ability (Olney, 2018). Our method allows for a disentangling of such effects.

At the player level, team synergy is also much more persistent, with pcWAR lending itself more easily to prediction than tcWAR. This can be seen in Table 4 in the coefficients of the regressions of the current season’s pcWAR on the talent level of the player (e.g. Star, Role, Scrub) based on his previous season’s WAR and its interaction with the previous season’s pcWAR value. The persistence of a player’s complementarity effects is increasing in past performance, with Star players exhibiting nearly five times the persistence as Scrub players and about 1.5 times as much as Role players. This suggests that player complementarity expectations based on past performance may be an appropriate guide for teams to judge their own players.

Just as we did with teams, we can use these regressions to define “player complementarity wins-above-expected” by taking their residuals and summing them over a player’s career. Figure 9 plots the resulting measure against each player’s average of his previous seasons’ WAR values. The vertical lines in the figure correspond to the FanGraphs thresholds, while the horizontal lines are used to highlight the extremes of the distribution. Each dot in the figure then represents a player’s career complementarity wins-above-expected, with notable examples highlighted in order to identify players in our sample who have exceeded or fallen short of expectations on this dimension.

Fig.9

Scatter plot of individual players’ career complementarity wins-above-expected against the average of his previous season’s wins-above-replacement over the 1998-2016 seasons for competing measures (Fangraphs (fWAR) - panel A, Baseball-Reference (bWAR) - panel B). Vertical lines denote threshold values from Fangraphs for Scrub/Role (WAR = 1) and Good/Star (WAR = 4) players, with outlying player values referred to in the text labeled in each panel.

The good players make their teammates better paradigm is also highly evident in this figure, but the proximity (or lack thereof) between certain players also draws out some interesting comparisons. For instance, the early career of Clayton Kershaw and late career of Randy Johnson look very similar on this dimension whether they are measured on an fWAR or bWAR basis. In contrast, Derek Jeter represents an extreme outlier in this analysis for a Star player with a career complementarity wins-above-expected that is both negative and from four to six wins less than Adrian Beltre, the career leader during our sample period. There are also a handful of Role players according to WAR with career complementarity wins-above-expected on par with the top 10 Star players in our sample (e.g. Mariano Rivera, Carlos Beltran, and Jim Thome), and a handful of Star players that show negative career complementarity wins-above-expected (e.g. Jose Abreu, Albert Belle, and Derek Jeter) on an fWAR or bWAR basis.

5.1Age-position profiles and intangibles

We construct our conditional average age-position profiles for player complementarity by extending the pcWAR dynamic regressions discussed above according to,

where pos is an indicator variable for a player’s primary defensive position, including the designated hitter and a “utility” category for players who tend to play multiple defensive positions, age is a player’s age, X is a vector of player characteristics including WAR and controls for MLB and team games played, and Z is a vector of league, team, and manager indicator variables. The estimated coefficients of these regressions are summarized in Table 4.

Figure 10 plots our conditional average age-position pcWAR profiles with 95% confidence intervals on an fWAR and bWAR basis. The conventional wisdom that older players make for better teammates is certainly consistent with these profiles, as they tend to slope upward with age on average across almost all positions. However, we want to caution anyone from taking the results from this regression as “causal” estimates of age on team synergy, as the estimated coefficient is most likely also confounding a selection effect. In other words, having good team synergy may make it more likely for a player to remain in the game for longer.

Fig.10

Age-position player complementarity profiles based on our pcWAR metric as constructed from competing measures of wins-above-replacement (Fangraphs (fWAR) - panel A, Baseball-Reference (bWAR) - panel B). Bootstrapped bias-corrected and accelerated 95% confidence intervals clustered on player observations are shown in gray, with solid lines corresponding to average marginal effects over ages from 20-40 for each position.

Some additional interesting patterns also emerge from this analysis. For instance, the slopes of these profiles tend to vary by position. Second basemen and catchers tend to have profiles that are less steep than other infielders; relief pitchers tend to have steeper profiles than starting pitchers; and the profiles of designated hitters and utility players tend to be among the steepest that we estimate. The level of the profiles also varies depending on whether or not they were constructed on an fWAR or bWAR basis, with complementarity effects generally more positive across all positions and ages when measured on theformer.

By conditioning these regressions on so many observable dimensions, we can also isolate the player Intangibles of team synergy. In other words, we can measure the individual contributions to team wins through player complementarities that are not associated with any covariates in the above regressions. We use the residuals, ξ_it, from these regressions to rank active players through the 2016 season on their career average Intangibles. Positive residuals capture players whose contributions to team synergy exceed their conditional age-position profile, whereas negative residuals correspond to players who fall short of their profile.

Figure 11 displays our Intangibles rankings, where the left-hand panel shows the top 25% of players on this dimension and the right-hand panel shows the bottom 25%.¹⁹19 Our top players are now very different than who they were for pcWAR, with the exception of Joey Votto who shows up in the top four of both rankings. Kevin Keirmaier is the undisputed active leader on this dimension of team synergy, with an average contribution of a little more than 0.1 wins coming from his Intangibles.

Fig.11

Ranking of active MLB players through the 2016 season on their career average Intangibles values constructed from competing measures of wins-above-replacement (Fangraphs (fWAR) - panel A, Baseball-Reference (bWAR - panel B). Left-hand side of each panel displays the top 25% of rankings, while the right-hand side shows the bottom 25% of rankings.

At this point, a word of caution is warranted. Many of the players who we find have negative Intangibles are the same type of player that any MLB franchise would be happy to build their team around. In other words, depending on a player’s position, his age, and current manager, etc., he could still have a strong positive influence on his teammates even despite a negative Intangibles measure. A more appropriate interpretation of the rankings in Fig. 11 is then that they provide an indication of those players who have a knack for exceeding expectations on the dimension of team synergy. We call this the “David RossEffect.”

The esteem with which the 2016 World Champion Chicago Cubs held their teammate David Ross and his contribution to their success has by now become well known. The ability of a team to identify players like him is, therefore, a potential source of competitive advantage that is made possible by our player complementarity metrics. What makes David Ross uniquely suited to our analysis is that, as a back-up catcher, his WAR defines him as a role player; but, as a teammate, he is routinely characterized as someone who makes everyone around him better. We are able to provide evidence to support these claims with our metrics.

Figure 12 plots the pcWAR and Intangibles values for David Ross through the 2016 season against his WAR values. More than one labeled instance of a season occurs whenever he was traded. For most of his career, and across several different teams, David Ross exhibited the sort of beneficial relationship with his teammates that his reputation attests to, evidenced by his mostly positive pcWAR values. Furthermore, his Intangibles reveal a player who tended to outperform his age-position profile even at low levels of WAR and with limited playing time.

Fig.12

Scatter plot of pcWAR values against competing measures of wins-above-replacement (Fangraphs (fWAR) - panel A, Baseball-Reference (bWAR - panel B) for David Ross in each season of his career. More than one value appears for seasons where David Ross was traded to another team in-season.

Players such as David Ross are true “diamonds-in-the-rough” according to our analysis, with their full impact on team performance likely to fly under the radar according to traditional performance metrics. Given how rare that we find that this type of player is in our analysis, one might expect that MLB teams would be willing to pay a premium for their services. For example, others have already documented the importance of wins-above-replacement in pricing player services in MLB.²⁰20 For this reason, we might expect our pcWAR metric to also be priced into player compensation. Whether or not this extends to a player’s Intangibles, which are much more difficult to observe than age, position, and the other variables that we condition on, is unclear.

5.2Putting a price on complementarity

To see how MLB teams have historically valued player complementarity, we run player-level regressions of log annual salary adjusted for inflation (using the U.S. Consumer Price Index) as our dependent variable on a player’s career WAR and pcWAR through the previous season.²¹21 This regression takes the following form,

where pos is an indicator variable for a players’ primary defensive position, including the designated hitter and a “utility” category for players who tend to play multiple defensive positions; age is a player’s age on January 1st of the year in which season t occurs; teamExp indicates the number of seasons the player has appeared in with his current team prior to the current season; mlbExp is the number of MLB games in which the player has appeared through the previous season; FA is an indicator variable which takes on one of three values denoting whether a player is in the pre-arbitration (0–2 years of service), arbitration-eligible (3–5 years), or free agent-eligible (6+ years) stage of his career; and α_i is a player fixed effect.

The use of a player fixed effect focuses our analysis on the variation “within” player salary histories. For this reason, we restrict our sample of players to only those with careers that began at some point during the 1998-2016 seasons. As we will soon explain, the inclusion of the FA indicator variable serves to capture important differences in how player salaries are determined throughout a career based on the labor market structure of MLB and changes in the bargaining power of players according to service time.²²22 We interact this variable with teamExp to then capture the impacts of trades and other reasons for team changes on service time and player salaries. The inclusion of pos and its interaction with age and experience then serves to capture any nonlinear variation in how players’ career earnings trajectories vary across defensive positions.

We are primarily interested in obtaining estimates for γ and β, the coefficients on career WAR and pcWAR, respectively. Interacting these variables with our service time-status indicator allows us to estimate how these skills may be differentially priced over a player’s career. As the first column of Table 5 shows, cumulative WAR is indeed priced differentially throughout a player’s career. During the pre-arbitration stage of a player’s career, a one win-above-replacement increase in career WAR through the previous season leads, on average, to a statistically significant earnings increase in the current season of 10-12 percent. For a player in the arbitration-eligible portion of his career, this increases slightly to 14 percent. Finally, players with six or more years of MLB service time see an average increase in earnings of 4-5 percent for each additional unit of career wins-above-replacement.

To understand the pricing pattern demonstrated in this result, it is useful to consider the bargaining position of the player. The pre-arbitration period corresponds to a player’s first three years of service time, measured by days spent on the 25-man roster of any MLB team. Unless released or traded, players are bound to the team that drafted them during this period. The vast majority of these players earn either a minimum salary determined by collective bargaining between MLB and the MLB Players Association or a somewhat higher salary on a season-by-season basis that is at the discretion of the team. Performance and salary are, therefore, likely to be only somewhat correlated during this time. Furthermore, even if a player were to sign a long-term contract during this time, they lack the bargaining power they would have if their services were being priced by the entire league, an economic situation referred to as monopsony.

If a player still has not signed a long-term contract after three years of service, they become eligible for salary arbitration, whereby the player and team submit proposed salaries to an independent third party that makes a binding determination on the player’s salary, largely on the basis of similar player performances.²³23 Though players still have limited bargaining power during this period, the slight increase in the return to cumulative WAR that we observe is consistent with their improved bargaining position afforded by the arbitration process. When a player has not signed a long-term contract after accruing six or more years of MLB service time, he becomes eligible to sign with any team as a free agent.

Once a player enters free agency, it is much more common for him to sign a multi-year contract. Multi-year contracts add a further complication to our regression, since their pricing reflects a weighted combination of both past and expected future performances. This could largely explain the smaller coefficient that we find on cumulative fWAR during free agency. However, the competitive landscape of free agency may also force teams to consider a broader range of factors as they submit contract offers to players. In fact, our estimated regression coefficients on cumulative pcWAR suggest that a player’s complementary skills are perhaps one of the additional things considered.

The specifications marked (1) in Table 5 show that cumulative pcWAR is also priced differentially throughout a player’s career. The effect on earnings of a one win-above replacement increase in career pcWAR is large, negative, and statistically significant (with the exception of pre-arbitration players on a bWAR basis) in the pre-arbitration and arbitration-eligible periods. This suggests that the lack of competitive pressure in these years allows teams to avoid compensating players for their complementary skills. However, this effect reverses once a player becomes eligible for free agency, as the same increase in pcWAR now leads to a small salary gain that is only statistically significant on an fWAR basis.

It may seem counterintuitive that the marginal return to a unit increase in cumulative pcWAR is roughly on par or higher than that for a unit increase in cumulative WAR during free agency. However, as a relatively scarce resource (its standard deviation is nearly 15 times smaller than cumulative fWAR or bWAR), it makes sense that cumulative pcWAR is priced as such on the margin. That said, it could also just as easily be the case that what we find reflects a team’s pricing of some of pcWAR’s underlying correlates. For instance, if MLB teams follow the conventional wisdom that high-performing players will have the biggest synergy effects, they may simply pay more for individuals who they expect will rank highly in the future on metrics such as fWAR and bWAR. The coefficient on cumulative pcWAR would then reflect this fact.

To investigate this possibility, we run an alternative earnings regression that instead considers separately our Intangibles measure (i.e. ξ) and the observed component of pcWAR (i.e. pcWAR - ξ). These results, marked as specifications (2) in Table 5, show that the two individual components of cumulative pcWAR are indeed priced differently in free agency (i.e. their coefficients have opposite signs) and in a similar fashion across fWAR and bWAR. A player with positive career Intangibles would be undervalued relative to his contribution to the team; and, conversely a player with negative career Intangibles would be overvalued. In contrast, the observed component of cumulative pcWAR for either player would still be appropriately valued. This result suggests an important application for our metrics. Though teams appear to value their players’ complementary skills, they clearly “misprice” their Intangibles. This is most likely because they are not easily observed and, therefore, difficult to evaluate in an efficient enough manner in which to price them.