Two questions about calculating replacement level
by trusttheprocess on Mar 16, 2010 2:01 PM EDT
Email
Print
Apologies if this is evident to others, or if it's already been answered, but I had a question about replacement level.
I was reading an article I believe at FanGraphs discussing the value of closers (maybe in regards to the Joe Nathan injury?). They made the argument that replacement level for closers is different than it is for other positions. The argument was that you don't replace the closer with freely available AAAA type talent. You replace the closer with the eighth inning guy, replace the eighth inning guy with the seventh inning guy and so on until you're replacing the long reliever with the mopup guy and the actual replacement level reliever gets inserted into the the mopup job. This makes sense to me, but it led to two questions:
1. In determining the replacement level for a closer, is it appropriate to just compare the closer to the eighth inning guy? Would it not be something more like (closer minus eighth inning guy) plus (mopup guy minus 'true' replacement level guy)? It seems like just using the eighth inning guy fails to account at all for the new man in the pen.
2. Would positions players not operate the same way? Would you replace a starting outfielder with your AAAA outfielder or would you replace your starting outfielder with your fourth outfielder, and replace your fourth outfielder with the AAAA guy? Put another way, would you replace your number 1 outfielder with your number 5 outfielder OR replace your number 1 outfielder with your number 4, and replace number 4 with number 5? I probably could have worded that better, but the same principal to replacing best reliever with second best reliever would apply to replacing best outfielder with next best outfielder and so on down the depth chart.
If that is the case, then wouldn't the replacement level 'value' for an outfielder be the fourth outfielder, the replacement level 'value' for an infielder be the utility infielder, etc?
Perhaps this is accounted for in the calculation of replacement level. In which case, disregard my ignorance.
33 comments
|
0 recs |
Do you like this story?
Comments
Your questions deserve a better answer than this, and hopefully others will be more helpful
I think you are correct that replacement level for closers, and to a lesser extent setup men, is off. I think it is set the way it is because closers are (incorrectly) viewed as “special.” So they calculate closer replacement level in a special way. And their WAR (in some calculations) gets a special bump for leverage. Why pitchers get bumps for leverage and position players don’t is beyond me. I don’t think either should.
The immoderate moderator
because relief pitchers "earn" their leverage (usually)
starters and position players don’t
I'm not a sabermetrician, but I do play one at FanGraphs.
Can't get enough of me? Check out my Twitter feed.
by Matt Klaassen on Mar 16, 2010 2:25 PM EDT up reply actions
That still doesn't make sense to me
To some extent, position players do earn their leverage. If the bases are loaded in the 9th inning and a poor hitting position player comes up to bat, he’s going to be pinch hit for. But if it is a good hitter that comes up to bat, they are going to let him hit. And pinch hitters also earn the right to hit in key situations as well.
But, much more importantly, WAR should be about overall performance, not situational performance. Players do what they do. WAR shouldn’t account for when some players did what they did, but not others.
The immoderate moderator
by Scott McKinney on Mar 16, 2010 2:35 PM EDT up reply actions
WAR accounts for value
for relievers, their value as pitchers can be leveraged. By regressing leverage to 1, WAR (as implemented at FanGraphs) takes into account the chaining phenomenon and avoids overrating reliever. If you sort relievers by gmLI descending, it becomes quite apparent that relievers are mostly earning their relative leverage.
I'm not a sabermetrician, but I do play one at FanGraphs.
Can't get enough of me? Check out my Twitter feed.
by Matt Klaassen on Mar 16, 2010 2:42 PM EDT up reply actions
I see that it is being done, but I still don't see why
WAR accounts for value for relievers, their value as pitchers can be leveraged
The value of position players could be leveraged as well. But that is not done because most don’t believe in clutch hitting. But it appears that the prevailing sabermetric mindset is that clutch pitching is for real. In the very least, they believe that pitchers should get credit in WAR for clutch performance, while position players should not. That really doesn’t make sense to me at all.
The immoderate moderator
by Scott McKinney on Mar 16, 2010 2:45 PM EDT up reply actions
it's not because of a clutch "skill" relievers have at pitching better when the "chips are down"
it’s because they can be inserted into the game at “clutch” moments, and this is generally done relative to the reliever’s skill (Rivera and Broxton pitch when leverage is high, generally, and R.A. Dickey pitches when leverage is low).
It’s a relief to at least be arguing with someone who thinks that FGs overvalues relievers, since most think that they’re undervalued (these people tend to like WXRL).
Over the last three seasons, FanGraphs has only valued three reliever-seasons over 3.0 WAR, and all were around 3.0 WAR. Papelbon and Rivera in 2008, and Rafael Betancourt (obviously fueled by ’roids!) in 2007. No only was 3+ WAR in 2009 (Broxton was 2.9, and no one else was close).
I'm not a sabermetrician, but I do play one at FanGraphs.
Can't get enough of me? Check out my Twitter feed.
by Matt Klaassen on Mar 16, 2010 2:52 PM EDT up reply actions
It’s a relief to at least be arguing with someone who thinks that FGs overvalues relievers, since most think that they’re undervalued (these people tend to like WXRL).
Over the last three seasons, FanGraphs has only valued three reliever-seasons over 3.0 WAR
I think most people (even sabermetricians) look at those WAR numbers and think “we know those closers are very, very valuable, so the WAR formulation for relievers must be off.” When I see closer and reliever WAR numbers, I think "yes, those guys who pitch 40-80 innings per season really aren’t very valuable. That’s why I’d trade Soria immediately if not sooner.
I think the tweaks to WAR for relievers (and the further tweaks that some would like to see) are the cart pulling the horse. The numbers should help us find the truth. We shouldn’t assume we know the truth and then tweak the numbers until they tell us what we already have concluded.
The immoderate moderator
by Scott McKinney on Mar 16, 2010 3:03 PM EDT up reply actions
1. We know that teams (try to) put the best relievers in the highest leverage situations
2. We know that high leverage situations are worth more in win probability
Reliever WAR isn’t perfected yet, but I think it’s pretty good at addressing your concerns.
I'm not a sabermetrician, but I do play one at FanGraphs.
Can't get enough of me? Check out my Twitter feed.
by Matt Klaassen on Mar 16, 2010 3:13 PM EDT up reply actions
1. We know that teams (try to) put the best relievers in the highest leverage situations
Yes, but I’d like the total value stat to reflect only how well they actually perform. I don’t need or want part of that total value to include how good the manager or GM thinks the pitcher is.
2. We know that high leverage situations are worth more in win probability
That is definitely true. But high leverage hitting situations are also worth more in win probability. Should David Ortiz get a WAR bump for those seasons in which he hit exceptionally well in close/late situations? WAR could include that, but I really don’t think it should.
The immoderate moderator
by Scott McKinney on Mar 16, 2010 3:20 PM EDT up reply actions
The answer to both is similar — WAR is based on value on the open market, and relievers go onto the market as players who can be inserted into appropriate game situaitions, starting pitchers and nonpitchers can’t. We aren’t rewarding relievers for a mythical “ability” to perform in tthose situations (a la Ortiz’s “clutchness,”), but for their ability to pitch, and the fact that their “position” allows them to be inserted into those situations, which isn’t the case for others.
I'm not a sabermetrician, but I do play one at FanGraphs.
Can't get enough of me? Check out my Twitter feed.
by Matt Klaassen on Mar 16, 2010 3:26 PM EDT up reply actions
starters and non-pitchers can't be inserted for clutch situations
but they can be removed.
Zapp Brannigan/Dayton Moore quote of the day: "[my most important sabermetric stats are] runs scored and runs driven in"
by SagehenMacGyver47 on Mar 16, 2010 3:27 PM EDT up reply actions
so...
I'm not a sabermetrician, but I do play one at FanGraphs.
Can't get enough of me? Check out my Twitter feed.
by Matt Klaassen on Mar 16, 2010 3:36 PM EDT up reply actions
So, position players and SP’s both earn their presence in close/late/clutch/high leverage situations to some degree. They show they have earned it by staying in the game in those late situations. The poor players show they haven’t earned it by being pulled.
The immoderate moderator
by Scott McKinney on Mar 16, 2010 3:46 PM EDT up reply actions
even if that were true
that would be a relatively insignificant portion of their value relative to relievers, for whom it is crucial for when they are put into the game
I'm not a sabermetrician, but I do play one at FanGraphs.
Can't get enough of me? Check out my Twitter feed.
by Matt Klaassen on Mar 16, 2010 4:06 PM EDT up reply actions
Perhaps
But I really don’t know how crucial it is in measuring a reliever’s performance to measure how high the leverage was. Even if they earn the leverage, why should their total value include the playing situations they’ve earned? I guess it depends on whether you want the “total value” stat to be the total value of their performance or total value on the market. If what we’re looking for is total value on the market, then WAR is way off on relievers, especially closers. Closers are valued much more highly on the open market than their WAR gives them credit for. But if we are looking for market value in WAR, then somehow the dollar value in the market of players of a particular position (or usage). This is a long way of saying that WAR really doesn’t look like it is trying to represent market value of players. Either that, or it is doing a poor job of it.
The immoderate moderator
by Scott McKinney on Mar 16, 2010 4:17 PM EDT up reply actions
WAR is not completely the market value of players.
WAR is the amount of value that the player is likely to have (projection) and how valuable the player was. (counting stat)
Go Royals!
The open market this is tough
Closer are generally overvalued, but the chaining somewhat accounts for the value they do have with respect to other players without overdoing it. I do think teams like SEA, TBA, and others are catching on.
I'm not a sabermetrician, but I do play one at FanGraphs.
Can't get enough of me? Check out my Twitter feed.
by Matt Klaassen on Mar 16, 2010 4:24 PM EDT up reply actions
I just was noting that
starters can still have their PT managed relative to more important game states, not as flexibly as relievers, but still managed by removing them
Zapp Brannigan/Dayton Moore quote of the day: "[my most important sabermetric stats are] runs scored and runs driven in"
by SagehenMacGyver47 on Mar 16, 2010 6:38 PM EDT up reply actions
oh, and batting order gives certain players more ABs
though whether that has implications on leverage I don’t know…but the most likely batter to bat in the 9th inning is #1, then #2, etc.
Zapp Brannigan/Dayton Moore quote of the day: "[my most important sabermetric stats are] runs scored and runs driven in"
by SagehenMacGyver47 on Mar 16, 2010 6:43 PM EDT up reply actions
Are all relievers given the same numerical leverage weighting?
Let's just trust the process.
by trusttheprocess on Mar 16, 2010 3:33 PM EDT up reply actions
No
Using their seasonal gmLI, their unleveraged value is multiplied by (gmLI+1)/2
(FanGraphs lead nerd David Appelman should come by and correct me if I’m wrong)
I'm not a sabermetrician, but I do play one at FanGraphs.
Can't get enough of me? Check out my Twitter feed.
by Matt Klaassen on Mar 16, 2010 3:35 PM EDT up reply actions
Thanks for the response.
i’m not necessarily saying it’s off, I’m just trying to understand how it’s configured, and why it’s not configured similarly for position players.
Let's just trust the process.
by trusttheprocess on Mar 16, 2010 3:32 PM EDT up reply actions
no problem
it’s a bit confusing
I'm not a sabermetrician, but I do play one at FanGraphs.
Can't get enough of me? Check out my Twitter feed.
by Matt Klaassen on Mar 16, 2010 3:35 PM EDT up reply actions
To say what Matt is saying another way NYRoyal...
You like linear stats so you can compare players, but WAR is not a linear stat, it is a value stat. If you want to compare relivers to eachother, go to a linear stat like xFIP. WAR is a counting stat like RBI’s.
Go Royals!
Yes, it is a counting stat
And that is a key reason that RP’s, who pitch relatively few innings, have such a low WAR as compared to other pitchers or position players. But for some reason (not a good reason IMO), relievers get a special leverage bump that no other players get.
The immoderate moderator
by Scott McKinney on Mar 16, 2010 4:19 PM EDT up reply actions
I think what you are saying is that leverage is a myth?
High leverage situations have a much higher chance of determining if you win or lose a game, that is why teams with good bullpens outperform their pythag. It seems to be clearly demonstrated by pythag and win probability that relievers do have more to do with a team winning then the limited innings that they pitch, or in other words, the leverage of the situation.
Yes, this does make WAR less useful in determining how close “good” relievers verses “bad” relievers are in relation to each other (like linear stats do) but relievers just have this special value (leverage) inherent in their postion, and it effects a reliever’s market value. Just like positional ajustments effect the market value of postion players and why moving a below average fielding Butler to DH where he does not lose much value, (.3 War or so) so that Callaspo can play everyday instead of sitting on the bench (and losing 2 WAR in value because you have to play to have value with WAR), is a good move.
Go Royals!
High leverage situations have a much higher chance of determining if you win or lose a game
Yes, and if you want WAR to include how players perform in situations of various leverage, then it should be done for all players, not just relievers. Or performance-by-game-situation shouldn’t be included at all.
Yes, this does make WAR less useful in determining how close "good" relievers verses "bad" relievers are in relation to each other (like linear stats do) but relievers just have this special value (leverage) inherent in their postion, and it effects a reliever’s market value
Now, we’re back to WAR being a market value stat. It doesn’t make sense to include some elements of market value to make it a “market value” stat, but not include others that would make it a true measure of market value. Why should this element of market value be included for relievers, but not for other players at other positions? Relievers are so unique that this should be included for them and completely ignored for all other players? That just doesn’t make sense.
The immoderate moderator
by Scott McKinney on Mar 16, 2010 4:58 PM EDT up reply actions
Its a special situation of relievers.
For the batters, there are nine of them against one pitcher, so the leverage situations for the batters are so spread out that it would make very little difference to their WAR values, while relivers face on average about 5 batters an inning, while the batters have one at bat every other inning. So the reliever has about 10 times more leverage added to his value then any hitter, thus a postion players WAR is not influced much by leverage, In addition a lot of the postion players value is determined by defense (1/2 the time his team is not at bat while in high leverage situations).
So leverage its actually like 20 times greater for the reliever. Late inning close game relievers tend to have a high multiplier to leverage situations put on them versus a postion player, and that is why you have to count leverage as value for them and don’t have to for a postion players.
Clutch perfomances normally washes out with postion players (Pat Tabler aside), but for relievers leverage is where most of their value comes from versus a mop up reliever, and it doesn’t matter that two given relivers are close in xFIP, the slightly better reliver who succeeds in high leverage situations is vastly more valuable then the mop up guy who never gets to pitch in those situations. WAR is a counting stat. If you don’t pitch in high leverage situations, you don’t get half the value for them by setting in the bullpen watching the game.
Go Royals!
Clutch perfomances normally washes out with postion players (Pat Tabler aside), but for relievers leverage is where most of their value comes from versus a mop up reliever, and it doesn’t matter that two given relivers are close in xFIP, the slightly better reliver who succeeds in high leverage situations is vastly more valuable then the mop up guy who never gets to pitch in those situations.
So in situations where the manager uses an inferior pitcher as closer (like Borowski over Betancourt, Gregg over Downs and Frasor, and many other examples), the closer gets a nice bump to his WAR for….no reason. Because he performed particularly well in those high leverage situations? No, just because he happened to be in the game at those times. Consider this a clear weakness of WAR for relievers, and especially closers.
WAR is a counting stat. If you don’t pitch in high leverage situations, you don’t get half the value for them by setting in the bullpen watching the game.
Yes, it is a counting stat. And with regard to relievers, IMO, it is counting one thing that it shouldn’t count. Merely being the guy who gets the ball at a particular time doesn’t make the total value of your performance higher.
The immoderate moderator
by Scott McKinney on Mar 16, 2010 5:59 PM EDT up reply actions
Not sure if these are complete answers to the right questions...
But..
1. At Fangraphs, we account for the way the ‘pen works (e.g., setup man replaces hurt closer) by “bullpen chaining” — leverage (I think it’s gmLI now) is brought into war by doing RAR/WAR (unleveraged) NOT times gmLI, but times leveraged “regressed” to 1, or RAR times (gmLI +1)/2. So, let’s say a reliever as a 1.8 gmLI for the season, his RAR will be multiplied by 1.4 to account for bullpen chaining.
2. Other positions don’t operate that way because starting pitchers and nonpitchers aren’t placed into the game (other than pinch hitters — a issue that is complicated and not worth pursuing here, as pinch hitters are so ineffecitve relative to hitters in general that it wouldn’t make a big different) based on leverage, they start the game. Relievers are (typically, if your manager is average) based on effectiveness. No, managers don’t use their bullpens perfectly optimally, but generally, the best relievers get the highest leverage situations, and so on (note: this apparently does not apply to Jamie Wright).
I'm not a sabermetrician, but I do play one at FanGraphs.
Can't get enough of me? Check out my Twitter feed.
but isn't the same at least partly true for fielders,
in the sense that replacement fielder doesn’t replace the actual starter, he replaces the backup (gets chained behind the backup)? And both the starter’s and the backup’s playing time is determined at least to some extent by game situation relative to skill, be it a pinch hitter or a defensive sub. Maybe this impact is small enough that it is not worth counting.
Zapp Brannigan/Dayton Moore quote of the day: "[my most important sabermetric stats are] runs scored and runs driven in"
by SagehenMacGyver47 on Mar 16, 2010 3:01 PM EDT up reply actions
" Maybe this impact is small enough that it is not worth counting."
yup.
I'm not a sabermetrician, but I do play one at FanGraphs.
Can't get enough of me? Check out my Twitter feed.
by Matt Klaassen on Mar 16, 2010 3:12 PM EDT up reply actions
Thanks for the answers, here are some more questions.
If a renegade manager were to operate their bullpen without any adherance to a chaining strategy – say they just rotated relief appearances in order or drew names out of a hat or some other means of just being completely random – and the relievers didn’t have ‘roles,’ would the fangraphs calculation of reliever WAR be rendered inaccurate? How does fangraphs account for swingmen that spend some time in the bullpen and some time in the rotation? Are only the relief innings given that leverage boost?
With regards to the possiblity of ‘chaining’ of players down the depth chart – are you saying that it doesn’t happen that way or that it’s just not measurable?
Let's just trust the process.
by trusttheprocess on Mar 16, 2010 3:31 PM EDT up reply actions
Comments For This Post Are Closed
Secondary Sidebar
User Tools
FanPosts
Community blog posts and discussion.
Certain photos copyright © 2012 by Associated Press or Getty Images. Any commercial use or distribution without the express written consent of Associated Press and Getty Images is strictly prohibited.
