As the Royals have begun what I'm sure will be a historic Cactus League season, I thought this would be a good time to revisit the issue of spring training stats.  A year ago, I wrote the original STSAM article which set off a firestorm of controversy (otherwise known as general agreement, with a few dissents).  So I thought I'd reiterate the salient points, and hopefully expand and explain them a bit more.  Then I'll give some relevant examples from 2008 which I believe illustrate my point.

Essentially the entire field of baseball research (and even a good deal of baseball traditionalists) recognize that spring training stats are meaningless.  Good or bad spring training stats tell you basically nothing about how good of a season a particular player or team is going to have.  The two sets of stats simply do not correlate and a variety of studies have proven this.

There is one small exception to this rule.  As I pointed out late last March, John Dewan of Baseball Info Solutions, most known for his Plus/Minus fielding metric, found that players who have a slugging percentage in spring training 100 points or more higher than their career average are more likely to "performed better than their career average during the upcoming season."  Interestingly, sometimes Dewan refers to a 100-point SLG increase threshold and sometimes he refers to a 200-point threshold.  His methodology has been questioned somewhat, but I think there is something to it.  However, we need to recognize how little Dewan is saying here.  A big SLG increase in spring training leads only to some increase in regular season performance over career averages.  He's not talking about an impending breakout season; just something better than the player's career averages.

But there is a broad consensus that save this one small exception, spring training stats have no meaning.  A few weeks after my original article, Joe Sheehan at Baseball Prospectus echoed my sentiments (I'm sure my writing inspired him).  I think this quote pretty much sums it up:

I’ve been making the point about the meaninglessness of spring training stats all month, and I’m making it again, because I continue to see these stats quoted in stories about baseball as if they have meaning. They don’t. Spring training stats...are meaningless.

So, if these stats are largely meaningless, why is that?

1. Sample size.  Every player's official spring training stats represent a very small sample size.  For starting pitchers, this means 15-25 IP.  For relievers, 5-15 IP (or less).  For position players, this means 40-70 AB (regulars) and even less for backups.  Actual spring training "A games" are only a small part of the work players do in spring training.  There are also "B games," minor league games, intrasquad games, simulated games and of course all of the practices and workouts.
2. Uneven competition.  There is a wide variety of talent in spring training.  A player will end up facing some complete scrubs, some ok minor leaguers, some decent major leaguers and some great major leaguers.  When looking at a player's spring training stats, you never know if he faced a disproportionate number of good or bad players.  The level of competition changes over the course of a game and throughout spring training.  There are more bad players early in spring training and they are weeded out as spring training progresses.  Also, early in spring training, the pitchers are usually said to be ahead of the hitters.  So spring training stats also can reflect when a player got his playing time and thus, the quality of players he played against.
3. Unreal play.  Not every player in spring training is always playing as if it were a real game.  And I'm not just talking about the veterans who cruise through spring training and don't give 100% because they know they have a starting spot locked up.  There are also many games throughout spring training when pitchers are working on particular pitches, so they throw almost all fastballs, or don't throw one of their breaking pitches to decrease strain on their arms.  Position players also might be working on contact in some games, or bunting or some other particular skill.  All of these things are done to work on one element of a player's game without regard to actual game outcome or the player's stats.
4. Rarefied air.  For a variety of meteorological reasons, the ball carries very well in Arizona.  That can pump up power numbers and hurt pitching stats.

Last year I provided several examples of the disparity between spring training and regular season stats in 2006 and 2007.  There were many such examples in 2008 as well.

Luke Hochevar

Spring Training

ERA 2.25

K/9 3.38

BB/9 1.13

Regular Season

ERA 5.51

K/9 5.02

BB/9 3.28

Gil Meche

Spring Training

ERA 2.70

K/9 8.10

BB/9 1.80

Regular Season

ERA 3.98

K/9 7.83

BB/9 3.12

Zack Greinke

Spring Training

ERA 6.43

K/9 5.79

BB/9 3.21

Regular Season

ERA 3.47

K/9 8.14

BB/9 2.49

Joakim Soria

Spring Training

ERA 5.40

K/9 9.72

BB/9 1.08

Regular Season

ERA 1.60

K/9 8.82

BB/9 2.54

Billy Butler

Spring Training

AVG .341

SLG .610

Regular Season

AVG .275

SLG .400

Justin Huber

Spring Training

AVG .316

SLG .632

Regular Season

AVG .246

SLG .393

Joey Gathright

Spring Training

AVG .316

SLG .395

Regular Season (San Diego)

AVG .254

SLG .272

Damon Hollins

Spring Training

AVG .314

SLG .571

Regular Season (Omaha (AAA))

AVG .220

SLG .408

Mark Teahen

Spring Training

AVG .295

SLG .545

Regular Season

AVG .255

SLG .402

Ross Gload

Spring Training

AVG .293

SLG .488

Regular Season

AVG .273

SLG .348

The difference between the spring training and regular season stats is both significant and all over the place.  Some do much better in spring training than in the regular season and some do much worse.  And there were instances in which the two sets of stats were similar.  And this is the whole point with spring training stats.  Sometimes they will be indicative of what the player does in the regular season.  But often they will differ greatly.  You just never know.

This isn't to say that spring training performance is necessarily irrelevant.  Players' skills and tools can change from year to year, particularly with the improvement of younger players and the deterioration of older players.  Managers, coaches and front office personnel spend several weeks with these players, monitoring and evaluating them.  They will see (or at least attempt to determine) if a pitcher has added velocity, added or improved a pitch or changed his mechanics to improve his control.  They will see if a hitter's pitch recognition has improved, if he's hitting the ball with more power or if an older player has lost a step.  But these evaluations cannot made in a meaningful and reliable way by use of spring training statistics for the above reasons.  Close observation and tools-based evaluation on a daily basis is necessary to see if a player's tools have improved or declined.  Certainly those observations can have important consequences to a variety of player personnel decisions.

So, while it is nice to cite some numbers when talking about who should be on the 25-man roster or who should have a good or bad season, there's really no reason to use spring training stats.  Major league stats from recent years provide much better evidence about how good or bad a player is and what should be expected from him in the near future.  Advanced projection systems help too.