When is the sample size significant enough to judge?

As everyone who follows statistics in any field knows, you can never judge statistics too soon for fear of being wrong in assumptions. While some trends are easy to see from past statistics in combination with current statistics, when should we take this year for what it is? More specifically, when should this year's stats start to count in what we expect? In fantasy baseball, as far from reality as it is, I always wait until 40-45 games into a season to give up on someone to search for another player. In real baseball, it depends on at bats, innings, and other things compared with previous stats. That said, we are all stat geeks. Who among us can give me the at bats and innings where statistics become significant? It is a difficult task, and the question begs an obvious answer that we can use common sense to make sense of the stats so far. But is Billy Butler a singles-hitting guy w/ his .360ish slg but obvious ability to hit for average and RBI? Is Callaspo a contact-hitting Robinson Cano in disguise? Can German still produce? Should we even worry about Guillen's slow start? Are Greinke's and Bannister's starts significant or simple Runelvys Hernandez-ish (yeah, had to bring this up) hot streaks to start a year? All of these questions can be answered at the same time as the sample size question. All theories and answers are relevant here. Discuss. Gracias.
0 recs |
6
comments
Comments
sidenote to butler question
how many doubles has he attempted for that he did not get? dude gets out on the bases a lot… might greatly affect slg pct and some kind of real slg pct in how he actually hits it and brings guys in.
Never giving up on your team is what makes you a good fan.
by kcisbetterthanstlateverything on Apr 30, 2008 3:54 AM EDT 0 recs
That's a very difficult question
There is no simple answer and there is no magic number. You can’t say that after X at bats, we know how good of a season a guy is going to have or what kind of player he is. 200 AB’s aren’t meaningless, but it is also just 1/3 of a season for a regular. A player can easily have a bad or good first third of the season (or first quarter or first half) and then be just the opposite for the rest of the season.
In my opinion, instead of looking for a bright line, I think we should take various sample sizes for what they’re worth. 50 AB’s or 100 or 200 or 400 are what they are. They are parts of a season that become more meaningful the larger they are. For established players, you can view current small sample size stats through the prism of what they’ve done before. For players with less of a major league track record, you just have to be patient and resist the impulse to pass judgment or label a player early.
I probably disagree with you.
by NYRoyal on Apr 30, 2008 5:37 AM EDT 0 recs
I have heard Trey say
That 300 ABs is a meaningful sample size for him. Of course, he also bases some of his decisions on matchups that include just 20-30 ABs, so he doesn’t seem to stick to that.
Relive Royals History at royalsretro.blogspot.com
by RoyalsRetro on Apr 30, 2008 9:37 AM EDT 0 recs
It depends on what you're talking about
for example, with Guillen, I don’t think it’s that significant yet. Without looking back at the numbers, I think of him as a guy who, to put it mildy, is a contact hitter. Not a lot of walks - so his OPS is going to very BA/SLUG dependant. Those kinds of guys are going to be streakier, since BA is so variable and dependant on BABIP and stuff. So, although I’m not a big Jose Guillen fan by any means (and think the Royals paid way to much for him), I’d say he’ll almost certainly get better - he’s a streaky hitter by nature. And, of course, having the slump at the beginning of the season makes it more noticable. In the big scheme of things, if he had this April in August, but had hit .280/.350/.500 in April, he could end up with the same numbers for the season, but most people would look at the counterfactual season and see it as much better. But now I’m just rambling and avoiding work.
In other words, the sample sizes so far are generally (how’s that for a qualification) not significant with reference to things that are reliably measured by stats, particularly if they deviate from prior performance. But that’s just my take on it as a layperson.
by devil_fingers on Apr 30, 2008 10:16 AM EDT 0 recs
math
from a more mathematical standpoint, the answer to your question depends on the level of confidence you want to have. 1 AB is enough to be statistically significant if your desired level of confidence is low enough.
Statisticians will tell you, as a rule of thumb, that 30 observations are when inference can start to be done as non-biased samples begin to show normal distributions. Obviously, the more the better though.
Personally, 100 AB or 75 in. pitched would work for me as workable numbers, but what do I know, I’m only an economist.
by ZeppelinDZ on Apr 30, 2008 11:06 AM EDT 0 recs
The Hardball Times
had an article on this within the last two weeks or so. The short answer was different stats stabilize with different sample sizes. I don’t have time to look for it right now, but maybe someone else can find it.
by Gopherballs on Apr 30, 2008 5:42 PM EDT 0 recs








