PECOTA Baseball Projections Question
-
ZigZagCardsFan
- Perennial All-Star
- Posts: 5526
- Joined: July 29 06, 6:26 pm
PECOTA Baseball Projections Question
Is PECOTA widely accepted as the most accurate projection system? If not, is there another projection system that is more accurate?
-
greenback44
- Hall Of Famer
- Posts: 11664
- Joined: June 26 06, 8:54 pm
- Location: In a Small Town with Jack and Diane
Re: PECOTA Baseball Projections Question
There really isn't a best projection system. The decent ones are all close enough that it's not worth worrying about which one is best unless you're trying to market one or the other.ZigZagCardsFan wrote:Is PECOTA widely accepted as the most accurate projection system? If not, is there another projection system that is more accurate?
-
jim
- Red Lobster for the seafood lover in you
- Posts: 50393
- Joined: May 1 06, 2:41 pm
I do remember reading some studies on this, and PECOTA probably does the best job of any available system. PECOTA is very complex, taking just about everything you can think into account, and apparently that complexity adds to it's reliability.
How much more reliable? Not much, not much at all. The simplest of all projection systems (Marcel I think it's called), does nothing other than regress to the mean and comes very close to PECOTA. So far all of the fancy things PECOTA does, it only improves forecasting slightly apparently.
How much more reliable? Not much, not much at all. The simplest of all projection systems (Marcel I think it's called), does nothing other than regress to the mean and comes very close to PECOTA. So far all of the fancy things PECOTA does, it only improves forecasting slightly apparently.
-
Arthur Dent
- Hall Of Famer
- Posts: 12536
- Joined: April 25 06, 6:43 pm
- Location: Austin
-
Jocephus
- 99% conan clips
- Posts: 64992
- Joined: April 18 06, 5:14 pm
ok...lets see...its um...its...ok...you need...you need to think of it...of it like as a sort of paradox...to...another dimension where baseball is so a part of everyday life you can't escape it...but then you go to this other place where baseball exists but no one talks about it or discusses it or enjoys it...then the black hole that you are in traveling around space in returns you planet earth...and then you watch some baseball and say "wow...i've really regressed to the mean in galactical baseball mete-schemas"...Arthur Dent wrote:I've never been clear on what "regression to the mean" is. Can someone explain it?jim wrote:The simplest of all projection systems (Marcel I think it's called), does nothing other than regress to the mean and comes very close to PECOTA.
-
haltz
- Hall Of Famer
- Posts: 22626
- Joined: November 9 06, 6:45 am
- Location: a proud midwestern metropolis
If the "mean" is the person's/league's true talent, or score, or batting average or whatever, then the larger the sample the better you can gauge the person's true talent. The smaller the sample the more likely that luck could be playing a role in the outcome. So, it's a statistical phenomenon where extreme scores and outliers (or all scores) are regressed back to the mean, since luck doesn't hold true from test to test (or season to season). That's as best as I understand it, and I wouldn't be able to explain the math without *studying*, or perhaps at all for that matter (but for some reason I have a feeling that you would).Arthur Dent wrote:I've never been clear on what "regression to the mean" is. Can someone explain it?jim wrote:The simplest of all projection systems (Marcel I think it's called), does nothing other than regress to the mean and comes very close to PECOTA.
-
jim
- Red Lobster for the seafood lover in you
- Posts: 50393
- Joined: May 1 06, 2:41 pm
Tango did a good job in his book explaining this.
Think of a coin. You know all other coins come up heads 50% of the time, but you flip this one 3 times and it's tails. Based on what you know about coins, if you use that to regress this coin to the mean it will still be at 50%. In other words, the 3 flips don't provide enough information yet. But if we flip it 1,000 times, and only 25 times it comes up heads, now we are talking about possibly an unfair coin. Even though we know what coins normally do, here we have enough sample that the coin is unfair.
Replace "coin" with "baseball player" and "50%" with the corresponding stat (batting average, ERA, etc...) and you have the basic model.
Here is what Tango said:
"When estimating a player's true talent level or projecting future performance, one needs to adjust his past performance toward the mean performance of similiar (not all) players. The simplest approach is to add a specific number of average performances to his performance record."
To estimate wOBA for the next season of a player, you need to add approximately 220 average AB's to his totals.
Think of a coin. You know all other coins come up heads 50% of the time, but you flip this one 3 times and it's tails. Based on what you know about coins, if you use that to regress this coin to the mean it will still be at 50%. In other words, the 3 flips don't provide enough information yet. But if we flip it 1,000 times, and only 25 times it comes up heads, now we are talking about possibly an unfair coin. Even though we know what coins normally do, here we have enough sample that the coin is unfair.
Replace "coin" with "baseball player" and "50%" with the corresponding stat (batting average, ERA, etc...) and you have the basic model.
Here is what Tango said:
"When estimating a player's true talent level or projecting future performance, one needs to adjust his past performance toward the mean performance of similiar (not all) players. The simplest approach is to add a specific number of average performances to his performance record."
To estimate wOBA for the next season of a player, you need to add approximately 220 average AB's to his totals.
-
Phyrkrakr
- All-Star
- Posts: 1514
- Joined: January 15 07, 2:51 pm
- Location: St. Louis
I think my brain just exploded...Jocephus wrote:ok...lets see...its um...its...ok...you need...you need to think of it...of it like as a sort of paradox...to...another dimension where baseball is so a part of everyday life you can't escape it...but then you go to this other place where baseball exists but no one talks about it or discusses it or enjoys it...then the black hole that you are in traveling around space in returns you planet earth...and then you watch some baseball and say "wow...i've really regressed to the mean in galactical baseball mete-schemas"...Arthur Dent wrote:I've never been clear on what "regression to the mean" is. Can someone explain it?jim wrote:The simplest of all projection systems (Marcel I think it's called), does nothing other than regress to the mean and comes very close to PECOTA.
-
skmsw
- Perennial All-Star
- Posts: 6340
- Joined: April 18 06, 7:12 pm
- Location: The Hub
Several good explanations on this thread already so I'll just add a simple, one-sentence version:Arthur Dent wrote:I've never been clear on what "regression to the mean" is. Can someone explain it?jim wrote:The simplest of all projection systems (Marcel I think it's called), does nothing other than regress to the mean and comes very close to PECOTA.
Regression towards the mean is the notion that for any observation that is significantly higher or lower than usual, the most likely future observation will be something closer to normal.
-
tangotiger
- Fringe Major League Player
- Posts: 101
- Joined: April 20 06, 8:33 am
- Contact:
I like skmsw's one-sentence version.
Just remember that all data you see is a SAMPLE. You are trying to infer the TRUE rate, based on this sample.
Imagine starting with true known rates. You have a die, and you "win" when you roll a 1,2,3,4. You roll that die 12 times, and you end up, on average, with 8 wins. But, sometimes it'll be 4 or 11 or whatever.
Imagine you have another die, that you win when you roll a 1,2. Roll that die 12 times. On average, you'll get 4 wins, but sometimes you end up with 8 or 11.
Imagine you have 100 such die. 30 rolls a 1,2,3 for a win. 20 rolls a 1,2. 20 rolls a 1,2,3,4 for a win. 15 rolls a 1, and 15 rolls a 1,2,3,4,5 for a win.
Now, you have no idea which die is which. All you see is results. And you see one die that won all 12 games. Now, we know none of them will always win 12 out of 12, because I told you the distibution was between 1/6 to 5/6. But, each die has a certain chance of rolling 12 for 12.
You then figure out the chance that the 1/6 can roll 12 for 12, the 2/6 can roll 12 for 12, etc.
Your best guess of the die that rolled a 12 for 12 is that it came from that distribution, but more likely from the higher end. You are regressing the result (12 for 12, or 100%) toward the mean (50%) a certain amount (based on the sample number of throws).
If you instead happened to roll 24 for 24, now your regression is much smaller, since it's extremely unlikely that a die that wins 1/6 could roll 24 for 24.
Just remember that all data you see is a SAMPLE. You are trying to infer the TRUE rate, based on this sample.
Imagine starting with true known rates. You have a die, and you "win" when you roll a 1,2,3,4. You roll that die 12 times, and you end up, on average, with 8 wins. But, sometimes it'll be 4 or 11 or whatever.
Imagine you have another die, that you win when you roll a 1,2. Roll that die 12 times. On average, you'll get 4 wins, but sometimes you end up with 8 or 11.
Imagine you have 100 such die. 30 rolls a 1,2,3 for a win. 20 rolls a 1,2. 20 rolls a 1,2,3,4 for a win. 15 rolls a 1, and 15 rolls a 1,2,3,4,5 for a win.
Now, you have no idea which die is which. All you see is results. And you see one die that won all 12 games. Now, we know none of them will always win 12 out of 12, because I told you the distibution was between 1/6 to 5/6. But, each die has a certain chance of rolling 12 for 12.
You then figure out the chance that the 1/6 can roll 12 for 12, the 2/6 can roll 12 for 12, etc.
Your best guess of the die that rolled a 12 for 12 is that it came from that distribution, but more likely from the higher end. You are regressing the result (12 for 12, or 100%) toward the mean (50%) a certain amount (based on the sample number of throws).
If you instead happened to roll 24 for 24, now your regression is much smaller, since it's extremely unlikely that a die that wins 1/6 could roll 24 for 24.