The Red Sox season is a case of Dr. Jekyll and Mr Hyde

In the first 36 games of the season the Red Sox went 17-19 while scoring 4.22 and allowing 4.47 runs per game. In the following 35 games the Red Sox went 26-9 while scoring 6.54 and allowing 3.91 runs per game. A 50% increase in runs scored over the same time period is usual.

Even more strange is if you look at the first 35 games the Red Sox never scored more than nine runs in a game. In the following 36 games they have scored more than nine runs 8 times. If I plot the run total frequency of the first 35 games of the season versus the following 36 games you get two totally different distributions.

The Standard Deviation for the first 35 games was a relatively tight 2.72 while for the following 36 games the Standard Deviation has ballooned 4.31. I have never seen a so flat a distribution of runs as the Red Sox have had in the last 36 games. Most of the runs scored distributions I have seen look like Chi Squared distributions

Cleveland Indians are better than Sabermetricians Predicted

When I was at the Sabermetric Seminar in Boston. The Indians success in the first quarter of the year was a topic of discussion. The explanation given by Tom Tippett was that the Indians where over performing against the model and would over the course of the season return to their expectation. In support of that an expected run chart was put up showing the Indians with the greatest positive actual run differential versus expected run differential. The Red Sox were underperforming in respect to this measure.

While I understand there will always be statistical anomolies and periodic straying from the mean, I am not so sure that this is the case here. Modelers have a tendency to explain away differences from reality compared to there models as variation. While that may and will be the case sometimes for a three standard deviation outlier we are talking about a 3 in 1,000 chance. Rather than take that bet I would check to see if my model failed to take something into account. In the case of the Indians improvement, I would be more likely to look for shortcomings in my model because the Indians are a Sabermetric driven team and the guy who runs their analytics is a very talented guy. Teams do not share their models so there is no way of know if the various model are similar or even what input Data they use. A general impression from the Sabermetric conference is that Sabermaatricians do a lot of regression to the league mean which will smooth out the data, but may also underemphasize relevant data.

I believe even a quick look at even high level data for the Indians suggests their performance is not a wandering away from the mean but a shift in the mean.  Most of the difference in 2011 can be  attributed to the 233 runs scored in 46 games or 5 runs per game compared to 4 run per game in 2010. This can be explained because most sabermetric models fail to incorporate injuries into their models which was a factor in 2010 for the Indians and would negatively effect their run prediction in 2011. A lack of injury prediction and weighting due to past injuries in Sabermetric models is a major disconnect in Sabermetrics and needs to be addressed. The Healthcare industry has made great strides in this area in recent history with the use on ensemble methods.

Gelman writes about Bill James

I always knew that Professor Andrew Gelman of Columbia was a well known Statistician and Social Scientist, but when he writes an article in the Baseball Prospectus now he is famous. It is always good to see a statistician write about a sabermetrician. Although these two fields are really the same it seems they try to separate themselves from each other.

It was interesting to me that Gelmen wrote about James as a baseball outsider not too different from what James was to baseball in 1984 when he wrote the "Inside-out Perspective" article. Baseball may always need the outsider prospective to push it along because its traditions and beliefs are so deep.

I thought one interesting issue that Gelmen touched upon was how little of the real work in sabermetrics gets published. When Gelmen works on a topic he publishes a paper that discusses his approach, provides an example and the code to run the example yourself. Not so in Sabermetrics. I find little detail in the published articles and very little code. This results in people like James moving away from positions and theories without explanation. I think this hurts the development of Sabermetrics in some ways. My view on how science is developed is the path of how gravity was discovered through a series of theories that we accepted and then rejected. First there was nature abores a vacuum, then there was nature abores a vacuum up to 32 feet and then finally there was gravity at 32ft/sec.

It was a fun article to read in preparation of my attendance at the Sabermetrics Seminar at Harvard May 21-22

1918 Cubs have the real Curse from 1918

So I guess it is not even a new story that the 1918 Cubs may have thrown the World Series. I had wondered in a previous article if predictive analytics would have spotted a fixed game as an outlier and therefore possibly fixed. The great thing about baseball is there is so much data and people have studied it all. The answer seems to be that there is nothing in the overall performance or individual numbers of the cubs players that indicate a problem. I guess there were a couple of plays that were questionable, but most believe not enough to prove a fixed Series. One of the players is pitcher Phil Douglas who is later banned from baseball for life in 1922 for seeking to get paid to lose the pennant. Also I love that Douglas was one of 17 pitchers allowed to continue to throw the spitball after it was banned in 1920. The other player is Max Flack who is better known for being traded to the other team after the first game of a double header.

I took a brief look at the box scores yesterday, and I agree there is nothing that stands out. It looks like a good low scoring series. I do not believe these Cubs would have gotten caught by the numbers even now if they did in fact throw the series. The funny thing is that this is the last World Series that the Red Sox won before "the Curse" that resulted from trading Babe Ruth to the Yankees.  Given that the Cubs still have not won a World Series since they lost the 1918 World Series maybe there is Karma in baseball.

Home Field Advantage

With the Red Sox having their only wins so far this year at home along with their reputation of being tooled to perform better at Fenway, I decided to take a quick look at the American League teams Home versus Away wins in the Red Sox Bill James era (2003-2010). Listed below are the differences between home wins and away wins for each team per year in the time period along with their corresponding mean and Standard Deviation.

2010 2009 2008 2007 2006 2005 2004 2003 Mean STD Tampa Bay 2 20 17 8 21 13 12 19 14.00 6.59 Boston 3 17 17 6 10 13 12 11 11.13 4.88 NYYanks 11 11 17 10 3 11 13 -1 9.38 5.71 Toronto 5 13 18 15 13 6 13 -4 9.88 7.10 Baltiore 8 14 7 1 10 -2 -2 9 5.63 5.93 Chi White Sox 2 7 19 4 8 -5 19 16 8.75 8.65 Minnesota 12 11 18 3 12 7 6 6 9.38 4.78 Cleveland 7 5 19 6 10 -7 8 8 7.00 7.13 Kansas City 9 16 1 1 6 7 8 -3 5.63 5.90 Detroit 23 16 6 2 -3 12 4 3 7.88 8.51 LA Angels 6 1 0 14 1 3 -2 13 4.50 6.02 Texans 12 9 1 19 -2 7 13 15 9.25 7.07 Oakland 6 11 11 4 5 -3 13 18 8.13 6.47 Seattle 9 5 9 10 10 9 13 7 9.00 2.33

It is no surprise that the Red Sox have a strong Home/Away Wins record (2nd),

but I was surprised that the Tampa Bay Devil Rays were even better in this

statistic. Also note how dominate the AL East is in Home Wins versus Away Wins.

Now for the Red Sox in 2011 having all your wins at home is still bad.

Why are the Red Sox better today? Sabremetrics or Construction?

I saw an article this morning from an MIT professor that predicted the Red Sox would win 100 games this year. That is a pretty bold statement since the Red Sox have only won 100 games in a season three times (1912, 1915 and 1946). However, it got me to wondering how have the Red Sox become so good in recent history. I often heard comments the claim that it is the payroll or the genius of Theo Epstein. Whenever I am with statistics guys, it is the hiring of Bill James and the use of Sabremetrics that made the difference. I have a third theory to put forth as the major reason for the improvement of the Red Sox in recent history, construction at Fenway. Oddly, this started the same year that Bill James was hired by the Red Sox, 2003.

From 1995 to 2002 the Red Sox had a combined record of 695-582 winning 54.42% of their games. From 2003 to 2010 the Red Sox had a combined record of 749-547 winning 57.79% of their games.

Year	W	L	Winning %		Year	W	L	Winning %
2010	89	73	54.94%		2002	93	69	57.41%
2009	95	67	58.64%		2001	82	79	50.93%
2008	95	67	58.64%		2000	85	77	52.47%
2007	96	66	59.26%		1999	94	68	58.02%
2006	86	76	53.09%		1998	92	70	56.79%
2005	95	67	58.64%		1997	78	84	48.15%
2004	98	64	60.49%		1996	85	77	52.47%
2003	95	67	58.64%		1995	86	58	59.72%
	749	547	57.79%			695	582	54.42%

So they are a got better after 2003 and Theo is a genius and Sabremetrics rules baseball. I am not so sure, and I think we reach those numbers based on a Simpson's paradox. Let me explain. If Sabremetrics had been the driving reason for the improvement the Red Sox. they would have gotten better not only at home but away as well. They did not. In fact the Red Sox improved massively at home, but got worse on the road. So what is the factor that explains this? In 2003, the same year Bill James was hired by the Red Sox, additional seating was added the Fenway park for the first time since it was 1946. While it was was always known that Fenway was helpful to certain types of hitters and pitchers and the Red Sox teams have always emphasized those players. I believe that construction made the park even more baised than it was before.



During the period 1995 to 2002 the Red Sox had a better away record than they did from 2003-2010.

Away Record W L % W L % 2010 40 41 49.38% 2002 51 30 62.96% 2009 39 42 48.15% 2001 41 39 51.25% 2008 39 42 48.15% 2000 43 38 53.09% 2007 40 41 49.38% 1999 45 36 55.56% 2006 35 46 43.21% 1998 41 40 50.62% 2005 41 40 50.62% 1997 39 42 48.15% 2004 43 38 53.09% 1996 38 43 46.91% 2003 42 39 51.85% 1995 43 28 60.56% total 319 329 49.23% 341 296 53.53%

For Home games it is a very Different story: Home Record W L % W L % 2010 49 32 60.49% 2002 42 39 51.85% 2009 56 25 69.14% 2001 41 40 50.62% 2008 56 25 69.14% 2000 42 39 51.85% 2007 56 25 69.14% 1999 49 32 60.49% 2006 51 30 62.96% 1998 51 30 62.96% 2005 54 27 66.67% 1997 39 42 48.15% 2004 55 26 67.90% 1996 47 34 58.02% 2003 53 28 65.43% 1995 43 30 58.90% total 430 218 66.36% 354 286 55.31%

MIT economist says Red Sox will win 100 games in 2011