Sabermeterics to predict pitching injury

I just finished The Extra 2% by Jonah Keri on the raise of Sabermetrics at the Tampa Bay Rays, and the resulting World Series that it won them. It is a quick read and a great introduction to the business of Baseball and how MLB teams have incorporated Sabermetrics into their management on recent years.

I met Keri up at the Sabermetric seminar at Harvard which was a fundraiser for the Dana Faber Cancer Institute. I believe they are going to do it again next year, and I strongly encourage anyone interested in Sabermetrics to go. It was my first time talking to people about sports statistics instead of running models and playing with data. Again if it happens again next year go!

In the last year at Bigcomputing we have done a great deal of work on predicting people's health for hospitals and healthcare companies with great success. The are a number of predictive analytics competitions that are trying to develop predictive models for things like predicting if a patient will be hospitalized within the week, month or year. The most well known of these competitions is the Heritage Health Care Prize for $3 Million dollars that is being hosted by Kaggle.com. Obviously with prizes of that size there is real potential to predict things like injury and disease.

Tom Tippett, head analyst of the Boston Red Sox, talked at the seminar about what things the Red Sox look at when they evaluate a player for contract. After this talk, I asked him if there was a predictive component for injury in their sabermetric models for predicting a players future performance. He said their was not. That surprised me, because I thought it could be done with the vast amount of data that is collected on the various players.

In Jonah Keri's book the Rays hired a guy who was able to predict injury of pitchers within a short time frame based on their Pitch F/X data. Josh Kalk published an article in Hardball Times called "The Injury Zone". He was later hired by the Rays where he has continued his work on Pitch F/X data among a mountain of other things. Injury prediction based on the physical and results data openly available in baseball is possible. Maybe even more so with the confidential information the teams have access to like Trackman data and medical and scouting reports. The key is being able to incorporate this into a predictive model with the idea of predicting injury. This type of risk analysis has a lot of value when you are talking about players who make an average of around $4M a year. I also love that Kalk's boss at the Rays was James Click. Click and Kalk should always work together.

Scientific American writes about Sabermetrics...sort of

In the June 5 issue of The Scientific American there is an article about baseball that looks at the chances of a batter being hit by a pitch. I am not sure that there is much significance to the finding of correlation between being hit by a pitch and temperature. I doubt there is even enough data in one season to make the kind of statements that the authors of this article make. However, if I was pitching and it was 95 degrees, I might bean the batter to get thrown out of the game and sent to the nice air conditioned locker room.

A couple of things jumped out at me in this data. First that less than .8% of at bats resulted in a hit batter. This number seemed much lower than I would have expected. The temperature choices also looked kind of arbitrary to me ( 95F and 55F). I mean starting with those temperatures couldn't you also draw a correlation that there are more hit batters in the middle of the season than in the beginning and the end. I also do not see a control for bias by ballpark or team which would have been interesting.

Just for fun I looked up the Don Baylor's and Craig Biggio's hit by pitch percentages which are 2.8% and 2.6% respectively. If Walter Johnson had been pitching to these two they would have gotten beaned every time they went to the plate. Although in Walter's defense he hit less than 1% of the batters he faced. When two players show such a deviation from the mean there must be more going on here than temperature because these two played even in cold whether.

I think the idea of looking at hit batters is an interesting one, but here I believe there was a strong desire to find a relationship with temperature. It would have been more interesting to look at all the potential factors in a hit batter (player, pitch, game situation, ball park, teams, weather, etc) and see what correlations existed.

Overall I am glad Scientific American took a shot at Baseball, but I wish they had taken a deeper dive into their chosen topic

A look at Batting orders

There is one Blog I read on sports statistics religiously and the is Phil Birnbaum's Sabermetric Research. It is a great read, and he looks at many aspects of lots of different sports as opposed to just baseball. If you have not looked at his stuff before check it out.

One of his recent postings dealt with a paper written by Nobuyoshi Hirostu who looked at if using expected runs was always the best way to determine the batting order or could a lineup with a lower expected runs produce more wins because of lower volatility. Nobuyoshi used a cut down version of the game to calculate the expected runs and ran a MC calculation to determine the winners of each potential matchup. For this experiment he used the 2007 season.

Out of the 600,000 potential matchup guess how many instances he found where the lineup with the lower expected runs won more than 50% of the games? 13! I was surprised there were not many more than that. I expected there would be a fair number of lineups of high batting average singles hitters that might have a lower expected number of runs but wins against a lineup of power hitters who score more runs on average but have great volatility due to lower batting averages.

Based on Nobuyoshi's approach to this problem I think the results are surprising, but correct. However, I can see some potential problems with how he constructed his model for analysis. First, by building a cut down model for expected runs he may have reduced the volatility of the various lineups and made the winning potential for a lower expected run lineup less likely. Second, the lineups were based on the player makeup of the various teams. For whatever reason, ( in baseball I usually assume tradition) most MLB have a lineup consisting of Power Hitters and Reliable Hitters. I think a very interesting question to ask is if this type of lineup is optimal. What type of lineup gets the highest expected wins, and does it do it with highest expected runs or some balance between high expected runs and lower volatility? 

Cleveland Indians are better than Sabermetricians Predicted

When I was at the Sabermetric Seminar in Boston. The Indians success in the first quarter of the year was a topic of discussion. The explanation given by Tom Tippett was that the Indians where over performing against the model and would over the course of the season return to their expectation. In support of that an expected run chart was put up showing the Indians with the greatest positive actual run differential versus expected run differential. The Red Sox were underperforming in respect to this measure.

While I understand there will always be statistical anomolies and periodic straying from the mean, I am not so sure that this is the case here. Modelers have a tendency to explain away differences from reality compared to there models as variation. While that may and will be the case sometimes for a three standard deviation outlier we are talking about a 3 in 1,000 chance. Rather than take that bet I would check to see if my model failed to take something into account. In the case of the Indians improvement, I would be more likely to look for shortcomings in my model because the Indians are a Sabermetric driven team and the guy who runs their analytics is a very talented guy. Teams do not share their models so there is no way of know if the various model are similar or even what input Data they use. A general impression from the Sabermetric conference is that Sabermaatricians do a lot of regression to the league mean which will smooth out the data, but may also underemphasize relevant data.

I believe even a quick look at even high level data for the Indians suggests their performance is not a wandering away from the mean but a shift in the mean.  Most of the difference in 2011 can be  attributed to the 233 runs scored in 46 games or 5 runs per game compared to 4 run per game in 2010. This can be explained because most sabermetric models fail to incorporate injuries into their models which was a factor in 2010 for the Indians and would negatively effect their run prediction in 2011. A lack of injury prediction and weighting due to past injuries in Sabermetric models is a major disconnect in Sabermetrics and needs to be addressed. The Healthcare industry has made great strides in this area in recent history with the use on ensemble methods.

Sabermetrics Seminar

I went to the Sabermetrics Seminar at Harvard this weekend. It was a charity event, and all the speakers came and talked on their own dime. I just want to thank those speakers for giving up their time for such a great cause.

The Seminar itself was an eye opening experience for me. The last seminar I went to was the R/Finance in Chicago. That Seminar, like most that I go to, is for hard core statisticians and computer scientists. I believe of the hundreds of attendees to R/finance I am one of the few without a PhD.  The presentations with the possible few exceptions of JD Long's honoring of Dr Suess were of a highly technicial level. The Sabermatrics Seminar was totally different. The audience varied from the Head of the Harvard Statistics Department and an eminent physicist to people with very limited mathamatical education. The presentations also ran the gambit from something that would be taught in a high school physics class to some fairly high level stuff. The great unifier in the room was these people loved baseball and where using mathamatics to expand their understanding of the game and increase their enjoyment. One Speaker, Dan Duquette, former GM of the Boston Red Sox, reminded us of the words of Flippe Alou to "remember to enjoy the game". Tom Tippet, Director of Baseball Information Systems, gave a great Q&A on the state of Sabermetrics in MLB today. I have included a link to a summary of the seminar here.

Sabermetrics is different than the other fields I work in. In Pharma, the models are widely shared, but the data is highly confidential. In Finance the models are confidential, but the data is basically public. MLB analysts seem to strongly guard both their models and there data viewing both as propietary. While I think this makes it a great opportunity for consulting, I believe it may hinder the rate of refinement. Kaggle has shown in a very public way that open collaboration on data and models yields astounding improvements in prediction.

It is a Sabermetrics Weekend so todays post is Sabermetrics

This weekend I am going to the Sabermetrics Seminar in Boston. Some might think that it is strange that I am excited about this given that I never played baseball, and I do not watch many games. However, the analytics being done is baseball is developing and expanding at such a rapid pace there is no way you can enjoy analytics and not be interested. Recently a friend of mine ran into Prof. Bertsimas and asked if he could have a copy of the now famous paper that he wrote predicting the Red Sox would win 100 games this year. Prof Bertsimas asked "are you a fan of baseball?" to which he responded "No, I am a fan of statistics". 

The development of Sabermetrics in the last 30 years has been to look at existing data and try to build predictive models out of that data. It was a good first step and produced some good results. This work revealed that some of the historical statistics, like ERA, were not good predictors of anything so Sabermetricians created statistics that were better predictors. This is all great, and it has taken Sabermetrics to where it is today.

The problem with the data that has been used today in baseball is that it is all result based data. The pitcher threw a strike or a ball, the batter got on base, etc. That is all changing. Welcome to the world of physical data in Baseball. This post on Beyond the Boxscore is a good example. It has taken the improvement of a players performance back to the physical location of his pitch not just that more of his pitches resulted in ground balls, but an attempt to answer why based on data not opinion. The technology exists not only to track data of a baseball as it crosses the plate but within the entire ballpark. First this is going to create an unbelievable amount of data that needs to be in studied in ways not currently used in baseball because of shear volume. Second this data is collected in real time which means the models could be updated in real time. Billy Bean may have had his 3X5 note card in front him, but the manager of the future may be holding his iPad with feedback on up to the last pitch and the suggested options with predicted results of those options.

One of the companies doing this physical data collection in baseball is Trackman. They also recently posted for an R developer. I can not wait to see what is coming!

Gelman writes about Bill James

I always knew that Professor Andrew Gelman of Columbia was a well known Statistician and Social Scientist, but when he writes an article in the Baseball Prospectus now he is famous. It is always good to see a statistician write about a sabermetrician. Although these two fields are really the same it seems they try to separate themselves from each other.

It was interesting to me that Gelmen wrote about James as a baseball outsider not too different from what James was to baseball in 1984 when he wrote the "Inside-out Perspective" article. Baseball may always need the outsider prospective to push it along because its traditions and beliefs are so deep.

I thought one interesting issue that Gelmen touched upon was how little of the real work in sabermetrics gets published. When Gelmen works on a topic he publishes a paper that discusses his approach, provides an example and the code to run the example yourself. Not so in Sabermetrics. I find little detail in the published articles and very little code. This results in people like James moving away from positions and theories without explanation. I think this hurts the development of Sabermetrics in some ways. My view on how science is developed is the path of how gravity was discovered through a series of theories that we accepted and then rejected. First there was nature abores a vacuum, then there was nature abores a vacuum up to 32 feet and then finally there was gravity at 32ft/sec.

It was a fun article to read in preparation of my attendance at the Sabermetrics Seminar at Harvard May 21-22

In Baseball too much data is never enough

A couple of weeks ago I ran across a post for a intern position at TrackMan which uses information of ball flight to improve performance. They have been very successful in golf. In fact I tried one of their units out over the winter. This job post was more interesting to me because it was looking for an analytic intern for baseball. My first reaction was just what baseball needs more data points in a hulking cloud of data.  Bill James and the Sabermatrics guys have already culled and studied the baseball stats to death even throwing out some stats as irrelavent and creating some others that are better predictors of results.

Then I realized the error of my ways. TrackMan is looking to enrich the result data with physical data. So not just if the ball was a strike or hit or even if it was a fast ball or a curve ball, but what was its speed, location and spin at points along its trajectory from mound to plate. This is very cool. In her talk at the NYC Rusers group Amanda Cox presents a heat map of Rivera's pitches crossing the plate versus other pitchers which was a simple piece of the total pitch but explains why Rivera was better in a very clear way (22:00). I believe this has the potential to change the way pitchers pitch and batters hit.

Analytics, Sabermetrics, Data Mining...Why can't we all just get along?

Sabermetrics was a term coined by Bill James to describe the analysis of baseball through objective evidence. Saber, or more accurately SABR, stands for the Society for American Baseball Research. With Bill James as its advocate. Sabermetrics has changed the way baseball is played. No easy task in a sport so encumbered by tradition. Baseball probably collects more data during a game than any other sport and each team plays at least 162 games a year. Rich data territory compared to the 16 regular season games played in the NFL.  Sabermetrics has taken a hard look at the core beliefs of what statistics make a good baseball player or team and runs them against the cold judgement of analytics. The results showed that some previously treasured statistics like batting average were not as important statistics as once thought, but others like on base percentage were better indicators. This is predictive analytics at it best. So it is time to call Sabermatrics what it is analytics.

It is funny for all the impact Sabermetrics has had on baseball I believe it is still limited by the traditions of baseball. Let me give you some examples.

The Blog Sabermetic Research talks about Buck Showalter changing the way his base runners play to gain 5 runs per year which he claims is worth $10 million dollars. Makes sense if the data he is using is good, but the key here is the decision is claimed to be made solely on the numbers.

Pitching is another story. In baseball a starting pitcher must pitch five full innings in order to earn a decision (win/loss).  Many talk about the difference between ERAs of starting versus relief pitchers. The data clearly shows that relief pitchers, even when they are the same person,  have an overall ERA .50 lower than starting pitcher or better. Tango on Baseball touches on the subject in this article. My question is that if relief pitchers have a better ERA than stating pitchers, and starters are generally accepted to be better pitchers than relievers why aren't starters being used like relievers? The impact would be huge! A quick pass says this .50 ERA reduction in starting pitchers would result in 40 less runs allowed by a team over the course of a season! Using Showalter math that is $80 million dollars. I believe the reason that this is not looked at as a solution is because of tradition. If starting pitchers where used like relievers they would never pitcher 5 innings, and therefore would never get  a decision. This would be a fundamental change in the way baseball is played.

In defense of Sabermetricians, there has been some discussion that ERA, like BA, is not a very useful statistic. This would mean that conclusions drawn from those statistics may not be as useful as they appear. I have not seen anything on starters versus relievers in terms of CERA, dERA, DICE or DIPS.

Why are the Red Sox better today? Sabremetrics or Construction?

I saw an article this morning from an MIT professor that predicted the Red Sox would win 100 games this year. That is a pretty bold statement since the Red Sox have only won 100 games in a season three times (1912, 1915 and 1946). However, it got me to wondering how have the Red Sox become so good in recent history. I often heard comments the claim that it is the payroll or the genius of Theo Epstein. Whenever I am with statistics guys, it is the hiring of Bill James and the use of Sabremetrics that made the difference. I have a third theory to put forth as the major reason for the improvement of the Red Sox in recent history, construction at Fenway. Oddly, this started the same year that Bill James was hired by the Red Sox, 2003.

From 1995 to 2002 the Red Sox had a combined record of 695-582 winning 54.42% of their games. From 2003 to 2010 the Red Sox had a combined record of 749-547 winning 57.79% of their games.

Year	W	L	Winning %		Year	W	L	Winning %
2010	89	73	54.94%		2002	93	69	57.41%
2009	95	67	58.64%		2001	82	79	50.93%
2008	95	67	58.64%		2000	85	77	52.47%
2007	96	66	59.26%		1999	94	68	58.02%
2006	86	76	53.09%		1998	92	70	56.79%
2005	95	67	58.64%		1997	78	84	48.15%
2004	98	64	60.49%		1996	85	77	52.47%
2003	95	67	58.64%		1995	86	58	59.72%
	749	547	57.79%			695	582	54.42%

So they are a got better after 2003 and Theo is a genius and Sabremetrics rules baseball. I am not so sure, and I think we reach those numbers based on a Simpson's paradox. Let me explain. If Sabremetrics had been the driving reason for the improvement the Red Sox. they would have gotten better not only at home but away as well. They did not. In fact the Red Sox improved massively at home, but got worse on the road. So what is the factor that explains this? In 2003, the same year Bill James was hired by the Red Sox, additional seating was added the Fenway park for the first time since it was 1946. While it was was always known that Fenway was helpful to certain types of hitters and pitchers and the Red Sox teams have always emphasized those players. I believe that construction made the park even more baised than it was before.



During the period 1995 to 2002 the Red Sox had a better away record than they did from 2003-2010.

Away Record W L % W L % 2010 40 41 49.38% 2002 51 30 62.96% 2009 39 42 48.15% 2001 41 39 51.25% 2008 39 42 48.15% 2000 43 38 53.09% 2007 40 41 49.38% 1999 45 36 55.56% 2006 35 46 43.21% 1998 41 40 50.62% 2005 41 40 50.62% 1997 39 42 48.15% 2004 43 38 53.09% 1996 38 43 46.91% 2003 42 39 51.85% 1995 43 28 60.56% total 319 329 49.23% 341 296 53.53%

For Home games it is a very Different story: Home Record W L % W L % 2010 49 32 60.49% 2002 42 39 51.85% 2009 56 25 69.14% 2001 41 40 50.62% 2008 56 25 69.14% 2000 42 39 51.85% 2007 56 25 69.14% 1999 49 32 60.49% 2006 51 30 62.96% 1998 51 30 62.96% 2005 54 27 66.67% 1997 39 42 48.15% 2004 55 26 67.90% 1996 47 34 58.02% 2003 53 28 65.43% 1995 43 30 58.90% total 430 218 66.36% 354 286 55.31%

MIT economist says Red Sox will win 100 games in 2011