Analyzing the Last Five Years of Rotisserie Baseball Drafts

How many of the top hitters and pitchers at the end of the year were actually drafted? How many of the top hitters and pitchers were not drafted and were picked up during the season?  Were hitters or pitchers drafted more accurately?  What is the dollar value earned by the players that were picked up during the season?  Is there a position of hitter that’s more reliable than other positions?

Have you ever asked yourself draft analysis questions like these?

What follows is a five year analysis (with colorful graphs and an enormous Excel file!) of how accurately our projections in the preseason depict what has actually happened at the end of the season. How well we drafted.  What positions yield the best returns.  What positions offer the most free loot.  And more.

Assumptions You Should Know

A number of the graphs depend on dollar value earnings for the “top 168” projected hitters or “top 108” projected pitchers.  The dollar values are calculated using the approach documented in “Using Standings Gain Points to Rank and Value Fantasy Baseball Players” assuming a 12-team league, $260 team budget, 14 hitters (C, C, 1B, 2B, SS, 3B, CI, MI, OF, OF, OF, OF, OF, UTIL), 9 pitchers, and a 70%-30% hitter-to-pitcher allocation.  That’s a total of 168 hitters and 108 pitchers.

These top projected players in the preseason were determined using Steamer’s preseason projections for that season (I downloaded the historical projections here).

I suppose using ADP results or expert rankings from the given year might give a better picture of the players that were actually drafted, but then you get into the question of what’s good ADP data, where to get it, what experts to use, league differences, lineup differences, etc.

To Be Clear…  The Goal of this Study

The goal of this is not to measure the accuracy of particular experts.  It is to determine which positions can we draft and get the most return on our investment.  To some extent this is a review of Steamer’s accuracy, but that’s also not my intent.  It’s my understanding (tell me if I’m wrong) that there are not significant differences between the top projections systems.  So whether we were looking at PECOTA, Steamer, or Marcel projections, we would see similar results.

How Much of a Return Do We Get For Drafting HItters vs. Pitchers?

People have long been telling us to, “Load up on hitters early in the draft”.

“Don’t overspend on pitching.”

“Wait on pitching until most teams already have one.”

I’ve always heard these things.  They sounded right.  But I can’t say I’ve ever seen the data to support it.

In looking at the chart below it is very clear that we are much better at identifying the top hitters than the top pitchers.  The top 168 hitters in the preseason provide about 70% of the dollars earned at the end of the season.  For pitchers, it’s more in the neighborhood of 40%.

With results like that it’s very easy to see why the hitter-pitcher split is not 50-50.

Hitters are safer investments than pitchers.  We’ve always been told this, but now you can see it.  And things have not changed in the new era of pitching that we’ve been seeing the last few years.  If anything, the gap seems to have widened.

If you’re curious about the math on this, read this paragraph.  Otherwise, skip on over this one.  Going back to our assumption of a 12-team league where each team has a $260 budget, that gives the league $3,120 to work with.  If 70% of that is earmarked for hitters, that means $2,184 of value is allocated to the pool of 168 hitters and $936 is designated for the 108 pitchers.  The 70% return on hitters means that we projected who the top 168 hitters would be in the preseason and by the end of the season those same 168 hitters were worth about $1,530 ($2,184 * 70%).  So about $654 of value came from hitters that were not drafted.

Why Does That Chart Say “(includes losses)” In The Title?

The calculations above were done in a “draft and hold” scenario, meaning if Cliff Lee is projected as a $20 player prior to the 2014 season and his end of season earnings are a negative $9, he returned a value of -$29.  These “losses” can really affect the calculations.

You’re probably thinking, “But not all pitchers (perhaps none) are held on a roster when they’re having a really bad season”.  And if a pitcher suffers a season ending injury, they would be cut immediately and also would not end with negative earnings for the season.

A bit further down you’ll see charts for scenarios where losses are eliminated.  So instead of negative $9 Lee comes in with just a $0 return.  

This scenario of Lee breaking even is a little too optimistic.  If a pitcher is struggling and on his way to a negative earning season, it can take weeks or months for us to make that determination.  As that time goes by you’ll be accumulating some of those negative earnings on your team.

The true return lies somewhere in between the “no losses” and “includes losses” lines.

How Much of a Return Do We Get at Each Position of Hitters Vs. Pitchers?

We’ve seen that hitters as a group are a greater investment than pitchers.  But what about the individual hitter positions?

There’s a lot going on here, but it’s very clear that any position of hitter is a safer investment than a pitcher.  Outfield appears to be a very consistent producing position. Rarely the highest in a given season, but steady.  SS and 3B fluctuate wildly.

1B might be the overall lowest returning position of all.  Prince Fielder, Joey Votto, Chris Davis, Paul Goldschmidt, Billy Butler, Eric Hosmer, Brandon Belt, Mitch Moreland, Kendrys Morales, and Adam Lind really dragged the return down in 2014, while Albert Pujols, Adrian Gonzalez, Joey Votto, Mike Napoli, and Lance Berkman disappointed in 2012.

We’re a long ways from the glory days of Todd Helton, Jason Giambi, Jim Thome, Carlos Lee, Paul Konerko days when 1B seemed like a lock every season.

But We Don’t Have To Draft and Hold.

As I mentioned above, very few of us play in draft and hold leagues.  We can replace struggling or injured players that appear to be negative earners.

There’s no way to measure exactly when a player will be dropped.  The best we can do is to remove the possibility of a player returning negative earnings.  A replacement level player would have earnings of $0.00.  So assigning a value of $0 to players that would otherwise have negative earnings makes some sense.  If a struggling player is performing at about the same level as available free agents, we would probably hold on to them and hope they can turn it around.

When we remove the possibility of losses, the gap closes.  Pitchers come much closer to the results from hitters.

Splitting all hitters into separate positions, pitchers are not usually the worst performing position in any given year.  But unfortunately pitchers do look like the worst performing investment

 The Truth Lies Somewhere in the Middle

We don’t play in draft and hold leagues and there’s not a magic fairy that tells us exactly when to drop a player before he starts earning us negative return.  The actual return we can earn for a hitter lies somewhere between the blue lines below.  The actual return for a pitcher lies somewhere between the red lines.

How Much of the End of Season Earnings Were Acquired During the Draft?

Let’s shift gears a bit.  Of all the earnings for each position at the end of the year, how much of them were obtained through the draft?

You can see in the chart below that we’re very efficient at drafting all of the earnings from 1B and C.  My guess is that’s not a function of us being better at projecting 1B and C.  It’s more due to the fact that these positions have a greater percentage of MLB starters drafted than other positions.  In a two-catcher league, 24 out of the 30 starting catchers are drafted.  First base is similar in that players are drafted to player first 1B, corner infield, or the utility spot.

So even though we saw above that 1B provides a pretty low return (when we allow for negative earnings), we still end up drafting nearly 90% of the value at 1B.  This implies that there are not a lot of valuable free agent pickups that come along at 1B.

How Many of the Top Earning Hitters and Pitchers Were Drafted?

Forget about earnings and dollars for a minute.  How many of the top 168 hitters at season’s end were drafted?  How many of the top 108 pitchers?

It’s starting to look like any way you slice it, hitters are a more sound investment.

How Many of the Top Position Players and Pitchers Were Drafted?

Let’s break hitters apart into different positions.

This does support the theory I presented above that we draft more 1B and C than other positions so we have a greater chance of getting more of the top players at the position right.  The more times you toss a coin, the more heads you’re going to land on.

Another thing that jumps out at me is the significant dip for 3B in 2012.  That year we actually did a good job of identifying 3B in the preseason.  Miguel Cabrera, Adrian Beltre, David Wright, and Ryan Zimmerman all earned more than projected.  Chase Headley was pegged as the 11th best 3B and we caught lighting in a bottle when he earned $27.  But we did not see Mark Trumbo, David Freese, Kyle Seager, Pedro Alvarez, Chris Johnson, or Todd Frazier coming.

It’s finally dawned on me why the OF line is so stable in all of these graphs.  It’s simply a numbers game.  When we draft 60-70 OF each season, that denominator stabilizes because of the larger sample of players at that position.  If we were only drafting 14-18 like at 3B, SS, and 2B, the lines would be subject to much greater fluctuation.

How Many Valuable Players Come Into The League Each Season?

This graph is the exact inverse of the  “% of Final Top 168/108 Players Drafted” chart above, but I think it’s helpful to view it from a different perspective.  Instead of thinking, “How good at we are identifying the good players in the preseason?”, we might flip it around and wonder, “How many good players are going to be available through free agency once the season starts?”.

This is the concept of “free loot” that I learned of from Peter Kreutzer’s Rotomansguide website (which I highly recommend checking out if you like thinking of the concepts in this post).

 No surprises here.  More pitchers become available through free agency.

Take a serious minute to think about what these numbers mean.  Of the nine pitchers you draft, four of them you’ll be completely wrong on.  They are not even going to be in the top 108 pitchers at seasons end.  Five right.  Four wrong.

Of the 14 hitters on your team, you’ll probably only be wrong on four of those too.  Ten right.  Four wrong.

Percentages can be deceiving at times.  So how about the actual number of undrafted players that end up in the top earners.

And by position.

I’m not sure there is much to take from this.  But I figure I’ll show it to you anyways.  I think this is largely due to the function of us drafting 108 pitchers, 60-70 OF, and 14-20 players at other positions.

The Data

Unfortunately the Excel file is so large that I can’t post it for you to view online, but you can download the file here (SGP Reliability Research for Download.xlsx).  There are tabs for projections and actual stats for hitters and pitchers for each season from 2010-2014.  The “SGP Charts and Summarized Data” tab summarizes the returns for each position, each year, and also contains all the graphs above.  The “Historical SGP Results” tabs summarize the projection and actual results for each hitter from 2010-2014.

Conclusions and Takeaways

There is a lot of information to take away from this.  But I now have very easy to understand visual proof as to why you are best off delaying on drafting pitchers.

There’s also no truth to the rumor that because pitching has become more dominating in recent years that you need to draft it any earlier.  Pitching comes out below hitting in nearly every measure in every season.  Even in the “pitcher’s era” we find ourselves in now.

I also realize I need to treat hitters and pitchers with different levels of patience in the season.  Looking back, I’ve found myself trying to practice high levels of patience with underperforming pitchers.  Given the higher level of turnover in the top 108 pitchers than the top 168 hitters, it’s fine to be less patient with pitchers and more patient with struggling hitters.  We’re wrong about four out of every nine pitchers!

Why It’s Important to View Things in Terms of Dollar Values

We don’t all play in auction leagues, but there is still value in converting player statistics into dollar values.  If we don’t convert player statistics into some singular number to rank players, the analysis above is not possible.  We could just use each player’s standings gain points, but there’s something more meaningful about using dollars.  And over time we’ve come to have an understanding of what a $30 or $40 season means.

What Is Left To Be Done?

I’ve answered a lot of questions above.  But this has led me to more questions.  So the return on ALL hitters is about 70% and the return on ALL pitchers is in the neighborhood of 40%.  What’s the return on all $30, $20, and $10 hitters?  Pitchers? What about $20 1B versus $20 OF?  How does the typical $10 SS pan out when compared to the $10 pitcher?  What about 30 year old hitters versus 27 year old hitters?

I’ll get there some day…

What Did I Miss?

Did you see something interesting in the charts above that I didn’t point out or comment on?  Let me know in the comments below.

What Do You Want To Know?

What questions were popping up in your head as you read this?  Let me know in the comments and I’ll attempt to find the answers.

Thanks for reading.  Stay smart.