How Bad Are We At Predicting Talent?

If sports drafts are any indicator, very bad

A few years ago, Cade Massey and Richard Thaler came out with a paper that looked at the "return on value" from these early draft picks. In essence, they constructed a model in which all the players at a given position – quarterback, running back, linebacker, etc. – were ranked according to the order in which they were picked in the draft. … As Thaler notes, if teams knew nothing, the player that went higher in the draft would outperform the lower ranked player 50 percent of the time. In other words, drafting talent would be roughly equivalent to a coin flip; all the scouting would be perfectly useless.

In contrast, if teams knew what they were doing – if they could effectively identify the best college players – then the higher draft picks should outperform their competition close to 100 percent of the time. So what did Thaler find? Flipping a coin is the apt metaphor, as the higher picks proved better only 52 percent of the time. The teams beat randomness, but barely.

In response the latest Jeremy Lin post, a reader writes:

People with looks out of Central Casting are always going to have an advantage in sports. Ask any tall person and they can recall the countless times they've met people and are invariably asked, "Do you play basketball?" There are countless baseball players who recieved more chances than they should have because "they just look like a ballplayer."

Billy Beane, the top draft pick who failed in the majors, used this issue in Moneyball. Here was a guy, Beane, straight out of Central Casting himself, who built a successful team on the cheap using players with skill sets that scouts with decades of experience passed over. Pudgy players who got on base a lot or efficient pitchers who threw "funny". You had to look past the exterior to see their talents.

More coverage of Lin's unexpected success here and here.