Slumping Sophomores (Regression to the Mean)

What is the “sophomore slump” and the “Sports Illustrated curse” and are they real or just superstitions?

The sophomore slump is when the top rookies in some sport usually perform worse in their second year.  Similarly, the Sports Illustrated curse is when an excellent player is recognized on the cover of SI and then suffers a decline in accomplishments soon afterwards.  It turns out that these phenomena are very real and have nothing to do with players being psychologically affected by public recognition.  You might think that players could avoid the curse if they never learn that they’re on the cover of Sports Illustrated, but it turns out that they’re pretty doomed anyway.  So what causes this?

We data wonks are very familiar with this phenomenon of Regression to the Mean and see it everywhere.  We see it when sequels to great movies don’t live up to the originals.  We think of it when people try to tell us that punishment for bad behavior works better than reward for good behavior.  We nod with understanding when we are told to rebalance our investment portfolios.  We cringe when people tell us how they made a tweak to their under-performing websites and the improvement was immediate and dramatic.

What it basically boils down to is this: those who performed badly were more likely to have had a bad day (or week or year) and those who performed well are more likely to have gotten somewhat lucky.  It seems obvious when stated like that; after all, how often were the worst performers lucky to do as well as they did?  However, the results that follow from this truth can be subtle and surprising.

If you tell a scientist that you felt like you were on your deathbed, took a pill, and then woke up the next day feeling better, he or she will not accept that as evidence that the pill worked.  They’ll say “you need a control” and a bunch of other wonky stuff that you think doesn’t matter because how clear and obvious could it be?  Well, their objection isn’t only that unless you’re actually dying, you’ll generally improve on your own, it’s also that having your worst day ever is an unusual event and is hard to repeat.  Like Seinfeld, you probably have the strange sensation that all your bad days and good days even out.  What is really happening is that your fantastic days are almost always better than the days after them and your horrible days are almost always worse.

What is less commonly known than the sophomore slump is the opposite situation: let’s call it the “losers lift.”  To demonstrate this, let’s do an experiment.  I just looked at the stats in my fantasy basketball league (shout-out to basketball.sports.ws) in an unusual way… from the bottom up.  Ranked by fantasy points per minute, as of 11/21, here are the WORST five players in the NBA right now (who’ve played at least 10 games so far)…

Mike Miller (CLE) – 0.23

Nik Stauskas (SAC) – 0.29

Will Barton (POR) – 0.35

Alan Anderson (BKL) – 0.36

Jason Maxiell (CHA) – 0.37

I predict that over the next week, most or all of these players will improve (assuming that I use my data wonk magic and give their basketball cards a pep talk, of course).  I’ll follow up in a week and let you know what happened.

In the meantime, rebalance those portfolios, lower your expectations for sequels, go ahead and reward your kids for good test scores, and for God’s sake, use a control (random untreated group) if you’re trying to determine if something works!  Also, I’m sorry to tell you that if you really enjoyed this post, the next one probably won’t be as interesting to you.

Author: Jay Cordes

Jay Cordes is a data scientist and co-author of "The Phantom Pattern Problem" and the award-winning book "The 9 Pitfalls of Data Science" with Gary Smith. He earned a degree in Mathematics from Pomona College and more recently received a Master of Information and Data Science (MIDS) degree from UC Berkeley. Jay hopes to improve the public's ability to distinguish truth from nonsense and to guide future data scientists away from the common pitfalls he saw in the corporate world. Check out his website at jaycordes.com or email him at jjcordes (at) ca.rr.com.