Juking the Stats

It’s extremely difficult to present an honest backtest because it’s close to impossible to distinguish one from an overfitted one … not unless the audience knows how to ask the right questions.

Overfitting is a huge problem in finance, including academia. If there isn’t any real skin in the game, there is a large incentive to cheat.

The Wire touches on the topic:

Making robberies into larcenies. Making rapes disappear. You juke the stats, and majors become colonels. I’ve been here before.

Marcos Lopez de Prado has been doing a lot of interesting research on this very topic. Check out an interview of him here.