What is overfitting and what it has to do with dating?

Overfitting is a concept from statistics, but I will use a dating example to make it easier to grasp. ​

So imagine that you have a girlfriend. But not an ordinary one. Your first serious girlfriend! Her name is Megan.

She's 20 years old. She's blonde.​ ​She's 170 cm tall. ​She studies at Harvard. ​

She loves: ​
- Star Wars,
​- Cuban music
​- and playing squash.

You really love each other. But after two years, you have to move from NYC to LA. And you break up. ​

These are Megan’s features placed on the chart.

So you think:
"I'll never find a girl like her! There're no 20 years old tall blonde Harvard students named Megan who love Star Wars, Cuban music, and squash." ​

And you're right - she was one of a kind. ​​But you're also performing overfitting, which closes your chance to fall in love again.

This line, connecting the features, symbolizes Megan.

What's overfitting? In simple words, it’s focusing on particular data inside the model (i.e., "I want to date a Harvard girl named Megan") instead of looking for wider trends (i.e., "I want to date an educated girl"). It's getting so specialized that it stops making sense.

Looking for a person that matches Megan’s exact line is a futile task. She was one of a kind.

Overfitted model can't help you predict what girls you would like to date. So you think you can only date Megan.

So what could you do? Look for more fundamental rules. ​ It may turn out that you like girls who are young, tall, educated, and love sci-fi, Latino music, and sports.

If your line tries to capture the trend of “Type of girls I like” instead of “Megan,” then the line will be different.

It may also turn out that you don't care about the hair color. So your date can be either blonde or brunette. Your model has become better at predicting "What types of girls I'd like to date." Thanks to that, you are open to more dating opportunities than before.

You can fall in love again!

As your line is focused on types of girls, you have more dating opportunities.

So why do people perform overfitting? Because relying on the past gives us a feeling of certainty.

You are sure that you loved that particular Harvard girl named Megan!

And if you say: “Okay, I like tall, educated girls,” then there’s a chance that dating some of them will be terrible.

That's the trade-off. It’s worth paying the price, though, because the overfitted model is only great at explaining the past. But very poor at predicting the future.

So overfitted model is like an expert that's too specialized. Just like Nicolas Butler's quote about experts: "An expert is one who knows more and more about less and less until he knows absolutely everything about nothing."