Hello my
friends, I hope you digested well the previous tutorial where we tackle
this big but yet important tool of our data preprocessing toolkit.
Indeed.
Now you know how to handle the case where you have some categorical data
in your data set, which is a situation you will encounter many times
in your future machine learning career.
And now we have two tools to cover, the first one being splitting
the data set into the training set and the test set,
and the second one being feature scaling.
So before we start, I'm about to answer
one of the most frequently asked questions
in the data science community, which is be ready for it.
Do we have to apply feature scaling before splitting
the data set into the training set, and to set or after?
I've seen this questions many times and you will find
that question in many forums of the data science community.
Some people will say that we have to apply feature scaling before the split.
Some people will say after the split and now I'm about to reveal the right answer.
There is only one right answer, which is, by the way, totally obvious.
After you get the explanation.
So the answer is we have to apply feature scaling.
After splitting the data set into the training set and the test set.
And now let me explain.
So first just to make sure everybody understands.
Let me explain the what first. And then I'll explain the why.
So of course splitting the data set into the training set and the test.
It consists of making two separate sets.
One training set where you're going to train
your machine learning model on existing observations, and one test set where
you're going to evaluate the performance of your model on new observations.
And it's important to understand that these new observations are exactly like,
you know, some future data that you're going to get and on
which you're going to deploy your machine learning model.
All right.
So that's this first tool.
And now feature scaling simply consists of scaling all your variables,
all your features actually to make sure they all take values in the same scale.
And we do this so as to prevent one feature to dominate the other,
which therefore would be neglected by the machine learning model.
All right.
So that's the what for both of these tools.
Now let me explain the why we have to apply feature scaling.
After splitting the data set into the training set and test it.
It's really obvious.
It is for the simple reason that the test set
is supposed to be a brand new set
on which you are going to evaluate your machine learning model.
So it's exactly like, you know, your training, your machine learning model
on your training set, and then later on, you know, after it is trained,
you're going to deploy it on new observations.
So what this means is that the test set is something
you're not supposed to work with for the training.
And feature scaling is as you will see, a technique
that will get the mean and the standard deviation
of your feature, you know, in order to perform the scaling.
So if we apply feature scaling before the split,
then it will actually get the mean and the standard deviation
of all the values, including the ones in the test set.
And since the test set is something you're not supposed to have,
you know, like some future data in production, well, you know, applying
feature scaling on the original data set before the split would cause some
what we call information leakage on the test set.
You know, we would
grab some information from the test set, which we're not supposed to get
because it is supposed to be new data with new observations.
So remember this the essential reason why you should not apply feature scaling
before the split is to prevent information leakage
on the test set, which you're not supposed to have until the training is done.