Hello and welcome to this tutorial.
Okay, so we are almost done with part one Data pre-processing.
I can't wait to start making the machine learning models.
We just need three more tutorials to make our data set perfectly prepared
before starting the models, and then we'll be good to go.
Okay, so today we are going to talk about the fact that we need to split
the data sets into a training set and a test set,
and I'll explain why we need to do that right now.
So let's go to Google Sheets.
Well I'm on Google Sheets but you can be on Excel.
You can be on whatever tool you want.
But here is the data set open with Google Sheets.
So we have our ten observations and this is the data set.
This is the whole data set.
And what we should do in any machine learning models
is that we're going to split this data set into two separate sets,
which are going to be the training set and the test set.
Now why do we need to do this?
Well, when you take a step back and focus on the name machine learning itself,
you understand that this is about a machine that is going to learn something.
Well, here it's your algorithm.
It's your model that is going to learn from your data
to make predictions or other machine learning goals.
And so your machine learning model is going to learn to do something
on your data set by understanding some correlations
that there is in your data set.
And imagine your machine learning model is learning too much on the data
set like it's learning too much the correlations, then I'm not sure its
performance would be great on a new set with slightly different correlations.
Or, you know, it's like a student who is learning by heart his lesson,
and then when he takes the exam, he might be in trouble
because he learned too much his lesson by heart,
and he does not manage to make the connection between what
he learned and the exam.
And it's the same for machine learning.
We are going to build our machine learning models on a data set,
but then we have to test it on a new set, which is going to be slightly different
from the data set on which we build the machine learning model.
So we have to make two different sets a training set
on which we build the machine learning model, and a test set on which
we test the performance of this machine learning model.
And the performance on the test
set shouldn't be that different from the performance on the training set,
because this would mean that the machine learning models understood
well the correlations and didn't learn them by heart
so that you can adapt to new sets and new situations.
Okay, so that's the idea about splitting the data sets into a training set
and a test set.
And now let's do it on R.
So here we are on R.
The section is really splitting the data set into the training set and test set.
And let's start coding the thing.
Now that you understand well the difference
between the training set and the test set, we're going to do it a little faster.
So new thing here we have to import a library.
We're going to import the library that is going to make a good split
of the data set into the training set and the test set.
And this library is called CA tools.
So let's import it
here I'm going to the packages tab to see the list of the libraries.
Here you can see that the CA tools library is here
I have it installed on the packages because I installed it before,
but it's probably not the case for you if you're studying R for the first time.
So we're going to install it to install a library in R, it's very simple.
You have to type install dot packages
and then in parentheses quotes.
And then the name of the library in quotes.
So here we type CA tools and we're ready to go.
So then you have to select this line and press Command or Control.
Press enter to execute.
Here it is.
And right now it's installing the package CA tools
okay perfect.
So then you have to put it as a comment because you will need to install it again.
But then as you can see we just installed the CA tools library.
But it's not activated yet.
You know it's not selected and we have to select it.
So to select a library you have two choices.
Either you click on the box here.
As you can see this generates two scripts here.
Or you know if you have some scripts
that you want to automate and execute once in a time.
Well, you can specify in your script that you want to include the library.
And to do this you just need to type library
parenthesis and CA tools the name of your library.
Not in quotes.
Actually, this time library tools okay, perfect.
And now you're ready to go.
We can check the tools is not selected right now.
And if I select this and press Command or Control to execute now it is.
Now it's selected.