Hello and welcome to this tutorial.
Now we are finally going to start preparing the data
so that our machine learning models run correctly.
And the first problem that we have to deal with
is the case where you have some missing data in your data set.
And that happens quite a lot actually in real life.
So you have to get the trick to handle this problem
and make it all good for your machine learning model to run correctly.
So let's have a look at the data set.
Here I'm going to my Google Sheets tab.
Okay so here is the data set.
And as you can see there is two missing data.
There is one missing data in the H column here for Spain
and one missing data in the salary column for Germany.
So how can we handle this problem?
Well the first idea is to remove
the lines, the observations, where there is some missing data.
So what we could do is to remove this line and remove this line.
But that can be quite dangerous because imagine
this data set contains crucial information.
It would be quite dangerous to remove an observation.
So we need to figure out a better idea to handle this problem.
And another idea.
And that's actually the most common idea to handle missing
data is to take the mean of the columns.
So here we are going to replace this missing data here
by the mean of all the values in the column h.
And that's the same for every feature that contains missing data.
We replace this missing data
by the mean of the values in the column that contains this missing data.
Okay. So let's do this.
First we need to take our data set.
So data set.
Let's start by taking care of the missing value in the h column.
So here we will need to take the h column.
And to do this in R we need to add a dollar sign here.
And here we choose h.
So by doing that data set dollar h we're taking the column age of the data set.
Then we're going to add equals.
And then we're going to add an if else.
So I'm going to type if else here then parentheses.
In the ifelse function you have to input three parameters.
The first parameter is your condition.
If condition the second parameter is the value you want to input.
If the condition is true
and this is the value you want to input if the condition is false.
So let's first input the condition.
The condition is going to be is.na parentheses.
Data set dollar age.
And that's it
is Na is a function that tells
if the value in the function is missing or not.
So by putting is Na data set dollars age.
We are checking to see if all the values in the column age are missing.
So this will return true if the value in the column age is missing
and false if the value in the column h is not missing.
Okay, so that's the condition.
And now the second parameter is the value that is going to be returned.
If the condition above is true.
And of course if the condition is true that means that there is a missing value.
And that means that we have to replace it with the average.
So here we're going to impute
as the second parameter the average of the column age.
Okay.
And to compute the average in R there is a simple way.
We can type RV then data set
dollar sign
age because we want to take the column age.
Then comma.
And then here we're going to add a function.
So we're going to type fund in capitals phone equals
then function then function x.
This is still part of the R syntax.
We're just making a function here which is going to be the mean function.
And then we have to specify what this function will be.
And so this function is of course the mean which is an existing function in R.
So here parenthesis x comma.
And here we're just going to add na dot
rmm equals true.
And that means that we ask.
And that means that we ask R to include the missing values.
When R will go through the whole column aid to compute the mean of the values.
And then we still need to close the parenthesis here again then comma.
So that's it for the second parameter.
And now we need to add the third parameter.
And in your opinion what is it going to be.
Well this third parameter is the value you want to return.
If the first condition is not true.
That means that the value in the column age is not missing.
So that means that the value exists.
So here we are simply going to put data set
age okay.
And that's done by typing this we replace the missing value
in the column age by the mean of the column age itself.
So let's select this and let's see what happens.
So command and control plus enter to execute.
And here it is and executed properly.
And now let's look at our data set by clicking on this tab here.
And good perfect.
The missing value that was here in the age column was replaced
by the mean of the column. Great.
So now let's do the same for the salary.
We're just going to copy this paste.
And we're just going to replace age by the salary here.
Here as well.
And also here.
Great.
And now we have to be careful with something.
It has to be a line here.
So we just have to do this.
And same for here.
This okay. And now we're fine.
Now we're ready to select this code section here.
Press Command and Control plus enter to execute.
Here it is.
Let's check our data set and perfect.
The missing value in the salary that was here was replaced
by the mean of the salary column $63,777.