Hello and welcome to this art tutorial.
So in the previous tutorials, who took care of the pre-processing phase?
And then we applied PCA on our data set to reduce its
dimensionality down to two new extracted features.
And now we are ready to build a classification model.
So speaking of classification model, we started with the logistic regression
model.
But actually from this point we can build any classification model
among all the classification models we made in part three.
If I go back to part three classification, this folder here
you have here all the models we made in this part three.
And basically from this point you can build any model
you want by just selecting your classification model.
For example, let's take support vector machine classification model.
Then you can just open the SVM or file.
And then basically all you need to do is take everything after the data
preprocessing phase.
That is from where you start to build your SVM model.
And select everything down to the bottom and copy.
And then in your PCA file you can include your classification model.
Right after applying PCA on your data set.
And so here I just replaced the logistic regression model by the SVM model.
And you can do this for any classification model you want.
Among the classification models we made in part three.
So that's very easy to replace your different models by this simple
copy paste so that you can try different classification models very efficiently.
So let's see what we get with this SVM model.
For example we just need to change here the name of the dependent variable.
This is not purchased but customer segment.
Here we go.
And that's the only thing we need to change
because the data here has input training set.
And this is the training set which is transformed.
The new training set composed of the new extracted features.
And so basically we are ready to select this section
and execute it to build our SVM classification model.
And now that the model is built,
we are ready to predict the new observations of the test set.
And so this line is ready.
And actually we don't need to change anything
because this index three here is the index of the dependent variable.
And since we reduce the dimensionality of our data set down to two,
that means that we have two features and one dependent variable.
And therefore the index of the dependent variable is still three.
And so we are ready to select this line and execute.
And now we have the predictions of the test set results.
So we can have a look y pred in the console.
Press enter.
And for each observation of the test set we have its prediction by the model
the SVM model.
So for example, the fourth one of the data set that belongs to the test set
is predicted to belong to customer number one.
And the one number 132 is predicted to belong to customer segment number three.
So very easy.
And then we can make the confusion matrix.
And since we don't need to change anything here
because this corresponds to the index of the dependent variable.
So we are ready to execute this as well.
Execute the confusion matrix is ready.
Let's have a look.
And while
perfect results we only get correct predictions.
As you can see, 12 ones were correctly predicted to belong to customer.
Segment number 114 ones were correctly predicted to belong to customer segment
number two and ten ones were correctly predicted to belong to customer segment
number three.
And then we have zero incorrect predictions.
So these are excellent results.
And of course we get 100% accuracy.
So now when moving on to the next part to visualize the training set results,
we should get amazingly
well-separated prediction regions and a very clear prediction boundary.
So let's check it out.
But now we have something to change.
And this is not a tiny change as we used to do, because now we have three classes.
And as you can notice in this code when we plot the prediction regions
thanks to this line,
well this code template allows us to do it when we only have two classes
because as you can see, we have this if else condition.
If y grid equals one, then the color is green and else if y green equals
zero, then the color is tomato and same when we plot the observations.
If the observations of the set that is the training
set belongs to class one, then it's green.
And if it belongs to class
zero, that is in the else condition the points will be red.
But now the problem is that we have three classes.
So we need to improve this code here to distinct the three conditions.
If y equals zero, if y equals one, and if y equals two.
So let's do it.
That will be good coding practice.
And speaking of coding practice, what would be very good is that you
try to do it before I do it in this tutorial.
So you can press pause and try.
And now I'm going to do it.
So basically we need to add one more condition.
The condition where y equals to.
So let's do it.
Let's add this new condition here.
If y grid
equals equals to then comma.
And then after this condition y grid is equal to two, we will put what we want.
And what we want is a new color because there is one color
associated to each value of y grid.
So we will keep spring green three for the case where y grid equals one.
And we will keep tomato for the case where y grid equals zero.
But for y equals two we need to introduce a new color.
And since we have here green and red let's put blue.
So a good color is actually deep sky blue.
Then come up to get the next conditions.
So so far what we see is that if y grid equals
equals to then the color will be deep sky blue.
Then if y grid equals one then the color will be green.
And if y grid equals zero and then the color will be red.
But this is not how it works.
It's not as simple as that because this is actually not a correct syntax.
Because this ifelse function expects three arguments.
The first argument is the condition y grid equals one.
Then the second argument is the result when this condition is true,
and the third argument is the result when this condition is not true.
So here we have a lot more than three arguments.
That's not right.
And so the trick to solve this is to put all this
that is the y grid equals one condition.
And then the results bring green three.
And then the result if y grid equals zero into the third argument of this.
If else function.
So that means that will get the first argument y grid equals two.
That's the condition.
Then the second argument deep sky blue.
That is the result when y grid equals two.
And the third argument all this in one same argument.
And so how can we include all this in one same argument.
Well we need to use another ifelse here, which will contain the other
two conditions where y grid equals one and y grid equals zero.
And so we need to be careful with the parenthesis
because we added a new function.
This new function if else. And here it is.
The new parenthesis is added.
And now it should be fine.
So let's recap.
We start with this first ifelse here.
So if y grid equals two then the color will be sky blue.
And then if y grid is not equal to two then we go into this new if else
and this new if else contains the two last remaining conditions.
That is if y equals one, then the color will be spring green.
Three.
And if y equals zero, then the color will be tomato like red.
And therefore we get our three conditions in the correct syntax.
So that's a trick.
It's actually quite common to do it in coding.
So it's good to know how to do it.
And that's the same to plot the colors of our observation points.
So we need to take this and paste it here again.
And then replace this one here by two.
So that is the new first condition.
If our observation point belongs to class two then we want to give it a new color
which will be a blue color but a different blue then this deep sky blue.
And so, you know, we need to get a good contrast
so that we don't confuse the color of the point and the color of the region.
So actually a good color to use here is blue three, blue three.
You'll see that it will give us a good contrast.
And so that's the first result of the first condition.
And then same.
We need to include the two remaining conditions here
into one argument that is inside a new if else.
So if else here in parenthesis.
And we don't forget to add the closing parenthesis here.
And here we go. This is ready.
So recap again if our observation point belongs
to class two then it will have the color blue three.
Then if it doesn't belong to class two then we go here.
And here we have two new separate conditions.
If our observation points belongs to class one
then it will have the color green four.
And if it doesn't belong to class one then it will have the color red three.
So that should be ready.
And then we have two tiny changes to add.
So remember in this line here line 49 with the column names
we need to input the real column names of the columns of the training set.
And these column names are not age and estimated salary.
That was for the previous classification problem.
Now the column names are of course
PC1 and PC2.
So here we just need to replace age by PC1
and estimated salary by PC two.
So that's compulsory.
That's actually what you need to input.
Otherwise you will get an error when you execute your code.
And then here it's not compulsory but it's better for the visualization.
You can replace age by PC1
and estimated salary by PC2,
but if you don't do it,
you will not get an error because this is just for the visualization.
This is just for the labels that you will see on the graph.
All right.
And then I think we're good I think this is ready to be executed.
Let's hope that I didn't make any mistake.
So we're going to try to execute this and let's see what we get.
So I'm going to select everything in this section.
So from here up to the top here
and let's execute.
All right. Good start.
It's running.
Let's see what we get.
Let's go into this plot tab.
It is still running.
And here we go.
We get our beautiful results.
So I hope you like my choice of the colors.
This is the deep sky blue.
And this is the blue three.
So that we get the contrast
between the observation points and the prediction regions.
So we can actually enlarge this if you want.
So as a quick reminder the points are the real observation points.
That is these are the ones that we have in our training set.
And the regions are where our model predicts
that the ones belong to the customer segments.
So for example,
the green
points are the ones of the training set belonging to customer segment number two.
And this green region.
Here is where the model predicts that the ones belong to customer
segment number two.
And same for the blue and red parts here.
All right.
So now we can quickly do the same for the test set.
So we actually need to do the same changes as we did for the training set.
That is let's start with the simplest one we need to replace here edge by PC1
the estimated salary by PC2.
So these are compulsory changes.
And we can also change the labels even if that's not compulsory changes.
Replace H by PC1.
Replace estimated salary by PC2.
And here we go. We are almost ready.
We need to make this big change here to add the third condition.
To add the third color and we can actually take these two lines here.
Copy them and
select this and paste.
All right.
We can do this because these are the same variable names as for the training set.
Because we use this set variable name here
for both the training set and the test set.
And so basically that's ready.
We can now select this whole section here
and execute to visualize the test set results.
So let's do it.
Here we go to processing.
The test set. Results are coming.
And we should get a perfect plot with no incorrect predictions.
That means that we should get all the green points in the green region.
All the red points where here we go.
All the red points, as you can see, are in the red region
and all the blue points in the blue region.
So that's perfect.
That's a perfect representation of 100% accuracy.
And so in conclusion, we were able to transform a data set
composed of 13 independent variables into this new data set of reduced dimension.
We were able to reduce the dimension down to two,
thanks to which we could visualize the results in two dimensions.
Okay, perfect.
We are done with this first section about PCA.
And now the interesting thing that we want to see
is how our next dimensionality reduction technique
that we are going to implement is going to do on this data set.
This next dimensionality reduction technique is LDA Linear Discriminant
analysis.
So we'll find out about that in the next section.
And until then enjoy machine learning.