Hello my friends.
Welcome back.
And mostly welcome to part four, making the predictions and evaluating the model.
And at the end of the previous tutorial, I actually asked you to do a little
homework, which is right here,
and which consisted of using your Ein and model to predict
if the customer with the following information will leave yes or no.
The bank.
So I hope you did well.
You actually had all the tools to do it.
And that's something we did before in part three classification.
So I'm sure you smashed this.
Let's see about that.
Let's implement the solution together.
All right.
So I left this code cell here to enter the solution.
But this is just a common.
So let's just remove it.
And now how do we complete successfully that homework.
Well of course we need to get first or a and model from which
we're going to call a new method.
You know, different than the add method or the fit method.
Right now of course, we need to call the predict method right.
Absolutely no trap.
We call this break method to predict if yes or no.
The customer with the following informations will leave the bank.
All right.
And now since this is a method I'm going to add some parentheses here.
Let me actually scroll down.
You know in case you're uncomfortable with the border at the bottom.
All right. So predict then okay.
So you have all the information of the predict method.
But we know exactly what to predict.
This is given in the instructions of the homework.
We have to enter these informations.
But there are a few things to pay attention to.
And there are actually three things.
That's why I really wanted to give you this homework,
because I wanted to see if you could pay attention
to three important things to include inside that predict method.
Let's start with the most essential one,
which is the double pair of square brackets.
Right.
It's very important to remember that any input of the predict method
must be a 2D array, whether it is the test set
to predict an in symbol of outcomes for an in
symbol of observations, or whether it is a single prediction.
Well, anything has to be in a double pair of square brackets,
which therefore makes a 2D array.
Basically, the format expected by the break method is always a 2D array.
Okay, so that was the first thing.
Then inside this 2D array
we're going to enter well the different informations here.
And that leads me to mention the second important thing
which you had to pay attention to which is about that variable here.
Geography France.
According to you, do we have to enter,
as you know, this first information here for the geography variable,
do we have to enter the string France or do we have to enter something else?
Well, of course we have to enter something else and that's something else.
Or the values of the dummy variables.
Right. That's the second thing you had to do correctly.
And therefore now we need to check what was the encoding for France.
Well remember if we have a look at our matrix of features
X right above, you know important data preprocessing.
So this is the matrix of features X, the whole matrix containing all the customers.
So in order to know what France corresponds to
you know, in terms of encoding, we just have to have a look
at the first observation before one hot encoding was applied.
Because indeed that first observation corresponds to France.
And so now we just need to compare friends here
to the one hot encoding resulting from what we did here.
And well, we can see that the first row here contains
100 as the values of the dummy variables.
And therefore France was encoded indeed into 100.
So the dummy variable values for the France country
and the geography variable is indeed 100.
And that's exactly what we have to enter here as the first parameter.
So let's do this.
Let's enter 100 and there we go.
Basically that enter is that first value of the first parameter geography.
All right.
And then all the rest is easy.
For the credit score we will enter indeed 600.
Then let's see the gender male.
So remember male was encoded as one and female was encoded as zero.
So here in order to enter the male gender we need to enter one.
All right then 40 years old very simply
we enter 40 then tenure three years.
We enter three here.
Then balance $60,000.
So we enter 60,000.
Then let's see number of products two.
So two.
Then does this customer have a credit card. Yes.
So one because one corresponds to yes and zero corresponds to no.
Is this customer an active member. Yes.
So once again an estimated salary finally $50,000.
So our last parameter value here is $50,000 okay.
Yes. All good.
Excellent.
And now I have a question for you.
Do you think we're done with this prediction.
You know using the predict method with all these informations.
And the answer to that is no.
And that leads me to the third thing you had to pay attention to, which is
the fact that remember to predict
method should be called under observations
onto which the same scaling was applied as in the training.
And since we trained our artificial neural network with the scaled values,
you know,
the scaled values of the features as well, the predict method has to be called
on to these observations to which the same scaling was applied.
And that was the third thing you must have not forgotten, which is the fact
that you have to apply your object here to scale that single observation, right?
That's super important.
Make sure to pay attention to this check if some scaling was applied
during the training. And yes, it is the case here.
And anyway, it is always the case for neural networks.
And therefore in the predict method.
Well we need to scale that single observation.
And the way to do this is of course by calling our subject
yes, this one from which we're going to call be careful
not the fit transform, because the fit transform is used
to remember, get the mean and the standard deviation of the values
in the training set in order to fit your scaler to the training set.
But then for the test set, we only need to call the transform method,
because if we fit it again to scaler,
well that would cause some information leakage.
You know, I explained this in part one data preprocessing.
Check it out again if you need, but remember that
on the test set or on new observations on which you deploy
your model in production, you can only apply the transform method.
And that's what we're going to call here transform them.
There we go.
Which has to take as input
the whole observation in these double pair of square brackets.
All right.
Perfect good good good.
So basically that's it.
Now we can run that cell.
Now we can get our prediction.
But remember that when compiling our.
And to you know enough to miser a loss function and the metrics.
Remember that in the output of our artificial neural network,
we chose a sigmoid activation function, which therefore will return
this prediction in the form of a probability,
meaning that instead of getting the final outcome
1 or 0, whether the customer left or stayed in the bank,
well, we're going to get the probability that this customer leaves the bank.
But you'll see it will be very easy to get the final prediction 1 or 0.
But first let me show you what we get with this prediction.
So we're going to put all this into a print right.
All this into a third pair of parenthesis.
Now let me play that cell.
And we get that, that the predicted probability
that discussed and released to bank is 0.04,
you know, 0.0 38, 0.04.
And therefore it is predicted
that this customer has a very low chance to leave the bank.
And if you don't want the outcome in the form of a probability, well,
the trick to convert this into the final prediction is very easy.
You just need to, as here, you know, just before the last parenthesis,
the larger symbol and then 0.5.
Why is that?
That's because you know all this.
You know and predict the next transform.
Then all the informations of the customers up to here
returns exactly the predicted probability.
And here we choose a threshold of 0.5 to say that if that predicted probability
is larger than 0.5, well, we will consider the final result to be one.
Right.
Because the predicted probability that the outcome is one is larger than 0.5,
meaning that there is more than 50% chance that the predicted outcome is one.
So we'll consider it to be one.
And however, if the predicted probability that the customer leaves
the bank is below 0.5, well, we will consider it to be zero.
Of course, you can choose a different value of the threshold,
especially when you have critical outcome, you know, for a critical decision.
And in that case you can, for example, increase the threshold.
But here let's just pick the default value 0.5.
This totally makes sense for our case study.
And therefore now if we re-execute this cell, we will
of course get the final outcome which will be of course zero or false.
You know, false.
We should not say goodbye to that customer or false.
You know, the customer won't leave the bank.
In other words, it will stay in the bank.
So good, good news for the bank.
But anyway, I just wanted to show you how to use your artificial neural network
to predict the outcome of a single observation, meaning a single customer.
Remember the three things.
First, you have to enter your informations
in a 2D array with a double pair of square brackets.
Then you need to be careful with the dummy variable values here,
instead of entering the string input for that categorical variable.
And then the third thing remember to apply the scaling from your subject
because your artificial neural network was trained with scaled values okay.
Great. All right.
So now let's finish this implementation in a flashlight.
You know the last two cells we have.
By the way I gave you some important notes here so that you can remember.
That's basically what I explained.
But anyway now we have two cells to go.
The first one is, you know, that cell which you know very well where we predict
the test results by displaying next to each other
the vector of predictions widespread and a vector of real results y test.
So we're going to do this the most efficiently by juggling
with our toolkit.
And finally same will make the confusion
matrix to get the final accuracy on the test set.
All right let's do this efficiently.
So we're going to go back into our machine learning.
Is it for there you know to juggle with our toolkit.
So I'm clicking this shortcut here
to get the original base of the folder machine learning A to Z.
Then I'm just going to go, you know in 2.3 classification to take any model.
Let's take, you know, logistic regression for example then Python.
Then I can open this implementation which contains indeed a lot of tools, you
know, which we can do with classification
as we are doing right now with R and N.
And now I'm just going to scroll down and get, you know, the final cells,
which will do exactly what we're about to do now with our an N
first to display next to each other
the vector of predictions and the vector of real results.
So let's get that cell.
Let's base that inside a new code cell right here.
But remember that now since our classifier which is no longer called classifier
but a and n, returns the predictions in the form of probabilities,
well, we just have to do a little something here, which is to
of course convert our predicted probabilities
which are in white bread, into predicted binary outcomes 001.
And the trick to do that is exactly the same as the trick we used here.
It is, of course, to update y pred the following way
by setting it equal to well, the boolean result of whether y pred.
You know, the predicted probability is larger than 0.5,
so that if y pred is between zero and 0.5,
then this new y bread will become zero, because this won't be true.
And if y print is larger than 0.5,
then this will be true, and therefore y bread will be equal to one.
All right.
And we'll get therefore the final predicted binary outcomes 001.
And then the rest is the same.
All this is good.
So let's play that cell and we'll get indeed displayed next to each other.
First on the left a vector of predictions y pred,
and on the right a vector of real results y test.
And now we don't have all the results displayed.
That's because we have a lot of observations.
But you know, we can already see that the predictions look pretty good.
That first customer of the test set. Be careful.
That's all the customers of the test set.
Well, that first customer in reality
stayed in the bank and was predicted indeed to stay in the bank.
However, that second customer actually left.
In reality, the bank but was predicted not to was predicted to stay in the bank.
Then the third customer stayed in reality
in the bank and was predicted to stay in the bank as well.
And then for these three last customers of the test set, well, they in reality
stayed in the bank and were predicted to stay indeed in the bank.
So the results look really good.
But the real way to check this is with our confusion matrix.
And therefore here in our logistic regression tool, you know, code template.
Well, we just have to scroll down a little bit more to find the code
for the confusion matrix and also the computation of the accuracy.
So let's just get this then let's just copy that into a new code cell.
Let's just paste it.
And now are you ready to get the final accuracy
of your very first artificial neural network on the test set right.
Meaning a new customer on which the model wasn't trained?
Well, let's check this out right now.
Let's play that sale. And very good.
We get an accuracy over 86%.
That's really good because it means that out of 100 customers,
well, 86 were predicted correctly to either stay in or leave the bank.
And we can actually see here the details of these predictions.
We had 1520
correct predictions that the customer stays in the bank.
Then we had 203 correct predictions that the customer leaves the bank.
Then we have 75 incorrect predictions that the customer leaves the bank,
and 202 incorrect predictions that the customer stays in the bank.
So anyway, that looks pretty good.
The accuracy of 86% is a very decent accuracy.
Of course, I challenge you to beat that.
You know, to at least get 1 or 2
extra percent of accuracy, but I actually experimented in myself
and it will be pretty hard to beat this, so don't waste too much time on this.
Plus, I rather encourage you to, you know, check out this free course that we have
on Artificial Neural Network for regression,
because I'd really like you to do anything you want with
artificial neural networks, whether it is to do classification
for either a binary outcome or a non binary outcome or regression.
Right.
With this course plus the other free course,
you'll know how to do anything with artificial neural networks.
So that's rather my recommendation.
And now of course I want to say a final congrats Lations.
You completed a very advanced branch of machine learning, so
you can really be proud of your progress.
And if you're not too overwhelmed, well, it's time to reach an even higher level
with convolutional Neural networks.
So as soon as you feel ready for it, well, join us, Kiril
and I, in the next section to tackle convolutional neural networks,
and until then, enjoy machine learning.