Hello my friends.
Welcome to part two of this implementation
where we're going to build together the artificial neural network.
I'm super excited to start.
We will do it in four steps.
And the first step we will initialize to a and as a sequence of layers.
Then we will add the input layer.
And the first hidden layer
composed of a certain number of neurons which we'll choose together.
Then we will add the second hidden layer.
You know, in order to build
indeed a deep learning model as opposed to a shallow learning model.
And then finally we will add the output layer
which will contain what we want to predict.
All right.
Let's do this.
We're going to tackle these four steps in this one same tutorial.
So let's do this starting with this one initializing
the a and then as sequence of layers.
So let's create new code cell.
And now let me explain how we need to proceed.
So the first thing we have to do is obviously to create a variable
that will be nothing else than the artificial neural network itself.
And guess what?
This artificial neural network variable will be created as an object
of a certain class, and that certain class is the sequential class,
which allows us exactly to build an artificial neural network,
but as a sequence of layers, as opposed to a computational graph.
Right.
You saw in the intuition lectures that an artificial neural network
is actually a sequence of layers, you know, starting from the input layer.
And then successively
we have the fully connected layers up to the final output layer.
That's what I mean by a sequence of layers.
And then the other type of neural network is indeed a computational graph,
which are, you know, neurons connected anyway, not in successive layers.
And an example of this is Boltzmann machines.
Right?
Restricted Boltzmann machines
or deep Boltzmann machines or great examples of computational graph.
Of course, they're not covered in this course
because this is really advanced deep learning,
but they are covered in our deep learning.
It is a course if you are really interested
in deep learning and want to go deeper into this branch,
well, I'll be happy to welcome you into the deep learning.
It is it course.
But now let's just get the right
introduction to deep learning with only fully connected neural networks.
And so that's why we will create our new variable, which we're going to call a.
And then and which will be nothing else.
Then the artificial neural network we're going to build.
And we will create that variable as an object
of the sequential class okay.
But of course the sequential class is not taken from nowhere.
It is actually taken from the models module from the Keras library,
which since TensorFlow 2.0 belongs to TensorFlow,
right before we had TensorFlow and Keras separated.
But now since the new TensorFlow, the new version of TensorFlow 2.0,
well, Keras was integrated into TensorFlow
and therefore the way to call the sequential class
here is to go first TensorFlow, which has a shortcut tf,
from which we're going to call the Keras library,
and from which we're going to call the models module.
Perfect and from which we call indeed that sequential class.
And so that all this, you know, creates this a and n variable
which represents our artificial neural network created
as an instance of that sequential class, which initializes
our artificial neural network as a sequence of layers.
And that's our first step. Congratulations.
Now you really made your step into how to build an artificial neural network.
So let's move on to the next step
which is to add the input layer and the first hidden layer.
And that's where we're going to start using the famous dense class
in TensorFlow and even in PyTorch, which is another great library
to build neural networks.
The way to add a fully connected layer into an artificial neural network
at whatever phase you are, you know, whatever the state
of your artificial neural network is, well, is to use the dense class.
And the way we use it is very simply by taking our artificial neural network
object.
You know, that instance of the sequential class
from which we're going to call one of the methods
of the sequential class and that method is Add.
You know, we certainly hope that there is an add method inside a sequential class.
Right?
So that's the method we need right now to add anything we want, whether it is a
hidden layer or a dropout layer, you know, which allows us to prevent overfitting.
Or, you know, we will see with convolutional neural networks that we can
also add, well, a conv 2D layer, which is a convolutional layer.
Well, we can add anything
but right now what we want to add is a simple fully connected layer.
And the way to add this is to enter in these parentheses because at is method.
Well it is to add exactly that fully connected layer
which will be a new object.
You know, it will be a new instance of a new class.
And that new class is of course the dense class.
So the fully connected layer
we're about to build will be created as an object of the dense class.
And therefore now the only thing we have to do is to call that dense class.
And this will create that fully connected layer object.
And at the same time it will automatically add the input layer.
All right. So let's call that dense class.
And once again the dense class is not taken from nowhere.
It belongs to a certain path of libraries.
And of course the root of that library is our TensorFlow library
then from which we're going to call once again the Keras library.
There we go.
From which this time we're not going to call the models module,
but actually this one, you know it's top of the list layers.
That's exactly what we need to add here.
This is the module that contains the different tools.
And by tools I mean classes to add.
Well any layer you want in your artificial neural network.
So layers here.
And speaking of these classes.
Well that's from this layer is module that we're going to call our dense class
which as any class can take several arguments.
And here we have to indeed enter these arguments.
The most important one is this one unit, which corresponds exactly
to the number of neurons, you know, to the number of hidden neurons
you want to have in this first hidden layer.
You know, not in the input layer.
We will automatically have our different features.
You know, in the input layer,
the input neurons will simply be all these features starting from credit scores.
That will be one neuron, then another input neuron, then another one.
You know, up to this one, all these will be the input neurons in the input layer.
But then when
we create that first hidden layer, we will have some hidden neurons inside.
And in this dense function.
Now well we can specify of course how many hidden neurons we want to have.
And now now comes the most frequently asked question in deep learning,
there's a very famous question how do we know how many neurons we want?
Is there a rule of thumb or should we just experiment?
Well, unfortunately, there is no rule of thumb.
It is just based on experimentation.
Or, you know, we call it the work of a novice.
You have to experiment with different hyperparameters.
You know, we call them hyperparameters in the sense
that these are parameters that won't be trained during the training process.
So unfortunately, there is no rule of thumb.
And therefore we just have to pick one number here which wouldn't sound
irrelevant or extravagant, and that number will be six.
I actually tried several numbers and I got more
or less the same accuracy in the end, so it's all good.
You can try different ones if you want, but six is totally fine.
So here inside this dense class, we will enter for our first parameter
which is unit.
Well units equals six.
Perfect. All right.
And now the next parameter.
That is important to note among this huge list of parameters
you can see many of them.
But no worries we will keep the default value for all of them
except this one, which corresponds of course to the activation function.
And you saw in the intuition lectures with Kirill
that the activation function in the hidden layer is of a fully connected
neural network, must be the rectifier activation function.
And therefore that's exactly what we must specify here.
We of course don't want no activation function.
So here we have to specify that we want to rectify the activation function.
And the way to specify this is to enter here
in the activation parameter.
Well the code name for the rectified activation function which is in quotes.
Well ReLU.
That's the code name for the rectifier activation function.
And that is all you have to enter here in order
to make a fully working first fully connected hidden layer.
Congratulations. Now you know how to build.
Actually, you know a shallow neural network and you will know
in a second how to build a deep neural network,
because the way to actually add a second hidden layer here couldn't be more simple.
The only thing that you have to do is just copy this line of code,
and then in a new line of code here
for the second hidden layer, you just need to paste it.
That's what I mean by this add method can add any new layer
at whatever stage of the construction process of your A, and then you're into.
Right? You can use this add method to add anything.
And the weight add a second
hidden layer is just the same as adding the first hidden layer.
Unless of course
you know you want to change
the number of hidden neurons, but you know six hidden neurons in the first hidden
layer and six other ones, and the second hidden layer is just fine.
But once again, feel free to change the hyperparameter values here.
Maybe you will get a better accuracy in the end.
And if that's the case,
well, please share it in the comments or by private message.
Okay.
However, now to add the output layer you have to do something special.
You know, something different than what we did here.
So let's do this together.
Let's create a new code cell.
And well let's actually paste what we just copied before once again here.
But this time we'll have to change two things which correspond
actually to the values of these two parameters.
But first let me explain why all the rest is the same.
Well, that's of course, because, you know, we are adding a new layer and this add
missing can add any layer you want, including of course, the output layer.
So here we're still using the add method to add this final output layer.
And then of course we still want our output layer
to be fully connected to that second hidden layer.
And therefore we're using again here the dense class.
So all good here.
But then these two parameters have to be changed.
And if you follow the intuition lecture
you should know what must be these two changes.
All right. So let's start with this one.
According to you what do we need to replace here.
Well that's of course this value.
But according to you six has to be replaced by which value?
Well, to get the answer we need to have a look at our dependent variable again,
which is this one.
Because remember the output layer contains the dimensions of the output.
You know, the output you want to predict.
And here since we actually want to predict a binary variable
which can take the value 1 or 0, well the dimension is actually one, right.
Because we only need one neuron to get that final prediction 001.
However, if we were doing classification with a non binary dependent variable
like a dependent variable that has three classes, let's say ABC,
well we would actually need three dimensions, you know three output
neurons to once again one hot encode that dependent variable.
Because of course
once again there is no relationship order between the classes A, B and C.
So A for example would have to be encoded by 100.
Then B would have to be encoded by 010.
And c would have to be encoded by 001.
And therefore you need three neurons getting these values zero and one
to encode your three classes A, B and C.
But here, since we actually have a binary variable,
a binary outcome, well, you only need one neuron to encode
these outcomes into 1 or 0, and therefore the value of that unit
parameter here that we have to replace right now is actually one
okay, one output neuron encoding the dependent variable.
And then second change corresponds of course to that activation function.
And more specifically to the value of the activation function.
Well once again remember in the intuition lecture is that
for the activation function of the output layer,
well you don't want to have a rectifier activation function,
but a sigmoid activation function.
Why is that?
It's because having a sigmoid activation function allows to get
not only ultimately the predictions, but even better,
it will give you the probabilities that the binary outcome is one,
so that we will not only get the predictions of whether the customers
choose to leave or not the bank, but we will also have for each customer
the probability that the customer leaves the bank.
And all this thanks to that sigmoid activation function.
So you definitely want that sigmoid activation function
for the output layer only you know all the other layers.
You know the other fully connected
layers will get indeed that rectifier activation function.
And now now I really must say congratulations
because we were actually done
with the creation of this very first artificial neural network.
So you can be proud of yourself.
You just built an artificial brain.
Was that hard? Was that overwhelming?
I don't think so.
And that is the beauty of TensorFlow 2.0.
So I hope you enjoyed this.
I hope you enjoyed building your very first artificial brain.
But that's not over now.
We only have a brain so far, you know, but which is totally stupid
actually because it was not trained yet on the data set.
So we're going to make it smart and we're going to make it smart in part
three training the CNN, in which we will first compile the A
and then with you know, an optimizer and then a loss function,
and then we will train finally our artificial neural network
on the whole training set in a certain number of epochs.
And you will see that the training process will be very exciting to visualize.
I can't wait to show this to you.
Let's tackle part three together in the next tutorial.
And until then, enjoy machine learning.