Hello my friends.
Are you ready for the most essential step of this implementation,
which is at the heart of sentiment analysis,
that is creating the bag of Words model, which we are ready to do now
because all our reviews are properly cleaned.
So we're going to get them into the bag of words model to create,
you know, the sparse matrix which will contain in the rows.
Well, the different reviews, you know, the same reviews as the ones in our corpus
and in the columns, all the different words
taken from all the different reviews, you know, all of them.
And each cell will either get a 0 or 1.
It will get zero.
If the word of the column is not in the review of the row, and it will get a one.
If the word of the column is indeed bar of the words
in the review of the row, all right, so that's what the Sports Matrix is about.
And the process of creating all these columns corresponding
to each of the words taken from all the reviews is called tokenization.
So that's exactly what we'll do in this new cell.
But first let me actually show you what we created.
You know, I just want to show you the corpus.
So actually right here we're going to create a new code cell.
And I'm just going to do a print of the corpus
so that I can show you indeed what we created.
So let's press play here.
And this will show the corpus.
All right.
So this is the first review after the cleaning.
You know, after all this cleaning process in different steps.
Remember I can actually show you the original review here.
The original review was wow with, you know, capital letters,
something to here with the three little dots and then left this place.
And after the cleaning process it became wow, love place indeed.
We removed all this stopwords such as, you know, this.
You know, that's an article that doesn't give any hint on
whether the review is positive or negative.
However, of course
we kept loved because loved means of course that the review is positive.
However, we transformed loved into love.
That's the process of stemming.
So we can simplify all the words by their roots.
And then of course, we kept place because that's of course not a step word.
All right then let's have a look at the second one.
Crust is not good.
All right.
So let's try to guess actually how it was transformed.
So crust was just transformed into crust with a lowercase c.
Then is was probably removed.
Right.
Because it doesn't give any hint on whether the review was positive
or negative.
Not was definitely kept because that's a negative statement.
And good was of course kept. Okay.
So after the transformation, you know, after all the cleaning,
this review must become crust with a lowercase c.
Not good.
Let's check that it is the case.
And oh okay.
So actually they removed the nuts, which is a bit strange
actually because you know, not clearly indicates a negative thing.
You know, a negative review.
We clearly have a difference between crust is good and crust is not good.
So I think we need to do some extra work here in order to not include
the not word from the stopwords.
And I'm going to show you how you can do this.
It's very easy.
So we're going to work again on this code.
I'm actually going to take this here, you know,
Stopwords or the English Stopwords.
I'm going to cut that then
right here in a new line of code, I'm going to paste that.
Then I'm going to create actually a new variable
which I'm going to call oh, underscore stopwords.
Right.
And which will be equal to exactly this.
But then what I'm going to do just below
is to take this again, which was now
created, as, you know, this whole end symbol of all the stopwords.
But we don't want to include not in this top word, because that's
clearly a negative term indicating therefore negative review.
So I'm going to paste that here.
And I'm just going to add here a dot remove.
And in the parenthesis I'm simply going to include in quotes not.
All right.
So that will not include the not word from the stopwords.
And therefore here
instead of
taking the set of the original and symbol of Stopwords,
well, we're now going to take the original and symbol of Stopwords.
Excluding this time the nuts word.
Let's see if it works.
I'm kind of improvising things here, but it might work.
So we actually have to restore the runtime.
So let's do this.
Restore runtime.
Yes we still have our data set. All good.
And now let's see if this works.
So we're going to re-execute the cells.
I can not do a run.
Oh here because the implementation is not over.
But let's import the libraries first.
Now the data set.
And now let's clean the text.
I hope this will work.
Let's play.
All right. This seems to be good. Good.
Now let's remove this output.
This was the previous output right.
And now let's print the corpus.
And let's hope that the second review is now no longer, you know crust good.
But indeed crust not good okay.
So let's press play.
And perfect okay. Good I'm relieved.
You know this was really bad to remove the nut because it's
clearly a negative term indicating a negative review.
All right. So much better now.
And actually you know same for the next one.
Nut tasty texture. Nasty.
That definitely means a negative review.
Let's actually check that right.
Yes. Not tasty.
And whatever zero negative review.
And same for this one.
All right. So good.
We have actually a much better model now.
So we can continue.
And we can mostly create the bag for its model.
All right. So let's do this.
Let's actually scroll down a bit and there we go new code cell.
And now let's proceed with this tokenization to create
a sparse matrix containing all the reviews in different rows and all the words
from all the reviews in the different columns,
where the cells will get a one if the word is in the review, and a zero otherwise.
All right.
So we're going to do this with actually scikit learn.
You know the tokenization process will be done thanks to a class from scikit
learn.
More specifically from a module of scikit learn called feature extraction.
And that class is called count Vectorizer.
All right.
So let's do this.
Let's start from scikit learn.
You know this library very well as k learn
from which we're going to call that feature.
There we go.
Extraction module from which actually you know it's not over.
We're going to get access to the submodule called text text
from which we're going to import that count.
Vectorizer class. Perfect.
I really love Google Colab when it assist me this.
Well okay.
So we have the class.
Now you know, what is the next natural step.
It is to create an instance of this class.
And we're going to call that CV as count Vectorizer which will be created
as, you know, an instance of this count
Vectorizer class perfect, which has to take as input
only one important parameter.
Can you actually guess what it is?
Well, it is actually the maximum size of the sparse matrix.
You know, the maximum number of columns.
Therefore the maximum number of words
you want to include in the columns of the sparse matrix.
And why is this important that because, you know, in our corpus of reviews
now with all the simplifications, well, we actually have still some words
that are not relevant or,
you know, not helpful to predict if a review is positive or negative,
even if they were not part of the stopwords.
And these include, for example, you know, text you,
you know, texture doesn't really help to predict it for review, positive
or negative or, you know, bank, you know, or holiday
or Rick and even Steve, you know, Steve doesn't help at all.
So we still have these words which, even if they're not part of the stopwords,
don't help at all to predict if a review is positive or negative.
And the way to get rid of them is by, you know, entering this parameter
that we're about to enter the way to get rid of them is just to take
actually the most frequent words, you know, the words that appear
most frequently in the reviews, because probably here Steve only appears once.
So if we only take the most frequent words, we won't include
Steve in this sparse matrix, you know, in the tokenization process.
So so that's the trick.
And so now we need to just choose a maximum size of the sparse matrix.
However, we can't really know now how many words
there are in total, you know before we take the most frequent ones.
So what we'll do in fact is we will leave this for now.
You know, we want enter this parameter.
Now we will run this cell once we create the sparse matrix,
which is actually going to be the matrix of features when training
or Naive Bayes model on the training set, it's going to be the matrix of features.
And therefore we will do a print in order to know the total number of columns.
And we will get therefore, the total number of words.
And then we can reduce that total number of words to a lower number
of the most frequent words
in the sparse matrix, so that we can simplify even more the bag of words model.
Okay, so that's what we'll do.
Therefore so far let's not enter anything.
Let's just continue to create that bag of words model.
All right.
And actually speaking of the matrix of features that's exactly our next step.
Here we are ready thanks to discount Vectorizer class
to create the matrix of features which is indeed that sparse matrix.
So we're going to call it x as usual as every
of our previous matrices of features.
So x equals.
And now according to you what is the next step here.
Well you guess that we're going
to create this sparse matrix thanks to our CV object.
So there we go I'm calling CV first from which I'm going to call now
a method which you know very well, which we already called many times.
And that method is the fit transform method.
All right.
Fit transform method which will indeed fit well.
You know, the input of this fit transfer method, which will be
you know, I'll tell you now the corpus, it will fit the corpus to X.
And what does it mean.
It means exactly that
it will take all the words from all the reviews in the corpus.
And then using this transform part of the method, it will put all these
words in different columns. So you see that's very simple.
The fit method will just take all the
words, and the transform method will put all these words into the columns.
That's it.
Nothing more. Okay.
So of course inside this fit transform method we have to input our corpus
of reviews of very cleaned reviews.
And then we just need to add here a two array.
Because actually you know, remember that the matrix of features must be a 2D array.
It has to be a 2D array.
Because then, you know, we will train the naive base model on the training set.
And this expects of course, an array as the format of its input,
you know, the matrix of features.
So you know X will be an array here.
Then it will be split it into the training set
and test it, you know with X train Y trend X and y test.
And then there we go.
We'll have the right array format to train the naive base model
on the training set composed of X train and Y train.
So two array.
Let's not forget the parenthesis.
And now there we go. We're almost done.
Our final step here is to create the dependent variable vector y.
And actually I will let you do this now because you know exactly how to do it
right.
We simply need to take that second column here because that's exactly
the dependent variable vector.
And we don't have anything to do here
because it's already ready with the binary outcome zero one.
And so.
Well, the way to get this is actually very simple.
And I'm actually thinking right now of an even simpler way,
which is to go to our data preprocessing template,
then take this line of code, because, you know, I'm very lazy.
And so I'm copying this and pasting it.
Right.
You know, deleting this and right here.
And that's exactly our dependent variable.
Right.
It is just taking
the last column of our data set, which is the same as the second column.
Right.
You can either put a minus one here or the index one.
But we want to make this a code template if we can.
So let's just keep that okay.
Wow. So good.
We are done with actually the bag of words model.
So now as we said we're going to run this to figure out
the number of columns in the matrix
X meaning the total number of words in that sparse matrix.
So let's play this cell in order to first create x and y.
And then we'll do the necessary to indeed get that total number of columns in x.
And that's exactly what we're ready to do.
Now, you saw that this cell executed properly.
And now the trick to get that number of columns in X,
or you know, that number of words resulting from the tokenization
is just to call the Len function here, which is going to take as input
this matrix of features x, and then only the first row.
Right.
Remember that the first index here
and the pair of square brackets corresponds to the index of the row.
All right.
So this will give us exactly the number of elements
basically in the first row therefore the number of columns of x.
So let's see let's press play.
And we're going to get now that indeed.
Well okay there are 1566 words resulting from the tokenization.
Basically we have 1566 words that were taken from all the reviews.
And for each of the reviews, we have either one in the columns
corresponding to the words that are in the review and zero
to all the other columns corresponding to the words that are not in the review.
All right.
So basically we have 1566 words.
And we can simplify this even more by for example
taking the 1500 most frequent words so that we can, you know, get rid of words
such as Rick, Steve and maybe, you know, holiday or,
or let's say, you know, faux.
I don't know what that means.
Rubber.
You know, this probably appears only once.
And, you know, words like that, words that don't help at all.
Predict if the review is positive or negative.
Okay. So that's the idea.
So let's just take, you know, the 1500 most frequent words.
And therefore to do this we just need to enter max
underscore features parameters.
There we go.
And in order to get only the 1500 most frequent words,
we just need to enter here 1500.
And feel free to
try with other values, like for example, the 1000 most frequent words.
But be careful not to remove too many words.
Okay? All right. So good.
Therefore now we're going to you know rerun that cell.
So let's do this.
Let's press play okay. Good.
And now if we rerun that cell we should get 1500 here.
Perfect.
So now we have a nice bag of words model with only relevant words
you know, that appear at least a certain amount of times
and without all the non relevant words that appear once
like Rick, Steve or that weird forward we saw in one of the reviews.
Okay, good.
And so now, well, we basically did the most difficult part.
We created the Bag of Words model.
And so now I actually have an exercise for you
which you're going to do by yourself first before we do it together.
It is of course, to do all the rest of the different steps here.
And you know how to do them because you basically have everything.
You have the matrix of features and the dependent variable vector Y,
which you can therefore split into a training set and test set,
you know, composed respectively of X train and Y train and x and y test.
Then you're going to use the training set composed of X train and white train
to train the naive base model on the train set.
Then you're going to predict a test result using the test set containing
therefore reviews and their outcomes on which the model wasn't trained.
And finally, you're going to make the confusion matrix and compute the accuracy.
Of course you're going to do this using your machine
learning toolkit containing all the code templates we built so far.
So you totally have the right to do that.
And I actually hope that you're going to do this
because I want you to be, as most efficient as possible.
And therefore that's exactly what we will do.
In the next and final tutorial of this section, I will show you how
to juggle with our diverse toolkit and especially the classification toolkit to.
In a flashlight, split the data set into the training set and test it.
Then train the naive base model on the training set
and predict the test results and making the confusion matrix.
I will show you that I will do this with only copy paste, nothing else.
We won't type any code now we have everything in our diverse toolkit,
but please make it first.
Please do it on your own first
and we will implement the solution together in the next tutorial.
Until then, enjoy machine learning.