Hello and welcome back to the course on Machine Learning.
Today we're going to cover off
some additional comments about the Naive Bayes classifier.
All right so let's have a look.
Today we're going to go through things.
Number one is the question why is it called naive.
Number two is the p of x and how we can potentially drop it sometimes.
I'll show you a quick shortcut.
And number three is what happens
when there's more than two features involved in your data set.
All right so let's get started.
Question why is this algorithm called the Naive Bayes algorithm?
Well, the answer is pretty simple.
The answer is because the Bayes theorem requires
some independence assumptions, and the Bayes theorem is the foundation
of the Naive Bayes machine learning algorithm, and therefore the Naive Bayes.
A machine learning algorithm
also relies on these assumptions, which are oftentimes not correct,
and therefore it's kind of naive to assume that they're going to be correct.
That's the reason why it's called.
Let's go back to our example and see what what that means.
So here we've got age and salary right.
And based on
those we're using the Naive Bayes algorithm to classify our data points
into people who drive to work or people who walk to work.
Well, the Bayes theorem, the way we apply it, actually requires
that age and salary have to be independent, or the variables
that we're working with in this case as salary, have to be independent.
And that is like a fundamental assumption of the Bayes theorem.
And then you can only apply it.
And then you can get those probabilities and so on.
But in our case, if you just think about it
fundamentally, it is probably not the case.
I probably if there is some sort of correlation between age and salary,
because as a person gets older, their experience grows,
or the number of years that they've spent in the workforce grows
and therefore their salary grows.
So it's natural for the salary to grow.
If age, it might not be a super strong correlation, for it's
not it's not for everybody.
But overall there is some sort of correlation.
So there are not absolutely independent variables.
And we're all you can even see that from the chart.
You can just by looking at our chart,
see that there's some sort of correlation between the two variables.
And therefore given that they're not independent,
you can't really apply the Bayes theorem and therefore you can't
apply the base algorithm to machine learning.
And that's why it's called Naive Bayes algorithm, because oftentimes it's applied
even though the variables or the features
are not independent or not completely independent,
and it's still applied and it's still gives good results.
And that's why it's called naive because it's a naive assumption.
All right.
Number £0.02 of x.
So let's have a look at what we performed.
So kind of like rewind and and now analyze
what we did in our steps in the previous tutorial.
So in step two what we did is we took p of x.
So when we were calculating p of x we drew a circle around our new data point.
We removed the data point just so it's not in the way.
And then we shaded the area.
And so what is p of x.
Well p of x is the likelihood that a randomly selected
point from this data set will exhibit the features
similar to the data point that we were about to add.
And as we agreed,
anything in that circle is deemed to be similar to our data point.
Another way to think about it is what happens if I throw in a random
variable or a random data point into this data set right now,
what is the likelihood that it will fall into the circle?
What is the likelihood that it will exhibit features similar
to the features of the point that I'm about to add into the data set.
So basically it falls into that circle.
And so p of x is the number of similar observations or similar observations
means observations similar to the points that we're about to add
divided by the total number of observations, which is student okay.
So it's four four.
We can see that there's four points in this circle right now divided by 30.
And the interesting thing here is that this result is the same both times.
So this is in step two.
In step one where we were calculating for the people who walk to work.
So the probability for people who walk to work it was the same.
So basically p of x doesn't change whether you're calculating it in
in the in step one where we were calculating the probability
that the person with these features, as a person who walks to work,
or if you're calculating in the step two scenario where you're calculating
if that person with these features is a person who drives to work
and therefore it's the same both times.
So what does that mean? Well, let's have a look at the formula.
So you can see that the formula in step one it was p of the probability
of person who exhibits features x that he walks.
He or she works at walks to work.
And that was a formula.
And CP of X is at the bottom.
And for step two it was a probability of a person
who exhibits features X being a person who drives to work.
And as you can see, p of x is at the bottom here.
So what did we do in step three. Right.
So let's move on to step three from here.
Step three we compared the two.
So now if we take these two formulas these right sides of the formulas
and put them into the comparison
like that, you will see that at the bottom the denominator is the same.
Now that we know that the nominator is not zero
and it's actually greater than zero, probabilities is never less than zero.
And we know it's not zero. So we can just get rid of it. Right.
We can multiply both sides by p of x and therefore the sign won't change.
And also we'll get rid of the denominator.
And that way we won't actually have to perform that calculation.
So that's one less calculation to perform.
So you can just compare the top parts of these calculations.
And so that is what is done a lot of the time.
So if you've done other courses on machine learning or if you've read some
maybe articles on machine learning, you will find that this is oftentimes
a case.
And also sometimes it is not mentioned that this is happening.
So sometimes it's assumed that or it can be assumed that you know what's going on.
So just be careful that look out for that.
And it's a it's a valid approach. As we discussed.
It's totally valid to do it that way.
But that is only if you're comparing the two, right.
If you're only comparing the two, then you can do that
if you actually want to calculate the value.
So we said 75%, 25%.
If you want to calculate the value, you can't do that because it'll be to
different realities because you're supposed to divide by a certain value.
It's not the actual value.
And moreover, if you want to like comparing is okay,
but calculating the actual value and maybe some performing some operations
or, you know, you calculate the value from this, you want to compare the value
from this scenario to a value from another problem, right?
Like a value
from a different kind of a scenario where p of x will be different, right?
Not from this particular example that you're working with.
You want to compare the probability of the person being a person
who walks to work in this example, to the probability of that person
being a person who walks to work from a different example, right
where p of x will be different.
If you want to compare across like that, that will also not work,
because if p of x is different, so be careful of that.
It's always the safer and safer way is always just to perform the full
calculation.
But if you're doing it often, or if you just want to save some time
or if you just maybe
reading other literature, then it's it's also good to know about this approach
where the denominator can be dropped when just comparing between the two.
All right.
So that was another point of like kind of a hint there or maybe a shortcut.
And so the last point for today, what happens when we have more than two
classes.
So as you remember in this scenario we only had two classes, the red and green
or the people who walk to work and the people who drive to work.
What happens when you have more classes?
How is the challenge different?
Well, when we have only two classes, we compared, as you remember,
we compared the people.
The probability that a person who exhibits features X walks to work.
So basically that new data point that we added,
what is the probability that it's a person who walks to work
versus what's the probability that that new data point
is a person who drives to work and it turned out that we were comparing
75% versus 25%, and 75% was greater than 25%.
And therefore the probability of the person of that person
being a person
who walks to work was greater than that person being a person who drives to work,
and therefore we decide to classify them as a person who walks to work.
And it's very straightforward.
And moreover, you'll find that every time when you just have two classes,
it will always add up to one.
So we didn't even have to calculate the second one.
We could have just stopped at this one,
because if this is 75%, this one is automatically 25%.
It's always going to be like that.
If you have two classes.
The way it changes, if you have three classes, it is it gets more interesting.
Right? So you've calculated one and there's this.
Then there's still two more left.
So if you're working with one with just two classes,
as soon as you calculate one you're done.
You can decide right away if it's greater than 50%.
Then you assign that class.
If it's less than 50%, yes, another class.
If it's equal to 50%, then you've got like a tie.
Whereas if you have 2 or 3 or more classes, then just calculating
the one won't be enough, because then you have two other ones
and you would still have to calculate at least another one.
So just means that it's more of an interesting
selection problem when you have more classes.
That's pretty much the main thing that, changes when you have more classes.
And there we go.
That's all for today.
I hope you enjoyed these extra tips on the naive Bayes classifier,
and I look forward to see you next time.
Until then, enjoy machine learning.