Hello and welcome back to the course on Machine Learning.
In today's tutorial, I'm going to show you my solution
of the challenge that I threw at you Lausanne.
So in the previous tutorial, we had the challenge
to calculate the posterior probability of somebody being a person
that drives to work a given that they are placed in the position
where we are placing the new observation for our data set.
So basically, what is the likelihood
of that new observation representing a person who drives to work?
And the way we need to calculate is using the Bayes theorem,
which is right in front of us.
As I know we're going to walk through it.
First we're going to calculate the prior probability.
Then we're going to calculate the marginal likelihood.
And then we're going to calculate the likelihood.
And you can compare along the way if you got the same result.
So let's go through it.
So there's our data set.
And now let's move it to the left to make some space.
And the first thing we're going to calculate is a prior probability.
The prior probability.
There's kind of two ways to think about it.
So the first way to think about it is if I just randomly select
a person from our data set right now.
So not including the gray dot,
not including the new data point, if I randomly select a person from there,
what is the likelihood of them being a person who drives to work right?
So that will be just the number of the answer to
that is the number of green dots or the total number of dots.
The other way to think about it is if I just randomly throw
a new data point into our data set, right?
Just randomly and without knowing anything about their age or salary,
then just knowing all of this prior information that we have the green dots
and the red dots, what is the likelihood of that person that we're adding?
What is their likelihood to be a person who drives to work?
Again, we don't have is very simple because we don't have any other choice
but to calculate the probability and assign the probability
just based on what we know.
And that is just to take the green dots,
add the 20 green dots divided by the total number of dots.
Here, and assign that as a probability.
So we don't have any other choice.
And therefore that's what it's calculated as.
So the probability of somebody being a person who drives to work
or a random dot selected out of our existing dots,
being a person who drives to work is a number of drivers,
which is 20 divided by total duration, which is 30.
So we go 20 over three.
So that was the prior probability we've done that.
Next one is a marginal likelihood.
So let's go ahead and calculate that.
And you'll find that the marginal likelihood
is actually going to be exactly the same as in the previous tutorial.
And we'll talk about this separately.
So again we're going to draw the circle around our observation.
We're to remove observations. So it's not in the way.
Then we're going to shade in this area.
And so now the marginal likelihood is the question what is the likelihood of
if I just pick a random dot from our data set just randomly.
What is the likelihood that I'm going to pick one out of here.
So what the reason why we put X here is, is because what is the likelihood of me
picking a observation that exhibits features
similar to the features of that point that we are adding to our dataset?
So the point we're adding to the data set we've just removed, it is over there.
And we've agreed that any dot inside the circle is deemed to be similar
to that dot, or in other words, deemed to be exhibiting similar features
so similar age and similar salary to that dot.
And therefore that's what we're calculating.
So P of X is very simple.
We just need to calculate the number of similar observations,
the number of observations that actually fall in here, which is four
divided by the total number of observations which is 30.
So that will give us the likelihood of a new dot falling here,
or the likelihood of if we just pick out a random dot from our dataset right now,
then the likelihood of it being one of these is four over 30.
There we go for all 30.
All right.
So that is our marginal likelihood done.
And now we're going to move on just the likelihood.
And this time is going to be the likelihood of somebody exhibiting
the features of X or being similar to the datapoint that we're adding,
given that we're only looking at people who are driving to work.
So let's have a look at that.
Here's our dataset.
And again we're going to draw the circle around our data point.
Take it out and then add that shading.
So now the question is given that we're only dealing
with people who drive to work, what is the likelihood that if we pick
one of them, that that person will be exhibiting features similar to X?
So because we're only dealing with the people who drive to work,
we can forget about the red dots.
There you go.
This shaded there faded out and now we're only dealing with the green dots.
So the question is, given that we're selecting a random point
out of all of the people that drive.
So this vertical bar drives means given that a person drives to work.
So we're looking at a random point out of these.
What is the likelihood that they will exhibit features similar to X,
which we agreed is the same as they fall inside this circle.
And the likelihood of that is one over the total number of green dots.
So there we go p of x, given that they drive to work,
is the number of similar observations among those who walk.
So inside of here is one
similar meaning similar to our new points that we're adding.
And then divided by the total number of workers.
And that's not 20.
So one over 20 that is our likelihood.
There we go.
So now we can plug these into the formula.
Calculate the posterior probability.
So it's going to be one over 20 times 20 over 30 divided by four with 30.
And it's 0.25.
So 25%.
So there we go.
That was step two of our Naive
Bayes algorithm or Naive Bayes classifier.
Hopefully you were able to follow along.
And also that I hope that you had chance
to perform that exercise on your own and you got a similar result.
And that's it for today.
I look forward to seeing you next time.
And until then, enjoy machine learning.