Hello and welcome back to the course of Machine Learning.
Today we're talking about the random forest and the intuition behind it.
And specifically we're going to be talking about the random forest
applied to regression trees rather than classification trees.
But the concept is very similar.
And you'll find that this tutorial is very similar
to the one for the random forest on classification trees.
All right, so random forest and ensemble learning.
Ensemble learning.
So Random forest is a version of ensemble learning.
You've got other versions such as gradient boosting and ensemble
learning is when you take multiple algorithms
or the same algorithm multiple times, and you put them together
to make something much more powerful than the original.
And let's see how this works.
So when you pick a random k data points from your training set.
So now we're kind of going to leverage a lot of what we talked about
in the section on regression trees.
So you remember there we had lots of data points.
And then we built a regression tree.
And or we built the decision tree and used that
to, forecast the value.
That would be assigned or the Y value for any, new element that would be
added to our data set as the average in the terminal leaves, basically.
So here what we're doing is we're using the whole data set.
We had.
And we're only picking k data points from their training set.
Then we're going to build a decision tree
associated to these k data points.
Rather than building a decision tree based on everything in your data set,
you just building a decision tree based on those data points
that just like sort of a subset of your data set,
then you choose the number of trees
that you want to build and you repeat steps on and to.
So you just keep building and building and building these trees.
You're building a lot of regression decision trees
and then finally you to use all of them to predict.
So for a new data point, make each one of you and trees
predict the value of y for the data point in question and assign
the new data point the average across all of the predicted Y values.
So basically,
instead of just getting one prediction, you're getting lots of predictions
by default.
Usually these algorithms are set to about 500 trees at least.
So you're getting 500 predictions for the value of y.
And then you're taking the average across those.
And in that way, you're not just
predicting based on one tree predicting or based on the forest of trees.
And that improves, the accuracy of your prediction because it is
you're taking the average of many predictions.
And therefore, even if one is, some forest difference and somehow,
one of the decision trees was built exactly,
perfectly because the weight of those data points were selected.
It just didn't turn out as a perfect tree or a great tree.
Even, if you were using it by itself, you'd get a bad prediction,
because using the average, it is less likely.
So you're going to get a more accurate prediction and more.
And the second thing is that they're more stable algorithms like this,
ensemble algorithms are more stable
because any changes in your data set could really impact one tree.
But to, for them to, really impact a forest of trees, it's much harder.
So therefore ensemble is much more powerful in that way.
And what this reminds me of is the game
that is often played at fairs or parties and things like that, where you have a jar
and inside this jar there's lots and lots of, for instance, jelly beans.
Or it could be marbles, or they could be like a huge net with balloons inside it.
And, we have one in the mall sometimes where
there's lots of balloons inside a net, in the ceiling.
And you need to guess how many balloons there are and who have a
guess is will get like a car can win a car.
And it's like a crazy prize for just guessing number of balloons.
And although this is not an example of specifically a,
random forest or regression on forest method,
it's still an example of an ensemble type of method.
So the best way or one of the ways to beat that game
when you need to guess the number of marbles in a jar,
for instance, is not to actually go and guess,
but it's actually to get a pen and a paper and stand next to the person
that's holding this jar, or that's conducting this event,
and you just stand next to them, and then you wait for other people
to come and guess.
Every time somebody guesses, you just ask them as soon as they're like, guess.
And then walking away, you ask them, hey, because usually they write down
their number and they put it inside a, like an envelope or something.
And then there is the winner is announced later on.
So they don't know whether they guessed right or wrong, but regardless of that,
they're walking away and you just ask them, hey, what number did you guess?
And you just write down their number, and then the next person guesses
and you write their number down, and you write their number,
and you keep writing the numbers down, and you just keep doing that until
you have like a substantial number of, entries, maybe 100
or maybe if it's a very popular contest and people are guessing like crazy, like,
trying to attempt or attempting the guessing,
then you might even get, like a couple of hundred.
Or even if you're very determined,
you might get a thousand of entries over a couple of days.
And then what you do is you just average them out.
Or if you don't want to maybe take the median.
If you don't want outliers like people just guessing random numbers
like 1 or 5 million, so you don't want them to affect you.
You just take the outliers out and then you average out
anyway, you either average it out or you take the median.
And statistically speaking, you have a much higher likelihood
of being closer to the truth if you take the average of people
because people are natural beings and they are kind of the visual perception
will be most likely normally distributed.
And therefore you once you hit the middle of that normal distribution,
you are more likely to be on the money than any one of them.
And that's pretty cool concept that that's an example of an ensemble method
where you're taking instead of just throwing that guess by yourself,
or taking the guess of one individual person, you're averaging out across
multiple guesses, and you're more likely to be the closest one to the truth.
And if the prize is given not just to the person
that gets the spot on, but to the person that guesses closest to the truth,
then you've got yourself a very powerful advantage.
using, data science.
So, if you if you have the patience and determination, then try it out.
Next time you see one of these games and see how you go.
Would love to hear back from you,
because I never have the patience to stand there and just count.
But it is, it is a statistical approach to a challenge like that.
So hopefully you enjoyed today's tutorial.
I look forward to seeing you next time.
Until then, enjoy machine learning.