Okay, my friends.
So now comes the exciting step.
The step where we visualize the results of Thompson
sampling, and mostly the step where we're going to find out
if Thompson's sampling can beat the UCB algorithm.
And about that.
Let me remind what we got with the UCB algorithm.
Remember, that was, you know, in the original implementation.
Remember that with 10,000 rounds.
Well, the UCB was perfectly able to find the best ad,
you know, the ad with the highest conversion rate,
which is of course the ad of index four or, you know, ad number five.
Then remember we tried with lower values of n,
we first tried with 5000, and indeed still with 5000, UCB
was perfectly able to identify the best ad with the highest conversion rate.
Then we even tried n equals 1000, and with n 1000.
Still the UCB was perfectly able to find that best ad.
However, in our copy, you know, in our implementation of UCB,
well we tried n equals 500.
And unfortunately, the UCB algorithm was not able to figure out the best ad.
You know, that ad of index
four with the highest conversion rate in 500 rounds, right.
It identified ad number seven.
So what we really want to see now, you know, when visualizing
the results of Thompson sampling is if Thompson sampling
can not only figure out that same best ad, you know, with the highest conversion
rate in 10,000 rounds, first, we will start with 10,000 rounds.
But mostly what we want to see is if it is able to figure out that best ad
in less than 500 rounds or, you know, 500 rounds, because if that's the case,
then that means Thompson sampling will beat the UCB algorithm.
So can't wait to try.
I promise that I actually have no idea because, you know, this is a new recording
that I'm doing,
and when I made these implementations, I only visualized the results with 10,000.
So I'm discovering the results with you.
And that's why I'm just super excited
to not only show this to you, but also to myself.
So we're all in the same row of seats, you know, watching the same show.
Okay, so let's do this. Enough talking.
Let's click this folder here to, you know, upload the data set into the notebook.
Let's not forget to do this because we actually have not run any cell so far.
So right now it is connecting to a runtime to enable file browsing.
And in a second we should see hopefully the upload button.
It is not coming.
But you know sometimes it takes time.
There we go.
All right so the upload button let's click it.
And now in your machine please find the machine learning.
It is that folder wherever you put it in which you had to download
at the beginning of each section, including this one.
Then once you find it let's go inside.
Let's go to part six now Reinforcement Learning
and section 33 Thompson Sampling.
And of course Python and ad CTR optimization.
Let's click open. Let's click okay.
And now we have the data set.
So I suggest that we do a run URL so that we can,
you know, quickly experiment with the different numbers of rounds.
And we're going to start with 10,000 of course just to make sure that Thompson
sampling works correctly. So let's do this.
Let's click run time here and then run.
And now all the cells will be running including this one.
Everything seems to be all good.
And whoa whoa whoa.
That's actually even more incredible than the UCB algorithm.
You know, it seems that this ad of index four
was actually very quickly identified as the best.
So I'm actually very confident about that question whether or not
the Thompson sampling will beat UCB in the sense that it will be able
to identify this ad of index four as the best one in less than 500 rounds.
Right?
Clearly, all these other ads here are smashed with respect to this one, right.
If we have a look at UCB again, see the other ads are
way more selected here than you know this one.
Okay. So I'm really really confident about this.
So I think we can directly not write 500 but you know 1000.
So I'm going to remove a zero here.
All right.
And then we're just going to do another run time
and then restart this time and run all so that we can restart everything
and rerun all the cells.
So yes here and now all the cells will be running again with this 1000 rounds now.
And let's see the new result.
And coming in a second.
There we go. Okay.
So now with 1000 rounds of course the other ads are a bit higher.
You know, I mean the bars of these other ads are a bit higher because of course
now we are on a different scale.
We're only with 1000 rounds, but still with 1000 rounds.
Well, that ad of index four was way more selective than the others.
And now time for the ultimate truth.
Will Thompson sampling beat the UCB algorithm, meaning
will it be able to identify the best
ad with the highest conversion rate in 500 rounds?
Well, that's what we're about to figure out right away.
So let's replace that 1000 value
here by 500.
Then let's click run Time again and then restart and run.
Oh are you ready?
We're about to reveal the ultimate truth.
Let's do this.
Yes. And now now all the cells will be running again.
And let's see if we can figure out the best
ad in 500 rounds up and congratulate.
Since Thompson sampling, it was indeed totally able to figure out that ad
of index four, with the highest conversion rate in as little as 500 rounds.
So that was expected.
You know, I actually expect this because I knew
from my educational background in machine learning, you know, reinforcement
learning with one of the topics I had, it's school in my masters of Machine
Learning that indeed Thompson sampling is more powerful
than UCB in most situations, and that's what we clearly see here.
Right?
Even in 500 rounds, that ad of index four was way more selected than the others.
You know, almost twice as much as this first AD and twice as much as this one.
So clearly Thompson sampling did an amazing job at quickly
identifying the best ad with the highest conversion rate.
So to the question should I try?
Should I choose UCB or Thompson sampling?
Well, of course
you can try the two because, you know, it just takes a few seconds to run the two.
But if you have a doubt, well, I would suggest to go with Thompson
sampling because indeed it is more powerful.
You know, it is faster to identify the elements with the highest rate.
So there we go.
I was really happy to implement these two models
with you, UCB and Thompson sampling, and I'm really glad that we enjoyed
at the same time, the final results on our notebook.
And now we're going to move on to another branch of machine learning,
which has become super popular over the recent years.
I'm talking about natural language processing,
and that is the branch of machine learning.
That, of course, allows us to build, you know, chat bots or machine translations.
So that's not what we will do in this part seven,
because we will cover the basics with sentiment analysis.
But still, it is great that you're being introduced to this branch,
because if you want to pursue a career in NLP, well,
this new chapter will definitely help you with the right basics.
So I can't wait to see you in this next part.
And until then, enjoy reinforcement learning and enjoy machine learning.