Hello my friends, and welcome to the final.
Practical activity of this course.
Yes, I must.
Start by saying that.
I'm at the same time excited.
But sad that this. Is the end of the journey.
But no worries, we're going to end on a.
Very, very.
Good note.
And that good note is.
About, of course, XGBoost.
It is a super powerful.
Machine learning. Model, which I absolutely.
Want. You to have. In the.
Toolkit because you will see that it brings exit and results.
In most machine learning problems.
And actually, the thing so cool about this is that it can be both used for.
Regression and classification.
So there we go.
Let's cross.
The finish line together in this final.
Tutorial.
By implementing. The XGBoost. Model.
So that model is given to you in person.
So just before we.
Enter this part, make sure to be on that same page.
I give you the link to this folder right before this tutorial.
In the article. So make sure to connect to it.
And now here we go.
Let's finish this journey together.
By entering part ten. And then the final section.
Of this course, section.
49. On XGBoost.
All right.
And as usual we're going to start with Python which contains two files.
First the data.
And second, the implementation.
And now. You probably also noticed that.
There are many files open now on my machine.
Write all these files.
And these are, you know, the.
Files that we. Implemented.
So quickly and.
Efficiently when doing that model selection demo.
At the end of part three classification.
Right. I gave you. This model selection folder.
With all these classification. Models.
Which were. Experimented on the same.
Data set, which is that data set data dot CSV.
And which I remind consists.
Of predicting if a breast.
Cancer tumor is benign or malignant.
Meaning that each row of this data set corresponds. To.
A patient. You know is a certain patient.
And for each of these patients, we.
Have several features.
From the clump thickness, the uniformity of.
Cell size. The.
Uniformity of cell shape, you know, all these features that are characteristics.
Of a tumor.
And with all these features, we were trying to predict if the.
Tumor is benign or malignant, it is benign.
If we get to class two.
And malignant if.
We get the class four. All right.
And we build and trained.
All of our.
Classification model.
Which are all the ones right here to.
Learn the correlations between.
All these features.
And that dependent variable telling if the tumor is benign or malignant.
And remember.
That we had. Different.
Accuracies for each of these models with the logistic.
Regression model, we had an accuracy of 94.7%.
With the K-nearest neighbors.
We had. An accuracy of 94.7%.
Again, with the.
SVM, we had an accuracy.
Of. 94.1%.
With the kernel SVM, we had a better accuracy actually.
With 95.3. Percent.
With Naive Bayes.
We got a lower accuracy of.
94.1% again.
And with decision tree classification, well.
That. Was the winner.
We got an amazing. Accuracy of.
95.9%. That was.
The number one on the.
Podium, followed by the kernel SVM with that accuracy.
And then unfortunately, Random Forest did not do any.
Good or, you know.
Not better than the others.
Because we got. A 93.5%.
Accuracy with it.
And so. What I want to. Do now.
As you probably have guessed, is to build the XGBoost.
Model and train it on the same data.
Set to see.
If it can take.
The throne holds by the.
Decision tree classification model.
In other words.
To see if it. Can beat.
That accuracy.
Obtained with the. Decision tree.
Classification model.
And well, maybe maybe that's the good note on which.
We will.
End the journey. Of this course.
Are you ready?
So let's do. This.
We're now going to build and train our XGBoost model on the exact same.
Data set.
And see if we can beat.
Basically.
An. Accuracy. Of 95.9%.
And not only.
We will test.
That on a single.
Test set, but also of. Course, now that we.
Learned k fold.
Cross-Validation in the previous section, we.
Will test this. On ten test. Fools so that.
We can get a relevant measure of. The accuracy.
And make sure.
That perhaps.
XGBoost will now become the number one on the podium.
With the. Ultimate machine learning. Throne.
So let's check this out right now.
Let's open this implementation.
With either Google Colaboratory or.
Jupyter Notebook.
I'm going to put. It. Last.
You know, just next.
To all our other classification. Models.
And now the notebook. Just. Opened.
But it is. Still in read only mode. So we're going to create.
A copy right away by clicking Save.
A Copy and Drive.
You can notice that.
All these are copies of the.
Original implementations.
Which are right here.
You know that's the.
Model selection folder and then.
Classification subfolder, which I gave you at the end of part three.
So you can run these codes again if you want.
But we already did that.
Just remember that the number one was indeed decision tree classification
with an accuracy of 95.9%.
And now we're going to.
See if we can beat this with XGBoost.
All right.
So of course no worries.
We won't re-implement all this.
We will quickly get to the core.
Of the implementation. And mostly.
The exciting part, which are the.
Results in this same. Tutorial.
Because indeed. All the cells of.
This implementation are.
Just cells. Taken from our diverse. Toolkit.
Right?
These three first cells are, as you recognize
perfectly, the cells of our data preprocessing template.
Right.
We first import the libraries and we import the data set with.
The exact same code.
I just put the name of the data. Set. Here.
And then we split. The data. Sets into the training set and.
Test set.
So this is all the data preprocessing phase.
Then we train XGBoost on the training set.
And of course I'm going to.
Delete this cell right away because that's the cell we.
Will re-implement together.
And then we.
Have the other tools of our other toolkits like the classification toolkit.
Because indeed this cell makes the confusion matrix
and prints at the same time the accuracy.
I actually already deleted the.
Output to make sure.
We get the full surprise.
By the end of this. Tutorial.
And then of course, as I told you, we are going to apply K-Fold
cross validation right at.
The end to make.
Sure.
That indeed we didn't get lucky on the.
Test set.
You know, if we indeed can beat all.
The other algorithms. So we will not. Only get a.
First measure of the performance thanks to a single.
Test set with this. Cell, and then we'll. Get the.
Ultimate measure of.
The accuracy with that cell.
Are you ready?
Let's start by.
Building and training.
XGBoost. On the training.
Set, which resulted. From the split. Of the data set.
Between the training set and test set.
And first, in order. To get the. Assistance of Google.
Collab, well, let's apply this reflex of uploading the.
Data into the notebook.
So, you know, I just.
Clicked this.
Folder button here, and then a second. Will see the.
Upload button to upload indeed. Our data set.
So let's click it.
And now let's. Go to. Our machine learning.
Is it Codes and Data sets folder.
Because you will.
Still find the data.
Dot CSV. File in this folder in. Parts.
And of course. So let's go into this folder.
Then let's go into part ten then section 49.
XGBoost.
Python.
And here.
Is the data set data dot CSV.
Of many patients with.
Tumors for which we have to predict if.
The tumor is benign or malignant.
So open.
Okay.
And now the data set is indeed uploaded into the notebook.
So there we go.
We can implement that cell and then run the whole code.
All right. So let's create a new code cell.
And there we. Go.
Let's build and train.
XGBoost. On the training set.
So you're going to see that it's.
Going to be super easy.
And in fact we won't even do it.
With scikit learn, but with a library called XGBoost.
In which. We.
Don't even have to install thanks.
To Google Colab, because it is one of.
The many. Packages already.
Installed on Google Colab already pre-installed.
So we have nothing to worry about
and we can just start building and training this model.
But first we're going to import. The class.
With which we're going to build this and this class belongs of.
Course, to this XGBoost library.
So there we go. We're going to start from.
This XGBoost library.
Right. It's built this way just.
Like the name of the model XGBoost. Indeed.
And from this. Library.
We're going to import.
Well, the class. That can build an.
XGBoost classification.
Model and which is. Called ex-KGB.
There we go. Google collab found it.
XGBoost classifier.
All right.
And now the next natural.
Step as usual, is to.
Create an instance of this.
Class which will be exactly the.
Object containing the. XGBoost. Model.
So once again. We're going to call it classifier.
All right.
And we'll. Create this classifier as.
An instance indeed.
Of the XGBoost classifier class.
Perfect.
And now the good news is that we won't have too much.
To worry about with this class, because.
There is not much parameter to tune. Right?
Basically, the default.
Version of the.
XGBoost.
Model will most of the time perform super. Well.
So all good here and now.
Of course.
We finished this. By. Connecting.
This extra boost classifier.
To our training set.
And the way to. Do this is by, of course.
Calling the.
Fit. Method from our.
Classifier object, which.
Will do nothing else than train this.
XGBoost classifier.
On the training set. Right?
So something we did many times.
So there we go.
Let's do it one less time in this whole machine learning journey.
But then I'm sure you. Will do it.
Many times once again in the future, in your future machine learning career.
So let's do this. We call our classifier.
From which we're going to call.
This fit.
Method. Which will.
Train the.
Classifier.
On the training set, which is composed.
Of first the features of the.
Training set represented by X train, and then.
The dependent variable of the.
Training set represented. By Y train.
And these are exactly, of course.
The inputs of the fitness it.
All right.
So in the.
Flashiest of the flashes we built.
And trained.
This XGBoost model.
On the training set.
We only had. To implement these three lines of code.
And then all. The rest. Is something we've already done.
You know this is the confusion matrix.
You have this code.
In all. Of your classification. Templates.
And finally. Of course, we apply the exact.
Same cell that we implemented just.
Before in the previous section to apply k for cross-validation.
So we are ready to run this code, but just before we do it,
I just want to show you the way to.
Build an. ex-KGB regressor. Model.
You know, an extra boost model for regression.
It's actually super simple.
The only thing you need. To change here.
Is just, of. Course, the name of.
The class, which wouldn't be ex-KGB classifier, but ex-KGB.
And as you will see, ex-KGB regressor.
And then, you know, you would just.
Replace classify here by regressor.
And then that's. It.
This way you will build a.
Regression model based on X boost.
All right.
But let's go back to our XGBoost classifier class.
And there we go.
Now we can just save this implementation and do a run all.
To. Find out.
If X boost. Is going to steal the throne of.
The decision tree classification. Model.
For this particular data set.
So quick reminder.
With the decision tree classification model we got the best accuracy.
You know the highest one of 95.9%.
And now let's find out if we can beat.
This with the XGBoost model.
Trained on the.
Exact. Same. Data set.
All right. So basically.
We're ready. Now we're just going to do a run.
All by.
Clicking runtime here. And now.
Are you ready I bet you are.
So let's do this three two.
One. Go.
All right. So all the cells are running now.
And we get. An impressive.
Accuracy.
Of 97.8.
Percent.
When I was telling you that we're going to end this journey on a good note.
Well, I was. Choosing my words.
Very, very carefully indeed.
That's just an amazing accuracy.
You know, there are.
Only three incorrect.
Predictions on such a sensitive problem.
You know, cancer prediction.
Well, this result. Is just amazing.
Here. Indeed, we almost. Get 98% accuracy.
With. These only three incorrect predictions.
That's just amazing.
But now we have. To check.
One last thing, because, you know, maybe we.
Got lucky. On this single. Test. Set.
Maybe that.
Single test, it was more favorable to actually boost.
On the. Other.
Classification models.
Which could. Explain why Extra Boost was number one.
And the only. Way to check this is by actually.
Computing other.
Accuracies. On other test. Sets.
And this. Is exactly what k fold.
Cross-Validation is about.
And that is.
Why this is the last cell of this. Implementation.
And we also have the result for this, which is, as we can see, still.
An amazing.
Accuracy. Of 96.50. 3%.
This is of course.
An. Average accuracy obtained as a result of the average.
Of ten different accuracies measured on ten different test.
Sets. And besides, we have a.
Rather small standard deviation of only.
2%, which is good.
Once again for this sensitive problem of cancer prediction.
So yes, XGBoost.
Is definitely number one here.
And that's why my friends, I'm just. Super happy.
That we end on this good note with this final, powerful.
Tool that you get.
In your machine learning toolkit,
because now you can start your post machine learning journey, you know, for.
Your career. In full confidence.
And about that, that will.
Be my final. Words to you in this course.
I wish.
You tons of great.
Success in your future machine learning projects.
I wish that you.
Are the talented data scientist
who brings the strongest insights and the highest value analysis.
To your. Team and to your clients.
Now you're totally able to do this.
Thanks to your complete and powerful machine learning toolkit.
With these, you're totally able to smash your future.
Machine learning problems.
So once again. I wish you the best. And I look.
Forward to seeing. You in another.
Course for. A new data science journey.
And until then, of course, enjoy machine learning.