Hello, my friends.
All right, let's do this.
Let's start this. Implementation.
So the first step is, of.
Course, to implement the data. Preprocessing phase.
And it will be slightly.
Different than what we.
Did before because we will only use actually from.
Our data preprocessing template.
This code cell.
Where we import, you know, the.
Widely used libraries.
Like numpy, matplotlib and vendors.
And then only.
This. Line of code, you know, to.
Load the data set. And that's it.
Then we won't.
Create a.
Matrix of features or dependent variable vector because here we're doing a.
Totally different thing the association rule learning.
And we don't have to. Split the.
Data set into a training set and test.
It, because we will directly learn all the rules through the whole data set.
Okay.
So right now we just want.
This first, you know.
To import the main libraries.
And then we'll. Get that.
Line of code to import the.
Data set.
So here I'm creating a new code cell.
I'm importing this.
And then we can load the data set.
However there is something important you should know about.
This a priori. Implementation.
It is the fact that for the first. Time we won't.
Use scikit learn.
Unfortunately, the cyclone library doesn't.
Include some classes or functions on a.
Priory.
Basically, it doesn't include the a priori model.
Therefore, we're not going to.
Use scikit learn to.
Train the model.
We will actually use another.
Library which is called a priory. You know.
priory.py is actually a.
Python implementation containing all. The.
Algorithm of the Priory model.
And that's.
What we.
Will get, you know, and use to train our.
Priory. Model here on the.
Whole data set, but exceptionally, because.
You know. Google.
Colab contains most of.
The libraries and packages pre-installed.
But here, exceptionally, Google Colab.
Doesn't include. That.
Appearing module.
So we will actually have to install it.
And this. Is very good that you see this because you.
Will encounter this situation sometimes, you know, rarely but sometimes.
And so you need to know how to install.
A certain package or a certain library.
From the web, you know, from online.
Because what we'll do now is we will enter a command, you know, Pip
command, which.
Will download first the binary file.
From the internet, you know, through link.
And then we'll install it inside. This.
Particular notebook.
All right.
Let me to. Show you let's do this.
So we're. Actually going to implement this.
Right before you. Know importing the main. Libraries.
Usually the packages are installed as a first step.
And to do this we need to start with an exclamation mark.
Then Pip you know that's the pip command that.
Allows us to. Install. Packages.
Then very simply.
Well speaking.
Of installing packages, the next thing to enter here.
Is install.
And then you simply enter the name of.
That library or module.
That you want to install.
And in our case. As we said it is a fire.
Be careful.
It is not a.
Priory, it is a fire like that AP or AI okay.
So exclamation mark pip, install a binary.
All right. And now let me show. You what it does.
As I said it's. Going to download it. First.
And then it will install it inside a.
Notebook. So here it is.
Finding it on the web.
And very soon we will see in the.
Output the download.
All right there we go. The download is starting now.
collecting it by.
Downloading.
It. From the link and then installing it and then, you know.
Successfully. Installed. A binary.
See it just takes a few seconds.
And this is the version of this.
Binary package. 1.1.2.
Maybe you'll get a different one, you know, if you.
Take this course way after I recorded this tutorial.
All right. So good.
Now you know how to install. A package.
Within Google. Colab.
But don't worry, most of the.
Packages and including the deep learning one like TensorFlow.
Are already pre-installed.
Okay, good.
So now well let's import the libraries now.
As we're here.
And now let's implement the. Data preprocessing phase.
So let's create a new code cell.
Then as.
We said we will just get that first.
Line of importing the data sets in our data.
Preprocessing. Template.
And we're going to paste. That.
Right here in our copy.
And that's it.
That's all we'll get from the template.
And now we will adapt and make it 100% customized.
To the primary model.
All right.
So first let's just replace.
The name. Of the data set here by.
The right name which is remember. Market.
Underscore.
Basket underscore. Optimization
okay.
And now that. We did this let's.
Think of the right thing to do.
Which is to upload the data set here.
Oh good.
So let's click upload.
All right then make sure to fine.
You know that machine. Learning is that codes. And data sets.
Folder on your. Machine. And if you haven't downloaded. Yet.
You can.
Download it right before this tutorial at the bottom of the article.
All right.
So let's go inside and let's go to part five Association.
Rule learning.
Then Priory then. Python. And there we. Go.
Let's select market basket optimization.
Let's click open. Click okay.
And it. Will load it.
And there we go. We have it.
Don't try to open it because it is actually very big.
And Google Colab won't allow you.
But you know if you want to have a look at it, you can go back to the folder.
And, you know.
Just double click it here, right.
And you can see it very well and see all the transaction okay.
So we'll actually leave it like that in case we want to see it.
And now. Back to.
Our implementation.
Let's see what we. Have to do. Next.
So we already loaded the data set as.
A pandas dataframe.
However we have to be careful with something.
Look at the data set.
Again and notice something.
Notice that you know on our. Previous data set.
The first row here.
Of this index one here contained actually.
Before. The names of the. Columns, right.
For example, if we take our social network ads data.
Set in. Part three.
Classification, we'll remember this.
First row here had first the age.
The name of the first column.
And then the estimated salary, which was the name of the second column,
and then the name of the dependent variable purchased.
And here in this. Particular data set.
Well, you know, we don't have names of columns because.
Each of these elements.
Here correspond to just.
Different products. Okay.
So there was no need to name any of these columns.
And therefore here we need to add something in our.
Read underscore CSV. Function to.
Tell that the first row.
Doesn't contain the names of the column, because if we don't say this.
Well, actually it. Will not.
Take that first. Transaction.
Because it will. Think.
That this first row. Are just names of. The columns.
We have to specify. This and the.
Way to specify this. And it's good that.
You know, this trick in pandas.
Is just.
Add another parameter in this read underscore.
Csb. Function.
Which is header, and we set it equal.
To num. Right.
So that we can specify that indeed there are no headers.
Meaning no column names. Okay.
That's what it means.
So now you know this. And therefore. It will.
Take into account, you know, it will.
Take the first row of this market basket.
Optimization data set containing all the transactions.
And we'll do this. It will take the first transaction.
All right. Good. So that was the first step.
But then we actually. Have another. Big step.
And that.
Has to do with the fact that when we will train the a.
Priori model on the data.
Set, well we will use a.
Certain function that. Is actually called a priory.
And of course that function.
Will take. As input. The data set.
But the thing is.
That it expects this data set to have a.
Certain format, and that format is unfortunately not append this data frame.
And therefore we have to recreate the data set, you know, from this original.
Pandas dataframe so that it can have.
This format expected by the a priori.
Function.
Which will. Train the Priory model on the whole.
Data set.
And so now the question is what is this format?
Well, that format is simply a list of transactions.
You know, instead of having the data set.
As a. Pandas dataframe, we want to.
Have the data set as a list. Of transactions.
You know, the transactions. Listed.
One by one.
With the different products purchased by the customers.
And that's exactly what we have to do now.
We have to. Create that list.
And the first. Step to create that list is.
Actually to initialize.
It as. An empty. List. Because then in order.
To create that list, we will have to populate it with.
The different transactions of. The data set.
And the way we.
Will do this is of course with a for loop.
You know, for loop.
Iterating all the 7500 transactions of.
The data set so as to populate it. Okay.
So let's do this.
First let's. Initialize. That list.
We will call it. Transactions and we will.
Initialize it as. An empty list.
All right.
So so far it. Is just an empty list. And now.
We will start for loop to populate this list of.
Transactions with. All.
The transactions contained in that pandas DataFrame.
Data set.
Okay.
So as we make that loop you.
Will totally understand.
What we we're doing.
So for.
Then. Classic iterated variable I.
You know, which.
Will take all the values from zero to.
7500.
But remember that the upper bound of a range is excluded.
So we will actually have to go up. To 7501.
Therefore here this iterated variable will go.
In the range from.
Zero to.
7501. All right.
They're actually 7501 transactions
because we start from zero and not 7500.
Right. We can check it out very quickly.
You know here we start from one.
And when we scroll down we go down to let's see.
Yes 7501. So that's the exact.
Number of transactions.
All right.
So all good.
For the range.
Now don't forget the column.
And now we begin the for loop.
And well you know building this list of transactions is very easy.
We will just use the append.
Function which means add and which will simply.
You know, add the. Different transactions of.
The dataset. One by one.
All right. It's a very classic way to build a list.
You know, you use.
The append function to add your different elements one by one.
So let me.
Show this to you very simply.
What we need.
To do here is take.
Our transactions. List, okay.
From which we're going to.
Call this.
Append function, which. Is a prebuilt.
Function of a Python list.
You know, Python has.
All these pre-built.
Functions.
And inside.
Well we're. Going to add the transaction.
But we have to add. It in.
A pair of square brackets.
Because it's will contain, you know,
all the different elements, you know, all the different.
Products purchased by.
The underlying customer in the transaction.
And this transaction must.
Be created as a list of. Products.
And that's why.
We have these new pair of square brackets.
To make that transaction a list. Of products.
So in the end we're.
Actually creating a list of lists.
You know, each transaction in the big.
List. Of transactions is. Actually a list. All right.
And so now there's a new little trick.
But which is good. That, you know.
It is a single row for loop.
Because now actually.
We need to do a second for loop to.
You know.
Get all the products of each transaction.
So let me scroll back up.
You know, the. First for loop will go.
From this transaction to this one to this one, to this one,
down to the last one at the bottom 7501.
But then we need to.
Do a second for loop, which will loop.
Over the different products in each transaction.
So you know it will. Add this product.
Then this one, then this one, then this one.
And since the maximum number of products in one basket.
Is actually 20. You know.
I put this first.
Transaction at the. Top so that we.
Can have the maximum. Size of the basket.
Which is 20.
And therefore what we'll do now is the second for.
Loop that will iterate from zero to. 20.
It will start from zero here.
You know. That's the index. Of the first. Column.
And then it will iterate to this one, then to this one, then to this one
up to the final index, which is index 19.
Actually, because it starts from zero there are 20 columns.
So it goes. Up to the index 19.
Okay. So that's the.
Second for loop.
And that's what.
We'll do right away.
Inside this.
Append function to.
Add all these.
Different elements in each transaction.
And for the transactions that don't have 20 elements, well that's totally fine.
We will still iterate up to the.
End, you know, up to the index 19.
But we will just populate the list.
With nones, you know, Nans values which mean empty,
so that our model will understand that the transaction here only has.
Three products. Okay.
So we can totally iterate up to 20.
And therefore let's start this second.
For loop. Here.
For now we need to take another.
Iterated variable, which we'll call naturally j for.
J in the range from.
Zero to be careful.
Not 19, but 20.
Because the. Upper bound is once again. Excluded.
So from 0 to 20 then what we will do.
And that's.
The syntax in a single.
Row for loop we have to do what we want to do with this.
For loop
at the beginning, you know, before the for and what we want to do is just get that.
Product in the transaction.
Well from the index. Zero to.
The index 19 and to. Access this product.
Well we will of course use our data set.
And play with the right indexes.
To get the right product.
All right.
So here we need to call first the data set okay.
And now let's add some. Brackets.
To add you know the index. Of the row.
And the index of the column that contains that product.
We want to include in this transaction.
So first according to you.
What will be the index of the row.
Well that will be of course I because I
iterate through all the rows of the data set.
And therefore now we're dealing actually.
With that particular.
Transaction of index I. Which were.
Appending into that transactions.
List, and therefore the row of the data set we are at.
Right now.
Is exactly I.
Okay, I will start from zero.
It will first get that transaction.
Then it will be equal to one.
So it will get.
That transaction and then this one and this one etc..
All right.
And so now we're dealing with a particular transaction of a particular row.
And that's.
Exactly the index of the.
Data set we need to input here.
And now for the column which index you think it's going to be.
Well that will be of course that. Index, you know, which.
Iterates through all the.
Columns of. Actually.
The transaction. Right.
J will go from. Zero.
To 1 to 2. To 3 to 4 up. To 19.
And therefore. Here.
The index will one for a column is of.
Course J.
All right.
So very good.
But now we need to add one final thing.
You know unfortunately we can.
Not directly access the cell of row.
And column J in the. Data set. In order to.
Access the cell we just need to add here dot values.
Because that's the part of that data set structure.
You know, this is an advanced. Structure by. Pandas.
That allow us to get access to the cells. All right.
So good. That, you know. This.
And now we're almost ready.
We almost. Have everything.
The only thing that's left
has again, to do with, you know, a certain expectation of the future.
A primary function which we'll use to train our primary model.
It is the fact that all the.
Elements in the list.
Must be strings.
They must. Be strings. Otherwise.
So a primary model won't be able to learn the rules.
And to make.
Sure that these, you know, values we populate inside.
Each of our transactions in our list.
Well. In order to make sure that these are strings, we can.
Force it this way by.
Putting that into. The.
String function.
Str, which will take us input exactly these.
Elements, meaning this.
Product. Of the transactions.
All right.
So now we make sure.
That the products are.
Strings you know in actually. Quotes.
And this.
Will give exactly what the primary model is expecting as.
The format of its inputs.
Good. Perfect.
So now we're done with data preprocessing.
As I told you, it's a. Bit different than before.
But now you know.
That for the primary model, you have to just create a.
List of transactions and make sure. That all.
The elements in each of your transactions are.
Strings.
Perfect.
So now let's execute this cell because we're ready.
Let's hope we didn't make any mistake.
Play running this cell and all.
Good. Perfect.
It just ran successfully.
Now we have. This list of.
Transactions containing all the same transactions as in.
This data set.
But in the format. Of a list. Good.
So now we're. Going to take a little break.
Because then comes the. Real.
Big. Important step.
Training the a.
Priori model on the data set.
And to do this we'll call this a primary function
that will take as input exactly this list.
Of transactions that is now. Correctly.
Populated in the right format to train the primary model on the data. Set.
So take a little break and as soon as you're ready.
Let's implement that. Next step together.
And until then, enjoy machine learning.