Hello and welcome back to the course on Machine Learning.
In the previous tutorial we talked about the hierarchical clustering,
the intuition behind it, and how it works.
But at the same time, we didn't quite understand
what the whole purpose of C was and what's what the benefit of of it was.
Yes, we went from a huge amount of
clusters where every single point or data element was considered a cluster,
and then to one big cluster, but as a result, now we have one huge cluster.
What's what's the point of all of it?
How do we get to the result?
We want the actual clustering.
So like in K-means for instance, we would have 2 or 3 clusters.
How do we get to that?
Right. Number of clusters.
So this is where the dendrogram come in.
And they will help us understand everything.
So let's get straight into it.
So here I've got a chart on the left which contains, six points.
And on the right if we go to another chart we're going to use this chart
to create a dendrogram.
Now, I know it might sound a bit
confusing at first, especially because we haven't talked about the grams,
but through creating one we will learn what they are.
So first off, just to make things a bit,
more legible, I'm going to add the points at the bottom
so that they're a bit bigger so we can see them better.
And so there they there they are.
The points just listed on the bottom, on the vertical axis
we've got Euclidean distances.
And it'll all make sense just now. So we're going to now
go through the algorithm and slowly create those clusters.
So to start off with every single point is an individual cluster.
Right.
So every single one of these points is an individual cluster.
Next what we're going to do is we're going to find the two closest points
which are these two.
And put them into one cluster.
So that's our step two in our algorithm.
So there we go.
That's the two closest points.
And now we're putting them into one cluster.
Now what do we want to do on this diagram here on the dendrogram is we want to
somehow signify that these were indeed the two closest points.
Because the dendrogram is kind of like the memory of the algorithm
is going to remember every single step that we were performing.
So there they are, those two points P2 and P3.
How do we signify that we've just connected them and that they were
the closest?
Well, to connect them we would use like a horizontal line.
But then where would we put it.
Would you put at the very bottom. Would we put a bit higher.
What's going to determine the distance.
How high are we going to place this line.
So this line is actually placed this height actually has a meaning.
This height is the Euclidean distance between them.
And it also represents the computed dissimilarity between the two points.
So the two clusters and what that means is the further away two points are.
So for instance P2 is that far away from P3.
And this could be a variable.
For instance, could be the age of a person.
Right.
And this variable could be, for instance, the salary of a person.
Right.
Or this variable could be how long a person has been of the company.
And this for this variable could be the salary of the same person.
So something like that.
So basically we can see that P2 and P3 are they have that distance between them.
Well whereas P2 and before have a greater distance between them.
And that means that these two points P2 and P3, they have a certain dissimilarity
which is measured by the distance between them.
So the distance represents the dissimilarity between
the two points and P2 and before also have a dissimilarity.
And it's greater because you can see that the distance is greater.
So, let's say if this was age and this was salary, these two points,
even though they're not identical, they are less dissimilar
in terms of age and salary than P2 and P4.
And again, these variables are just arbitrary.
I'm just calling on arbitrary variables. It could be anything else.
And this data set could not be employees.
It could be a machines,
it could be certain observations from nature and pretty much anything.
The point here is that the further away two points are, the more dissimilar
they are, and that is being measured
or captured in our dendrogram by the height of this bar.
How high we're setting it.
And then the bar itself just shows us that we connected P2 and P3.
All right.
So that's our first step in the dendrogram.
Next we're going to move on.
And we're going to proceed to the next step in our algorithm.
We're going to perform step three.
So we're going to find the next two closest clusters and connect them.
So here we've got or each each point out of these four is a cluster.
And then we've got this cluster.
Now we need to find the two closest out of all of them.
And let's say or from what we see these two are the closest.
So let's outline them.
There we are. And so now they form their own cluster.
Now we want to point that out in the dendrogram as well.
So again we're going to place this vertical horizontal line.
Again. How high do we place it.
Do we place it higher or lower than this line.
Well we agreed that this vertical axis represents the Euclidean distance.
And Euclidean distance
represents the dissimilarity between two of our observations.
So here we can see that P5 and p6 are actually further apart than P2 and P3.
And that is of course natural because if
p5 and P6 were closer then in the in the previous step,
we would have put P2 and P3 in one cluster.
We would have put P5 and P6 in one cluster.
Remember
we're always looking for the closest and then we're moving on to the next step.
So between P3 were the closest.
And that's why this distance is such P5 and P6 are further apart
from each other than P2 and P3.
So the distance has to be greater.
And that's why we're going to show that on the dendrogram.
You can see that this bar is set higher.
All right.
And the next step is to again repeat step three.
So we're going to look among these all of these clusters which are the closest.
So there we go.
So this one is the was the closest I'm going to go back here.
This cluster is closer to this cluster than to any other cluster.
And pretty much out of all the distances between the clusters this is the
the lowest.
Again a lot is here is determined by how you measure distances. You.
We can see that
the distance between P4 and this cluster is quite close to this distance.
But so we're going to say that this distance is the lowest.
All right.
So what do we do next is we combine these clusters into one cluster.
Let's do that.
There it is.
So now we have one cluster. Now we need to represent that somehow here.
So what we just did is we took this cluster that we have P2, P3
connected it with P1.
So again we're going to draw a line and we're going to draw vertical lines here.
Again.
And once once again the distance
here from P1 to the top over here
represents the dissimilarity between that cluster that we had.
And what's the point that we connected it to.
All right. So now let's connect.
let's find what's the next step.
Next step again is step three.
And we're going to look out of one, two, three clusters that we have
which are the closest.
Well all this is before and it's the closest to these.
Again.
There it is. We're expanding that cluster.
And now we're going to represent on the dendrogram.
As you can see the line is about the same height as this previous line,
because the distance between P1 and this cluster was about the same
as the distance between PS4 and PS5 and P6, maybe this one was a bit greater.
Sometimes it's hard to tell.
And that's why we have algorithms.
That's why machines do it for us.
One of the reasons, that's what our data gram looks so far.
And our final step is to combine these two remaining clusters, because by default,
they are going to be the closest since there are no other clusters.
So we're going to combine them and represent that on a dendrogram.
So here the line is very high because the distance it was just similarity.
It was very high between these two clusters.
And there we go.
So that is how we construct our dendrogram slowly from the bottom up
is being constructed.
And at the end we've got that one cluster.
So all of this is one cluster.
And that is what I mean when I say that the dendrogram contains
the memory of the hierarchical clustering algorithm.
So you can just by looking at the dendrogram
understand in which order these clusters were formed.
And here I've got an example.
So this is the actual example generated by computer
generated by an algorithm showing us the hierarchical clustering.
So we've got the points here and we've got the dendrogram over here.
So this is what it actually looks like.
All right.
So now we know how dendrogram are constructed.
In the next tutorial we will learn how to use them to enhance
our or actually execute our hierarchical clustering algorithm.
So there we go. I hope you enjoyed today's tutorial.
I look forward to seeing you next time.
And until then, enjoy machine learning.