Hello everyone and welcome to the introduction to support vector machines.
Lecture this lecture will discuss the formal definition of support to machines and then try to get an
understanding of the intuition behind support vector machines or as the EMS if you want the mathematics
behind this algorithm go ahead and read Chapter 9 of an introduction to Cisco learning support vector
machines or SVM as are also known or supervised learning models of associated learning algorithms that
analyze data and recognize patterns.
Their use for classification and regression analysis.
In this lecture we'll be talking about their use for classification given a set of trainee examples
each marked for belonging to one of two categories.
So binary classification and SVM training algorithm builds a model that assigns new examples or at least
test data points into one category or the other.
Making it a non probabilistic binary linear classifier in SVM model is a representation of the examples
as points in space map so that the examples of the separate categories are divided by a clear gap that
is as wide as possible and that's going to begin to set the intuition for an SVM that we'll see later
on through some plots and diagrams.
New examples are then mapped into that same space and predicted to belong to the category based on which
side of that gap they fall on.
All right let's go ahead and try to understand the basic intuition through looking at some diagrams
and some plotted out data.
Imagine we have the training data below.
Here we have two classes.
We have a blue class and a pink class.
Now what we're going to try to do is a binary classification for new points.
We want to put a new point on this plot where the horizontal axis is some feature one and the y vertical
axis is a feature too.
When we put a new point we wanted the and does it belong to the blue class or the pink class.
Intuitively what we could do is draw a separating hyperplane.
In the case of two mentions is just a line between the classes.
However we have lots of options of hyperplane that separate these two classes perfectly.
Notice that the pink black and green line would all separate the blue and pink training points perfectly
.
The question that arises how do we actually choose the line that separates these classes the best.
What we would like to do is choose a hyperplane that maximizes the margin between the classes.
So you'll see a diagram.
Typically it looks like this where you have a separating hyperplane here denoted as the dotted line
and then a margin that extends out from that hyperplane the vector points at the margin line touch are
known as support vectors and that's where the name support vector machines come from.
Here we can see the circle points that are the support factors.
Those are the training points that actually touch those margin lines.
We can expand this idea to nonlinearly separable data through the use of a kernel trick.
That means if you take a look at the left hand plot in two dimensions here you may have a X label on
y label and you'll notice that this data is not linearly separable because it's a loose circle in the
middle of blue triangles and red outer circle of red circles and there's no way we can draw a straight
line to separate these classes.
However through the kernel trick what we do is we end up doing this in a higher dimension.
In this case we look at a third Z label over on the right.
Now we can see that this is separable in the third dimension through another hyperplane.
You can check out YouTube for a really nice 3D visualization videos explaining this idea and if you
want the mathematics behind the kernel trick again go ahead and check out Chapter 9 of an introduction
to Stichel learning.
Now that you learn the theory behind support vector machines we can continue our understanding by building
these models with Python.
We'll start an example by predicting whether or not a tumor is malignant or benign by using a built
in breast cancer dataset that comes with sikat learn then your portfolio project will apply these same
concepts to the famous Iris flower dataset and then we'll learn how to tune our models for grid search
.
All right.
There's a lot of really exciting concepts coming up and I'll see the next lecture