All right so we've plotted our data and we suspect that there is a relationship there. 1
Now it's time to talk about the algorithm that we will use to help quantify this relationship between 2
budget and revenue. 3
In this lesson we're going to talk about the theory behind regression. 4
So, we know what our real data will look like. 5
But let's illustrate what the algorithm will do with a stylized example to help build our intuition. 6
Our linear regression 7
will get two kinds of data. 8
It will get our film production budgets and it will get our film revenues. 9
The budgets will be our feature, also called the independent variable and the revenue are what we are 10
trying to estimate - that will be our target. 11
What the linear regression will do is try and represent the relationship between the budget and the 12
revenue as a straight line. 13
But here's the rub - 14
what kind of line? 15
Let's think back to high school math class and let's think about what describes a line. From our math 16
classes, 17
we know that we can plot y as a function of X and that's a line. 18
And if we cut the y-axis at 10 then we say that our line has an intercept of 10 and if every time X 19
increased by 2 and it made y increase by 1, then we say that the line has a slope that is equal to one 20
half. 21
In that case our equation would look something like this: 22
y = 1/2 x + 10 23
And that means that the generic equation for line would be something like this. 24
It would be y = mx + c, where m is the slope and c is the constant. 25
So let me ask you this. 26
What part of the equation for the line would tell you about how strong the relationship is between x 27
and y? 28
In this case the slope is the key. 29
The slope tells us how much y will change for a given change in X - the larger the value of the slope 30
the steeper the line becomes. Let's take a look at an example where there is no relationship between 31
x and y. 32
Let's take a look at an example where there is no relationship between x and y. 33
If there is no relationship then we would simply have a straight line. 34
In this case the slope would be equal to zero. 35
But if there is a relationship between the two then the slope would be quite steep and the stronger 36
the relationship the steeper the slope. 37
But here's the thing. 38
There's a big difference between machine learning and pure mathematics; in machine learning, 39
we don't actually know the true relationship and that's why we refer to the slope and the intercept 40
as parameters and these parameters have to be estimated for our linear regression. 41
In fact we even use a different notation. In our notation, 42
we will replace the c for the constant with theta 0 and the slope coefficient will be written as theta 43
1 and also we'll change the order in this equation, so we'll have the constant first and then the slope. 44
And instead of writing y, what you'll also often see is h theta x where h stands for hypothesis. 45
This kind of notation is very popular in machine learning and even though it can look quite intimidating 46
when you first see it all you're looking at here is the equation for a simple line. 47
And the reason I'm showing you this right here is because if you ever pick up a book on machine learning 48
or you're reading some articles on the Internet you're going to be confronted with notation that's very 49
very similar to this. But at this point we still haven't talked about where the line ultimately comes 50
from. 51
How do we know which line to draw? Looking at the data we just have data points. 52
There is actually no line, 53
right? 54
And as a matter of fact, you can draw a whole bunch of different lines through the same set of data points. 55
So, which line is best? 56
Which line would you choose? 57
Which line has the best possible theta zero and best possible theta one? 58
If our dataset looked just like this, our job would be easy. 59
All we would have to do was connect all the data points with a straight line. 60
And this also seems like the best option because we would know that in this case our estimates for theta 61
zero and theta one would be very accurate. 62
However, real data looks more like this. 63
If we were to draw a line through this data then there would always be a gap between the actual value 64
and the line. 65
In other words there would be a difference between the actual data point and the point on the line. 66
The point on the line here, 67
that's called the fitted value or the predicted value. 68
But let's talk more about these gaps because it's these gaps that will help us choose the best possible 69
intercept and the best possible slope for our line. 70
These white lines are actually called residuals. 71
Now why will the residuals help us choose the best possible line for our data? 72
Let me show you another line that we can draw to this data. 73
If I draw a line down here, then what we see is that the gaps between the data points and the line are 74
much larger. 75
The residuals are way biggerm so the residuals can tell us something about how good the line is that 76
we're drawing on this chart. 77
So now we have a measure by which to compare the different lines that we can draw through the data, 78
all we have to do is look at the size of the residuals and choose the line with the smallest residuals. 79
And that's great because now our algorithm has a very clear objective. 80
The goal of our linear regression is going to be to calculate the line that minimizes these residuals. 81
But how exactly should that work? 82
Let's take a look at that first residual. That first residual is gonna be the difference between the 83
actual value, the y1, and the predicted value which is the one on the line and that second residual would 84
also be just the difference between the actual value in white here and the fitted value in green. And 85
the same is true for that third data point. 86
Now suppose we actually have calculated the values for these residuals and these residuals have the 87
values 10, negative 6, and 4. 88
In this case what we can't do is just add them up and find the lowest sum because that second data point 89
is below the line. 90
We have a negative number here. 91
So what we have to do instead, is we have to turn all of these numbers positive and the way we can do 92
that is by squaring the residuals. 93
Now what we've got is a single number - the sum of the squared residuals. 94
This is the number that the linear regression will try to minimize in order to choose the best parameters 95
for the line. 96
In other words; to find the best possible fit for our regression what we need to do is we need to choose 97
an intercept - the theta zero, and we need to choose the slope - theta one that minimizes the sum of the 98
squared residuals. And you'll see this number also being referred to as the residual sum of squares or 99
RSS. 100
So now that we've talked about the theory and built out our intuition behind our regression let's implement 101
this in Jupyter notebook. 102
I'll see you in the next lesson.