So hit our Jupyter notebook. 1
Let's investigate how wrong and how useful our house price model is. And will commemorate this with a 2
new section heading. I'm going to change this to Markdown here 3
and put two hash tags there and write "Residuals & Residual Plots". 4
And now what I'm going to do is I'm going to take some of our existing code and just copy it below. 5
We're going to reuse some cells, but modify them a little bit. 6
So I'm going to take this cell right here, 7
copy it, scroll down and paste it in, and I'm going to add some comments here. 8
I'm going to call this one "Modified model: transformed (using log prices) & simplified 9
(dropping two features)". 10
Now just because I put that in the comment doesn't mean we've actually done it. 11
So I'm going to come here and on this line, 12
we're not just going to drop the PRICE, we're going to put some square brackets around PRICE, then a comma and 13
then we'll drop two other features that we had earlier - Industry and AGE. 14
Now let me come in here and delete this part of the code. I'm going to hit Shift+Enter, 15
and now I'm going to add another comment here that reads "Using Statsmodel", so instead of our scikit-learn 16
we're to continue using statsmodel here for analyzing our residuals. Next, 17
I'm going to come up here, take these three lines of code, copy them and paste them in here. 18
Okay, 19
so now we're all set up for this lesson. 20
Now we can start looking at our residuals. 21
So I'm going to add a comment here. We said the residuals are equal to the difference between our observed 22
values and our fitted values or our predictions. 23
So the observed values y were y_train, and from y_train we can subtract the fitted values. We can find 24
these under "results.fittedvalues". 25
Let's take a look at what these residuals actually look like. 26
So residuals and Shift+Enter will split and shuffle the data, 27
train the model and then output our residuals. 28
And here we go. 29
These are all 404 residuals from our model. 30
If you're wondering what type of object we're working with here, then we can see it is a pandas Series. 31
That's quite nice because if it's a Pandas Series we can call a bunch of methods on it that the Series 32
has, like describe, 33
and if I do that, we can see that the mean of the residuals is very, very close to zero. 34
But more on that later. 35
So let me quickly comment this out, hit Shift+Enter and show you that you don't actually need to do 36
this calculation manually, because our results object from our Statsmodel 37
actually already has an attribute which allows you to fetch the residuals. 38
So "results.resid" is the name of said attribute and hitting Shift+Enter, 39
we get presented with exactly the same thing. All right, 40
let me comment this code out again, 41
scroll down a little bit, 42
hit Shift+Enter and get to the interesting part, namely the part where we are graphing our stuff. So I'll 43
add another comment here that reads "Graph of Actual vs. Predicted Prices". Now, 44
before I write this code, let me explain the rationale of what I'm about to do here. 45
I want to create a chart that visualizes how good our predictions are. 46
I want to see how our predictions actually stack up to the actual property prices, because seeing this 47
picture will put the r-squared from the regression that we calculated into context. If you remember from 48
our earlier lessons, that r-squared was kind of a nice one number summary for our regression, but it's 49
definitely not the only thing that we should be looking at. 50
Now let me ask you this, on this chart of the actual versus the predicted prices, what would this chart 51
look like if our model was perfectly accurate? 52
If it had like perfect predictive capabilities, what would you expect to see? In that case, 53
you'd expect that the predicted prices are kind of a really, really close match to the actual ones. 54
In fact you'd get all of these things lining up on a straight line, right? 55
You should have a very, very strong correlation between the model's predictions and the actual results. 56
Let's calculate this correlation and store it in a variable. So I'm going to say "corr = y_ 57
train.corr(results.fittedvalues)". 58
This here calculates the correlation method on our y_train object and we provide, as an argument, 59
the fitted values from our regression. 60
Let's output this value here, so I'm going to say corr and hit Shift+Enter. The correlation that we see printed out here 61
is a 0.89 approximately. 62
That's, that's quite high actually. 63
Which is pretty good. 64
I think one thing I'd like to do though is round this value here, so "round()" and then at the 65
end here a comma and 2, so I'm going to round it to two decimal places close the parentheses and hit Shift+Enter 66
again. 67
Nice. 68
Now let's create a scatter plot using matplotlib which we've stored in our plt variable. 69
So I'm going to say "plt.scatter()"; for the x axis, 70
we're gonna use our y_train, our actual values. For the y axis, 71
we're gonna use our fitted values, 72
so this was "results.fittedvalues". 73
Now I'm also going to add a color and a transparency to this scatter plot. For the color, 74
"c = ", 75
I'm going to go for maybe navy blue, so 'navy' in single quotes, and for the alpha, the transparency, 76
I'm gonna go for 0.6. 77
Now let me write "plt.show()" and take a look at what this looks like. 78
Here we go. 79
Now, it's a little hard to see what's going on here. 80
I think we need some labels on our axes and we should also plot a line on this graph that represents 81
the perfect fit, 82
so we have a reference point for this scatter plot. 83
Let me hide this plot by double clicking in here 84
and updating our code now. Now in adding our axis labels, 85
I'm going to show you how you can include scientific notation or LaTeX notation on the labels themselves. 86
LaTeX notation is something that we've used in our Markdown cell but so far we haven't really integrated 87
into our charts. 88
So let's do that now. 89
I'm going to have "plt.xlabel('Actual log prices'), 90
and here's the LaTeX notation: 91
"$$", 92
and then between the dollar signs I'm going to have 'y' 93
and then I'm going to have like an underscore and then an "i", 94
meaning this "i" will be shown as a subscript and anything between this opening and closing tag will be 95
treated as LaTeX 96
notation. Let me add a font size as well, so ", fontsize = 14". 97
Let's see what this looks like with Shift+ 98
Enter. 99
Here we go, as promised, we see that y has a subscript. Let's add the label for the y axis. "plt. 100
ylabel()" 101
And try this one more time, 102
say "Predicted log prices" and for the LaTeX notation, I'm going to say y hat underscore y and I 103
can do this by having that slash, "hat" for the LaTeX tag and then again I'll have "y _i" 104
also "fontsize = 14". 105
Take a look. Here we can see that the hat tag in the LaTeX notation creates this look carry symbol 106
above our y here. 107
Nice. 108
Now let's add a title to this chart. 109
And in the title we'll use both LaTeX notation and we'll use an f-string to show our correlation as 110
well. 111
So "plt.title(f'Actual vs Predicted log prices: 112
$$'", 113
and between the dollar signs, "y _i$ vs $\hat y_i$", 114
and then outside the dollar sign "(Corr {corr}), which is the name 115
of our variable here. And at the end of course we've got closing curly braces, 116
the closing parentheses for this one, single quotes finish it all off and maybe a fontsize of say, 117
17. 118
Let's take a look. 119
Fatal Exception. 120
What was the problem? 121
A typo in the LaTeX tag. 122
It should read "\hat". 123
There we go. 124
So I think this chast is already starting to look pretty good. 125
The only thing missing is that line that represents perfect predictions and we can add that up here 126
with "plt.plot()", plotting a single line and that line for the x and the y will be the same. 127
So in this case it would be "y_train, y_train" and I'm gonna give it a different color as well, so 128
color is equal to say "cyan". That'll make it stand out quite nicely, so check it out. 129
Here we go. 130
This cyan line would represent a perfect prediction. 131
So it goes through all these grid points here, 2 and 2, 2.5 and 2.5, 3.5 and 3.5 132
right? 133
And this allows us to see how our predictions stack up to the actual values very, very clearly. 134
The bigger the distance from the cyan line the bigger the residual. 135
In other words, in this part of the chart our model is doing really, really well. 136
But for these data points here we have much larger residuals. 137
Now what if looking at log prices isn't that helpful? 138
What if you wanted to look at the actual prices, the actual dollar values? Because that's much more intuitive. 139
All we have to do is reverse our transformation, right? 140
So let's create a second chart below this one. 141
I'll take this code here. 142
Copy it, paste it below. 143
And then all I have to do to reverse the transformation is raise our data points to the power of s so 144
"np.e**" 145
which raises it to the power and we'll do the same thing here "np.e", 146
raise it to the power of our residuals. 147
And of course we have to do the same thing here, "np.e 148
**", "np.e**" and then "Actual prices" in the label, "Predicted prices" in the label 149
and then for the title we'll also do that. To differentiate the two charts visually, 150
I'm just gonna go with straightforward "blue" for this one, so our scatter plot here 151
will just be blue. Shift+Enter updates what we see, and now our axes are transformed and we see our dollar 152
prices in thousands. We can even add that to the label. 153
So "Actual prices 000s". And here "Predicted prices 000s". Shift+Enter, and now our labels are updated. 154
Now that we've got our charts pretty much where we want them to be, we can start to interpret what it 155
is that we're looking at. 156
How do you think our predictions are stacking up to the actual values looking at these charts? 157
I think we're doing actually rather well. 158
And what we're seeing here pretty much ties out with the expectations that we had 159
looking at the high r-squared. 160
A lot of the predictions in fact are very close to this cyan line here. 161
Mind you, this cyan is not the best fit line for the scatter plot. 162
The line here shows where a points would have to be if our predictions were perfect and our residuals 163
would be equal to zero. 164
The cyan line represents where y hat would be equal to y. 165
But looking at this chart I do see something that kind of concerns me a little bit. 166
Take a look at this top right corner here. 167
There's definitely something kind of strange happening with the properties at the very top price bracket. 168
These are the properties at the fifty thousand dollar mark. Here we can see that our residuals are actually 169
fairly large and our model is performing quite poorly. 170
If you remember we actually visualized all our target values in our data exploration stage and then 171
we already spotted that there were quite a few properties at the top end of the price bracket and that, 172
given this histogram, these kind of struck us as outliers and we suspected that maybe these might cause 173
an issue down the line. 174
What I'm saying is is that - yes, there's room for improvement in our model, obviously. 175
but overall I think I'm pretty happy with what we've got so far. 176
After all, we're dealing with a real data set so it's nice to see things line up to the extent that they 177
do.