Hello everyone and welcome to the solutions lecture or the decision trees and ran the forest project
.
This lecture will go ahead and work through the code for the solutions.
Let's go ahead and jump to a notebook to get started.
All right.
Here I am at the project notebook.
I'm going to go ahead and just scroll down to get started.
First thing we need to do is import the usual libraries are going to import pandas as PD and import
pi as and T.
And then I'll do my visualization libraries such as MAP plot lib pipe plot as PLDT and importing seaborne
as asinus.
And finally since I'm in the notebook going to be port live in line so I can see my visualisations.
They want to use panderers to read in the lone data that CSC file into a data frame called loans or
go ahead and say PD reads CXXVI just using tab there and then start typing out alone.
And I should see loaners core data as a CSP.
Let's go out and check out the info head and describe methods on the loan so checking out loans info
.
We go ahead and see that there is around a little less than 10000 rows and there's 14 columns and if
you want more information about what these columns represent you can just come up here and read out
these column descriptions.
But we'll go ahead and continue on.
The described method in C loans describe to get some statistical information off to numerical columns
and then finally if you just want to see what a typical column looks like you just call loan's head
and pasand some rows.
You can see there's a credit policy purpose the re installment etc. FICO score and then some more information
.
Now we're going to go ahead and move on to some exploratory data analysis.
We'll go ahead and start some data visualization.
We're going to use seaborne empanadas built in plotting capabilities that you can also feel free to
use whatever library you want that we covered in that data visualization section of this course.
And one more note.
Don't worry about the colors matching here.
Just worry about getting the main idea of the plot.
First thing we wanted to do was create a histogram of the two Fika distributions on top of each other
one for each credit policy outcome.
First thing I want to do is check out the example plot here looks like OK have a histogram of the final
score and then you have blue equals credit policy equals 1 and then the red its credit policy equals
zero.
That means we're going to have to separate by credit policy.
I'm going to do this with pandas built in histogram.
I will say loans where this credit dot policy is equal to one.
And I'm going to go out and copy and paste that where credit policy is equal to zero and then off of
both of these I'm going to grab the psycho column and then I'm going to just call.
Got hissed off at this.
You can also do that plus the highest.
But either way it should work.
And then we will go ahead and do a couple more things.
If we just do this we're going to run this show this looks like we've got a plot that looks like this
which is OK but we want a little more information here.
Let's go in and increase the bins.
So let's go ahead and say 35 each.
So bins equals 35.
And then we want to color them if we want to say blue for an actual credit policy and then read for
not a credit policy.
And I'm going to go ahead and add in a label and that way we can call Piazzi legends at the end of this
.
I just say Piazzi legends.
This one is credit policy equals zero.
So I'm going to go ahead and say that credit policy
equals zero and then I'm going to do a similar thing appear so I can just copy and paste this line.
Except this is credit policy equal to one.
And I'm going to break this up so it's in in other lines.
We can actually read it a little easier.
All right.
And then finally I want to add in an alpha to this so I'm going to go out and say Alpha equals 0.6.
Just so there's some clarity.
How see through this is so alpha is equal to 0.6.
And then they go out and expand the figure size a little larger by saying Pulte figure fig size is equal
to 10 by six.
All right let's see if all this worked great.
And now we have a really nice looking visualization where I can very quickly see a lot of information
about the data right off the bat.
I can tell that I have more people in my data set that have a credit policy equal to one than a credit
policy equal to zero.
And I can also see that based off of the final score we can add that label him.
So we'll save Piazzi X label is the final score that based off of this final score.
People who tend to have a lower FICO score have a credit policy of zero.
And remember that credit policy.
One is if the customer meets the credit underwriting criteria of lending club dot com and it's a zero
if otherwise we want to go ahead and check this out by looking at this cutoff point.
And you can see that basically any one for final score of less than around 660 will automatically not
meet the credit underwriting criteria of Lending Club dot com.
So that's an interesting visualization.
So you get some information off of your data set.
Let's go ahead and create a similar figure except this time by the not fully paid column so not fully
paid because one not fully paid equals zero which is again or trying to predict when we go out and copy
this because essentially all we have to change is a set of credit policy we want not fully paid.
And I can just use tab autocomplete there so not and then tab and we'll go ahead and say instead of
credit policies will one not fully paid.
And again here will change just to not fully paid
and let's go in and run that.
And here again we can see that the majority of people have zero for not fully paid.
I mean the majority of people are actually fully paying off these loans.
And we can see that more or less a similar distribution between both not fully paid equals one and up
fully paid equals zero.
Doesn't seem to have the same sharp cut off on final score as the previous credit policy that you can
see it is essentially overlap.
The only differences the actual counts and you'll notice that there is kind of weird spikes at certain
points and that's just because of the way Fika credit scoring works that certain points will have a
larger distribution of likelihood of being your final score melts move along by creating a counterplot
using seaborne showing the counts of loan by purpose.
What's the color hue defined by not fully paid.
All right.
NORRIS do this.
I need a call Sinistar counterplot.
I look at the example plot here.
I want X to be purpose and they hue to be not fully paid.
So I'm going to say x is equal to.
Purpose.
My hewe is equal to the not fully paid column.
My data source is loans.
That's my data frame and I'm going to go ahead and set the Palit equal to set one just so I can get
this blue red Palette continuously and here's my plot.
I'm going to go ahead and increase the plot size the figure size is appeal to figure sig size.
And then if we make it 11 by 7 we won't get this kind of weird overlap on those labels.
Great.
So just by looking at this plot we can tell that debt consolidation seems to be the most popular reason
for wanting a loan and then credit card and or other reasons.
Our second and third place with things such as Home Improvement small business major purchase and educational
reasons being much lower on that list.
Something to note here is that the ratio between not fully paid and fully paid seems to be pretty similar
between all the reasons.
Let's go out and continue on by seeing the trend between FICO score and interest rate.
So we're going to recreate this falling joint plot and you can see here that as your FICO score increases
the interest rate you have to pay off that loan tends to decrease which kind of makes sense.
Let's go in and see if we can do that will say Esson s joint plot X's psycho and I will say why is interest
rate which is ANC rate data to the loans data frame.
And then I'm going to say.
Color is equal to purple or you can just use the default color.
And here we were able to recreate this plot.
And this makes sense that as your FICO score decreases it means you are excuse me as your FICO score
increases you have better credit.
So your interest rate on your loans is probably going to go lower and vice versa as your FICO score
goes down.
You have worse credit or interest rate you're going to pay off that loan it's going to need to go up
.
Now we want to do is create the following L-M plots to see if the trend differed between not fully paid
and credit policy.
And again you can check the documentation for L-M plot.
If you can't figure out how to separate this between two columns let me go in and show you how we can
do that.
They say s n s l m plot.
We need to first say what we want on the Y.
So if we look at the wind that's just going to be the interest rate I.A. rate.
And if we look at X we want that to be the final score.
And our data is going to be equal to loans.
We just take a look at this plot by itself.
We get something that looks like this which is essentially the plot appear just a linear model line
on top of it.
However we want to go ahead and add a couple more arguments here aren't the hue to be defined by their
credit policy column.
We go ahead and check that.
Now we have credit policy zero and one.
And again we noticed that kind of cutoff around the 650 mark and then the second thing I want to do
is actually split this up into two columns.
You can do that by passing in the call arguments.
C O L O say C O L is equal to not fully paid.
And I will set my palette equal to set one just so it matches some of the previous things we've done
.
And I'm going to go ahead and expand the figure size by saying six sizes equal to 11 7.
And now we are going to get a plot that matches what we wanted for the output.
So here we have the columns representing the not fully paid value and the coloring representing the
credit policy.
And you can see here that the behavior is relatively the same whether or not it was fully paid off or
on and whether or not they were denied the credit policy.
We'll go ahead and stop part one of the solutions lecture here.
And in part two we'll see how to do all of the categorical features in our data frame and we'll begin
to actually build our decision tree and ran the course model.
Thanks everyone.
I'll see you in the next lecture