Here I've got a confusion matrix.
And this confusion matrix has 10,000 records in it.
And it represents scenario number one which we'll be looking at.
As you can see this model has made 150 type one errors and 50 type two errors.
And but overall is predicted quite a lot correctly.
Now let's calculate the accuracy rate in this scenario.
The accuracy rate is the total correct divide.
But overall total and it's 9800 divided by 10,000 which is 98%.
Okay great.
But what are we going to do now?
Is we going to tell the model to stop making predictions
whatsoever, which is going to abandon the model completely?
And we're going to say that from now on, our prediction is always zero.
We're always going to predict that the event is not going to occur.
So basically what will happen to the confusion matrix is
these records will move from the right column to the left column.
And our new confusion matrix will look like this 9850 150.
And then nothing in the predicted column where we predicted
that something will occur.
And of course, that move is against all logic, right?
Why would you abandon a model?
But let's calculate the accuracy rate in this scenario.
Scenario two accuracy rate has the same formula.
In this case, accuracy rate is 9850 divided by 10,000.
So it's 98.5%.
The accuracy rate went up by half a percent.
And as you can see, what we did is we just completely stopped using a model.
But the accuracy rate went up and
that is why you should not
base your judgment just on accuracy rate, because things like this can happen.
And even though obviously you're not using a model anymore,
which means that, you're not applying any kind of logic
into your decision making process, your accuracy rate is going up.
So it's misleading you in, into a wrong
conclusion that you should stop using models.
And this effect is called the accuracy paradox.
And starting from the next tutorial, I will show you a much better way
to assess your models using the cumulative accuracy profile.
I look forward to seeing you then.
Until next time, happy analyzing!