1
00:00:00,150 --> 00:00:03,630
So now let's take a look at creating our confusion matrix.

2
00:00:04,170 --> 00:00:09,990
So to do that, we just have to import the confusion matrix function here from scale and what metrics

3
00:00:10,350 --> 00:00:12,510
we also import number, which we did previously.

4
00:00:12,510 --> 00:00:16,890
But will let's do it again just for some consistency in case you want to extract this block of good

5
00:00:16,890 --> 00:00:17,910
for future use.

6
00:00:18,600 --> 00:00:26,580
So all we're doing here, we're taking steps, reshaping it again and passing it on input into our AMAX

7
00:00:26,580 --> 00:00:32,160
function to get the maximum, to get a class of the maximum probability, then so that gives us now

8
00:00:32,430 --> 00:00:33,480
all our predictions.

9
00:00:33,810 --> 00:00:38,370
So that's why we feed it into a model that predicts excess to get them predictions here.

10
00:00:38,910 --> 00:00:44,610
So no, we have no predictions why we have all grown short levels, whitest.

11
00:00:45,030 --> 00:00:49,320
We just input them into the confusion matrix function and run this.

12
00:00:49,320 --> 00:00:55,560
And it should give us that table that we saw below and hopefully remember how to interpret this.

13
00:00:55,560 --> 00:00:58,410
This is an example of an image I can show you.

14
00:00:58,830 --> 00:01:02,130
So these here are the predicted levels a top rule here.

15
00:01:02,460 --> 00:01:07,500
These here the ground should labels, so you can see for all the ground truth labels that the Label

16
00:01:07,500 --> 00:01:11,700
Zero is predicted in nine hundred and seventy seven times that it was zero.

17
00:01:12,210 --> 00:01:15,990
One time it predicted it was a six or seven and eight and so on.

18
00:01:15,990 --> 00:01:21,120
And you can see the ones I've circled here, the ones where it's classifying like this here, classifying

19
00:01:21,120 --> 00:01:22,350
seven as 2s.

20
00:01:22,830 --> 00:01:24,300
In that case, where is that here?

21
00:01:24,310 --> 00:01:24,900
That's here.

22
00:01:25,170 --> 00:01:25,680
It's 11.

23
00:01:26,100 --> 00:01:32,580
So you can see when the ground shoot was a seven hour model predicted that it was a two 11 times.

24
00:01:33,270 --> 00:01:35,390
So this is in from this table here.

25
00:01:35,400 --> 00:01:38,670
This is this is the actual results for the model that we predicted above.

26
00:01:39,360 --> 00:01:41,260
So you can see this model isn't performing that well.

27
00:01:41,280 --> 00:01:45,330
You can see there's a lot of mistakes going on over here as well as of here.

28
00:01:46,500 --> 00:01:48,630
And what what it says here is 28.

29
00:01:48,630 --> 00:01:49,590
Let's take a look at that one.

30
00:01:50,040 --> 00:01:54,090
This would be the zero one two three four.

31
00:01:54,570 --> 00:01:58,950
So of Class four, twenty times, it's predicting it to be a nine.

32
00:01:59,070 --> 00:02:01,800
Actually, it's not good, but that's OK.

33
00:02:02,520 --> 00:02:03,660
We can do better, for sure.

34
00:02:04,440 --> 00:02:08,490
So here's a more guess some could that have got gotten off the internet?

35
00:02:08,940 --> 00:02:12,660
That basically allows us to plot a much nicer looking confusion matrix.

36
00:02:13,110 --> 00:02:17,370
We don't have to go through this good because it can be quite exhaustive to go through that.

37
00:02:17,790 --> 00:02:19,560
So this is run that function.

38
00:02:19,680 --> 00:02:22,020
This is to plot confusion matrix.

39
00:02:22,170 --> 00:02:22,890
These are the inputs.

40
00:02:22,890 --> 00:02:24,990
It takes some default values.

41
00:02:24,990 --> 00:02:29,070
So what we need to do is just input the confusion matrix and the target names.

42
00:02:29,670 --> 00:02:36,420
And so we get a confusion matrix again from our scaling function.

43
00:02:36,990 --> 00:02:40,000
Target names are basically the range of names is in.

44
00:02:40,050 --> 00:02:41,820
In this case, it's zero to nine.

45
00:02:42,300 --> 00:02:47,610
So we just generated using this range function and convert it into a list and we just use a function

46
00:02:47,610 --> 00:02:51,820
plot confusion matrix here, where the confusion matrix and the target names.

47
00:02:51,840 --> 00:02:59,460
And it gives us this nice, colorful color gradient output that we can use to visually assess performance

48
00:02:59,460 --> 00:02:59,790
better.

49
00:03:00,480 --> 00:03:05,520
So that's OK footwork, and this is just a nicer way to present it if you want to present it in a more

50
00:03:05,520 --> 00:03:08,760
professional manner, and it gives you a summary of the accuracy below.

51
00:03:10,410 --> 00:03:13,050
So now let's take a look at our poor class accuracy.

52
00:03:13,140 --> 00:03:17,160
Here's a fairly simple way to get this there are many different ways you can probably get this as well.

53
00:03:17,250 --> 00:03:21,990
You can just use our imagination if you've got good numbers, skills or panda's skills, you can easily

54
00:03:21,990 --> 00:03:28,580
just manipulate the prediction or put results and compare it to the white test and generate your 1Q

55
00:03:28,590 --> 00:03:29,070
report.

56
00:03:29,520 --> 00:03:34,890
So what we're doing here, in this case, we're taking the confusion matrix output, and we're taking

57
00:03:34,890 --> 00:03:35,910
that diagonal here.

58
00:03:36,360 --> 00:03:38,250
That's this diagonal one at a time.

59
00:03:38,770 --> 00:03:43,170
Well, one of the time we're taking the whole matrix here and then dividing it by the sum.

60
00:03:43,560 --> 00:03:49,570
That's a number that's a total number across each row, and that gives us class accuracy.

61
00:03:49,590 --> 00:03:55,650
So we just enumerate we just looked through the output of this matrix because it's going to be in.

62
00:03:56,460 --> 00:04:00,270
It's going to be one by one for each class, and we'll just print the accuracy below.

63
00:04:00,750 --> 00:04:02,430
So you can see now let's run this.

64
00:04:03,450 --> 00:04:05,490
You can see the accuracy for each class right here.

65
00:04:05,910 --> 00:04:08,860
So you can see it's doing fairly well for zeros and ones.

66
00:04:09,270 --> 00:04:14,910
However, it's doing a bit more poorly for fives, as well as tourists and nines, to an extent.

67
00:04:15,540 --> 00:04:19,200
So now let's take a look at the classification report.

68
00:04:19,800 --> 00:04:26,250
So you may remember the classification report gives us summary of Precision Recall F1 score, which

69
00:04:26,250 --> 00:04:31,720
was a combination of both precision and recall, as well as the support, which basically is sum of

70
00:04:31,720 --> 00:04:33,720
the number of images in that class.

71
00:04:34,350 --> 00:04:36,060
So to do that, it's quite simple.

72
00:04:36,060 --> 00:04:40,170
Again, you just import classification report from schooland dot matrix.

73
00:04:40,680 --> 00:04:44,340
Then you take the lightest test is a drunkard labels for test data set.

74
00:04:44,820 --> 00:04:48,000
These are the predictions when we feed excess into a model.

75
00:04:48,450 --> 00:04:53,160
So we have white bread, which we have from above before and we run this.

76
00:04:53,400 --> 00:04:55,840
And this gives us a nice little confusion.

77
00:04:55,860 --> 00:04:59,490
Sorry, old classification matrix right here and in.

78
00:04:59,650 --> 00:05:01,900
Yes, you forgot how to interpret these things.

79
00:05:02,620 --> 00:05:06,290
Interpret, recall, recall and remember the actual true positives.

80
00:05:06,350 --> 00:05:11,770
How many, how many, how many times the classifier predicted that class, as well as precision, which

81
00:05:11,770 --> 00:05:17,110
is a number of correct predictions over how many occurrences of a class within the test dataset?

82
00:05:18,190 --> 00:05:20,230
So he can use this graph here.

83
00:05:20,350 --> 00:05:26,830
These images here to understand properly what recall is and what precision is, as well as what what

84
00:05:26,830 --> 00:05:28,690
a higher recall means with little precision.

85
00:05:29,010 --> 00:05:30,880
Well, low recall means with high precision.

86
00:05:31,270 --> 00:05:35,350
So that's why I find these notebooks a very good resources for learning these things.

87
00:05:35,770 --> 00:05:39,520
I'll try to put as much notes and comments in accord to help you guys understand.

88
00:05:39,640 --> 00:05:45,640
So hopefully that was a useful lesson and we'll stop there and I will do the same thing in PyTorch.

89
00:05:45,760 --> 00:05:47,260
So stay tuned for that lesson.

90
00:05:47,380 --> 00:05:47,800
Thank you.