1
00:00:00,780 --> 00:00:06,330
Hi and welcome back to the course in this section, we'll take a look at a new metric that's used commonly

2
00:00:06,330 --> 00:00:10,950
for benchmarking classification models, and it's called the top one or top end.

3
00:00:11,310 --> 00:00:11,730
What's up?

4
00:00:11,730 --> 00:00:15,000
Five accuracy, also known as around one, around five.

5
00:00:15,480 --> 00:00:16,450
So let's get started.

6
00:00:16,500 --> 00:00:17,960
So what is this accuracy?

7
00:00:17,970 --> 00:00:18,810
What is this metric?

8
00:00:18,810 --> 00:00:19,230
Tell us?

9
00:00:19,260 --> 00:00:26,130
Well, remember when we input this colleague, this dog here into the pre-trained network, it would

10
00:00:26,130 --> 00:00:28,790
often sometimes written the wrong class, but returned.

11
00:00:28,800 --> 00:00:33,510
But if we inspected the results, you can see this is the output here.

12
00:00:33,960 --> 00:00:40,560
You can see sometimes that the actual correct class is maybe the second or third or fourth or fifth

13
00:00:40,560 --> 00:00:41,430
ranked class.

14
00:00:42,120 --> 00:00:43,110
That's interesting.

15
00:00:43,560 --> 00:00:48,150
And as I said, here rank and is a way to give to classify as accuracy a bit more leeway.

16
00:00:48,660 --> 00:00:53,880
So instead of it returning, Shetland Sheepdog has a particular class, which it would do naturally.

17
00:00:54,360 --> 00:01:00,080
When we consider around inaccuracy, we look at the top five or top tree, whatever it is.

18
00:01:00,120 --> 00:01:01,710
So let's assume and it's five here.

19
00:01:02,280 --> 00:01:09,490
So if the correct class belongs in the top five highest probability classes that are output from a little

20
00:01:09,600 --> 00:01:13,800
CNN, then we consider that as being correctly identified.

21
00:01:14,220 --> 00:01:20,100
And that's why for rank five accuracy, you see the accuracy of scores go close to ninety nine point

22
00:01:20,110 --> 00:01:21,930
eight percent in a minute.

23
00:01:22,320 --> 00:01:27,000
However, if you look at the rank one or top one accuracy, it's usually at 80 percent roughly if the

24
00:01:27,000 --> 00:01:32,340
state of the art model is sometimes 85 89 percent, depending on how good the model is.

25
00:01:32,850 --> 00:01:39,870
So basically, that's it sort of rank, and it's basically considers the top in classes with the highest

26
00:01:39,870 --> 00:01:43,530
probabilities and considers that as the predicted level.

27
00:01:43,590 --> 00:01:47,130
So if any of these categories here are ground, should it considers that correct?

28
00:01:47,760 --> 00:01:55,470
This is an example of the top one accuracy for image net and top nine percent up top five percent accuracy

29
00:01:56,010 --> 00:01:57,270
for imaging that you can see.

30
00:01:57,270 --> 00:02:04,530
This is the best, more efficient nutshell to model the second by efficient method piece by dash B-6

31
00:02:04,530 --> 00:02:05,310
wide model.

32
00:02:05,880 --> 00:02:09,780
The top one accuracy is a 90 percent, which is remarkably good for image net, by the way.

33
00:02:10,440 --> 00:02:13,770
And you can see the top five accuracy with ninety eight point eight percent.

34
00:02:13,770 --> 00:02:15,580
So that's also very, very good.

35
00:02:16,110 --> 00:02:19,410
We're getting close to human level performance on the that.

36
00:02:19,830 --> 00:02:22,260
I'm actually not sure what human level performance and imaging is.

37
00:02:22,260 --> 00:02:27,870
There actually are probably some benchmarks out there that take a random group of humans and test them

38
00:02:27,870 --> 00:02:28,180
on it.

39
00:02:28,200 --> 00:02:34,590
However, with a thousand classes, it will be quite subjective sometimes for people to actually assess

40
00:02:34,590 --> 00:02:35,160
those things.

41
00:02:35,730 --> 00:02:43,470
Anyway, this is it for the top end or top one, top five or rank and rank one around five accuracy

42
00:02:43,470 --> 00:02:43,950
metric.

43
00:02:44,160 --> 00:02:45,180
So that's it.

44
00:02:45,180 --> 00:02:51,390
And what we'll do now will go into the could into Keras and PyTorch, and I'll show you how we actually

45
00:02:51,390 --> 00:02:57,640
can get the top and accuracies quite easily for models that we've from pre-trained models, actually.

46
00:02:58,170 --> 00:03:00,840
So I'll see you in the coding lessons shortly.

47
00:03:00,960 --> 00:03:01,230
But.