1
00:00:00,360 --> 00:00:06,870
And welcome back to the course in this section, we'll take a look at making a low light image enhancer.

2
00:00:06,990 --> 00:00:10,680
Basically, our own vision of night mode effectively.

3
00:00:11,130 --> 00:00:12,150
It's slightly different.

4
00:00:12,690 --> 00:00:18,150
It works with just a single image and night mode, actually, and most phones takes a series of different

5
00:00:18,270 --> 00:00:21,060
exposures of the image and then fuses them together.

6
00:00:21,840 --> 00:00:26,340
Anyway, we're going to use a minute to do something that actually, I think is really cool.

7
00:00:26,820 --> 00:00:29,970
So open Notebook 71 and we'll get started.

8
00:00:30,120 --> 00:00:33,870
So I really explain what this model is going to be doing.

9
00:00:34,410 --> 00:00:39,570
And if you want, I can actually scroll down all the way to the bottom to see the final result.

10
00:00:40,110 --> 00:00:42,380
And you can see how well it works.

11
00:00:42,390 --> 00:00:50,610
You can see this is the original image here can barely make out that whatever's on that image do on

12
00:00:50,610 --> 00:00:51,150
that photo.

13
00:00:51,780 --> 00:00:58,800
Now, if you use the pill, the pillow auto contrast feature, you can actually get this image, which

14
00:00:58,800 --> 00:01:02,460
is better, but you can recover so much information and learn.

15
00:01:02,460 --> 00:01:09,330
It almost looks like a regular daytime exposed photo similar to this bookshelf here and some of the

16
00:01:09,330 --> 00:01:12,270
city's other items here, which we'll talk about later on.

17
00:01:12,900 --> 00:01:14,280
So let's get started.

18
00:01:14,280 --> 00:01:18,120
Sorry for all the scrolling, and I can make you a bit dizzy sometimes.

19
00:01:18,720 --> 00:01:25,950
So we're going to be training on trading, on the L-O-L dataset, and that's not laugh-out-loud or League

20
00:01:25,950 --> 00:01:26,700
of Legends.

21
00:01:26,730 --> 00:01:31,830
It is the low light dataset that his example of what I just showed you.

22
00:01:32,250 --> 00:01:39,390
It consists of pairs of images where it's normally exposed and then the dark and all new light, low

23
00:01:39,390 --> 00:01:40,440
exposure image.

24
00:01:41,160 --> 00:01:46,200
So it learns basically how to map those low exposure images to brighten damage.

25
00:01:46,710 --> 00:01:48,060
So how do we do that?

26
00:01:48,090 --> 00:01:53,340
Well, it's actually quite complicated, but for now, let's just import all libraries and download

27
00:01:53,340 --> 00:01:57,420
our dataset and then create our TensorFlow dataset here.

28
00:01:57,900 --> 00:02:03,780
Just some utility functions and we will be using to load and create datasets here.

29
00:02:04,980 --> 00:02:06,930
Now we can talk about the that's model.

30
00:02:07,440 --> 00:02:11,190
So the model actually is quite complicated, in my opinion.

31
00:02:11,610 --> 00:02:18,540
It consists of a feature extractor that works across multiple skills, and then it has basically a mechanism

32
00:02:18,540 --> 00:02:24,750
to exchange and fuse this information as it passes through the network, performing representational

33
00:02:24,750 --> 00:02:25,260
learning.

34
00:02:25,950 --> 00:02:31,680
And then it actually basically actually filters on different feature maps as well.

35
00:02:32,340 --> 00:02:37,110
And it has a recursive design that progressively breaks down the input signal to simplify the learning

36
00:02:37,110 --> 00:02:37,740
process.

37
00:02:38,010 --> 00:02:42,720
So it's quite it's quite a good network, and you can see the architecture diagram here.

38
00:02:43,740 --> 00:02:45,780
Now we're going to talk about the different features.

39
00:02:45,780 --> 00:02:47,910
So this is the SKF module.

40
00:02:48,390 --> 00:02:53,910
This is the module that performance fusing and selecting of different global features descriptors.

41
00:02:54,450 --> 00:03:01,410
It basically looks at different resolutions here, multi resolutions or different skills and fuses them

42
00:03:01,410 --> 00:03:02,880
together afterward here.

43
00:03:02,970 --> 00:03:07,200
And you can read about it to learn if you want to learn more about how it works in detail.

44
00:03:08,070 --> 00:03:14,640
And this is a function that creates the feature of fusion, that selective kernel feature fusion module.

45
00:03:14,700 --> 00:03:22,140
That's what this block of code does next week come across the dual attention units or you, which is

46
00:03:22,140 --> 00:03:30,630
used to extract features, as well as acting like a filter to allow more informative features to pass

47
00:03:30,750 --> 00:03:32,040
further along the network.

48
00:03:32,730 --> 00:03:37,440
So this part consists of a channel, attention and spatial attention.

49
00:03:37,830 --> 00:03:42,030
And if you want to learn more, you can read this description of them there.

50
00:03:42,840 --> 00:03:47,340
Next time we talk, we come across two blocks of code that perform those things, so we have a special

51
00:03:47,340 --> 00:03:49,500
attention block in the channel attention block.

52
00:03:50,040 --> 00:03:54,450
Another dual attention, a block which combines both of them together, as you can see here.

53
00:03:55,380 --> 00:03:58,680
So now we come on to the multi scale residual block.

54
00:03:59,160 --> 00:04:06,960
So this book is capable of generating especially precise output by maintaining high resolution representations

55
00:04:07,410 --> 00:04:11,790
while still receiving the rich, contextual information from the lower resolutions.

56
00:04:12,210 --> 00:04:20,880
So this may be more obvious or maybe consist of multiple fully convolutional streams connected in parallel.

57
00:04:21,330 --> 00:04:26,250
So it allows information to be exchanged across these streams, which is quite cool.

58
00:04:26,910 --> 00:04:34,420
OK, so it employs actually recursive residual design with Skype connections to ease the flow of the

59
00:04:34,440 --> 00:04:35,940
information pooling process.

60
00:04:36,330 --> 00:04:43,950
So this is that block of residual modules here in code, and you can take a look if you want to understand

61
00:04:43,950 --> 00:04:44,820
it more deeply.

62
00:04:45,450 --> 00:04:53,160
This here's the entire multi-skilled residual block there that combines everything you can see the new

63
00:04:53,160 --> 00:04:56,160
SKF you to this here, too.

64
00:04:57,180 --> 00:04:59,700
And now finally, we have the functions that.

65
00:04:59,780 --> 00:05:00,920
Bring together everything.

66
00:05:01,280 --> 00:05:06,620
So this is a recursive group here, and this is a burden that model that combines the previous models

67
00:05:06,620 --> 00:05:07,340
that we created.

68
00:05:08,030 --> 00:05:14,930
So if we get the final bernet model here, so now we can start printing our network.

69
00:05:15,320 --> 00:05:24,380
So we're not using something so unusual a lost function called the year loss along with Adam Optimizer.

70
00:05:24,380 --> 00:05:31,490
And for these types of networks, we're actually we're using peak signal to noise ratio of PSR as a

71
00:05:31,490 --> 00:05:32,030
metric.

72
00:05:32,210 --> 00:05:35,210
So that's how we measure how well this network is performing.

73
00:05:35,930 --> 00:05:38,190
So we have those functions defined here.

74
00:05:38,300 --> 00:05:45,050
This is a piece in our function in TensorFlow right here that we can use and we create a optimizes and

75
00:05:45,050 --> 00:05:47,240
then just to model that flipped fit.

76
00:05:47,690 --> 00:05:49,860
And then we plot the results.

77
00:05:49,860 --> 00:05:54,470
So this model takes quite a while to train maybe a of a low of just over an hour.

78
00:05:55,520 --> 00:05:57,950
But you can see the end of the peak nodes.

79
00:05:58,190 --> 00:06:00,980
The peak signal to noise ratio is steadily going down.

80
00:06:01,250 --> 00:06:06,480
That's really going getting better, I should say, in validation.

81
00:06:06,620 --> 00:06:08,580
And in the training.

82
00:06:08,960 --> 00:06:10,070
So that's quite good.

83
00:06:10,850 --> 00:06:16,250
And we can take a look at our graphs and you can see the loss as well as apps and are increasing as

84
00:06:16,490 --> 00:06:17,510
epochs increase.

85
00:06:17,660 --> 00:06:21,830
So you can train for even longer and possibly get better results as well, although that seems sort

86
00:06:21,830 --> 00:06:23,600
of oscillating around here.

87
00:06:23,990 --> 00:06:25,490
So 50 might be the limit.

88
00:06:26,300 --> 00:06:29,150
Maybe we can do other network enhancements to improve that.

89
00:06:29,810 --> 00:06:36,200
So now we come across what we looked at in the beginning, which is some inferences here, so we can

90
00:06:36,200 --> 00:06:39,250
run some inferences on some test images and test images.

91
00:06:39,260 --> 00:06:43,010
There aren't a lot of test images in this dataset, so we just run to the field.

92
00:06:43,700 --> 00:06:45,740
But you can see how well it works.

93
00:06:45,740 --> 00:06:53,060
You can take this dark complex, and I think it's like all parkour, empty buildings in and then blowing

94
00:06:53,060 --> 00:06:53,430
up here.

95
00:06:53,450 --> 00:06:57,380
Nicole, could that look see actually extract all the colors accurately?

96
00:06:58,370 --> 00:07:00,470
It looks like a low resolution daytime photo.

97
00:07:01,250 --> 00:07:07,820
So does this cupboard with the pot still in this book can actually see the text and the in all the colors

98
00:07:07,820 --> 00:07:08,720
and the stapler.

99
00:07:09,500 --> 00:07:14,090
So you can see how well this is working and you can create this.

100
00:07:14,090 --> 00:07:20,900
It's an EPA API if you want it quite easily with F+, and that's what we're going to do next.

101
00:07:20,930 --> 00:07:23,930
Actually, sorry, actually going to do the capture of OCR tracker.

102
00:07:24,380 --> 00:07:31,220
But then after what we're going to use the Flask Framework Web Framework in Python to create an API

103
00:07:31,640 --> 00:07:32,720
as well as a web app.

104
00:07:32,960 --> 00:07:35,010
So I'll see you in the next few chapters.

105
00:07:35,030 --> 00:07:40,920
Also, the OCR tracker is actually a very cool chapter two, so stay tuned for that.

106
00:07:41,000 --> 00:07:41,480
Thank you.