1
00:00:00,270 --> 00:00:06,000
Hi and welcome back to the course in this section, we'll take a look at PyTorch Lightning, which is

2
00:00:06,000 --> 00:00:12,150
a feature rich, high level PyTorch interface library that makes working with PyTorch much easier.

3
00:00:12,390 --> 00:00:13,950
It makes it much more like us.

4
00:00:14,520 --> 00:00:15,610
So let's take a look at that.

5
00:00:15,690 --> 00:00:17,550
So what is PyTorch lighting?

6
00:00:17,640 --> 00:00:23,100
Well, it's an open source Python library that provides a high level interface for PyTorch.

7
00:00:23,580 --> 00:00:29,400
And here's basically a snapshot of all the cool features it offers here, all of which we'll touch on

8
00:00:29,400 --> 00:00:31,050
in the next coding lesson.

9
00:00:31,830 --> 00:00:33,600
So why use lightning, though?

10
00:00:34,140 --> 00:00:34,960
What's the point of it?

11
00:00:34,980 --> 00:00:38,970
I mean, yes, it's a high level, easy to use library, but it's so much more.

12
00:00:39,540 --> 00:00:45,060
It offers a nice, easy to use modular approach to cutting applied to edge models, so you become more

13
00:00:45,060 --> 00:00:51,450
organized and less focused on the Python coding side and more focused on creating your deep learning

14
00:00:51,450 --> 00:00:51,960
models.

15
00:00:52,560 --> 00:00:58,140
It was intended for researchers to focus more on the science and research and spend less time worrying

16
00:00:58,140 --> 00:01:00,930
about how they deploy the model or scale up during training.

17
00:01:01,650 --> 00:01:08,790
A model created using lightning can be trained on multiple GPUs quite easily, as well as CPUs and also

18
00:01:08,790 --> 00:01:13,680
with using floating point sixteen precision, which can aid inference time significantly.

19
00:01:14,220 --> 00:01:19,590
So therefore, the couple's research could from engineering, meaning that you have more time to focus

20
00:01:19,590 --> 00:01:22,410
on your model as opposed to the coding.

21
00:01:23,670 --> 00:01:30,700
And also, it integrates so easily with logging tools such as Tensor Bullet Ml Fluid Neptunian, a comedy

22
00:01:30,720 --> 00:01:36,000
and one which is basically weights and biases, which is a very cool app.

23
00:01:36,180 --> 00:01:41,310
All of these are very cool, actually, and we'll be using Tensor Void in our coding demonstrations.

24
00:01:41,940 --> 00:01:47,280
It also provides an ultimate selection, auto learning read selection, as well as the ability to use

25
00:01:47,280 --> 00:01:49,260
callbacks which you would have seen in Keros.

26
00:01:49,740 --> 00:01:52,050
So let's take a look at the Lightning philosophy.

27
00:01:52,620 --> 00:01:58,770
Lightning proposes that we increase all our models in building blocks into a lightning module.

28
00:01:59,460 --> 00:02:05,010
This is a high level diagram that basically shows this is an auto encoder, CNN, and basically shows

29
00:02:05,010 --> 00:02:08,160
what the lightning module does.

30
00:02:08,880 --> 00:02:14,310
It requires basically us to reorganize our existing PyTorch code, though, so let's take a look at

31
00:02:14,310 --> 00:02:14,580
that.

32
00:02:14,790 --> 00:02:22,140
So here's a minimal example of applied applied to watch lightning coding, and you can see it does look

33
00:02:22,140 --> 00:02:22,890
somewhat familiar.

34
00:02:22,920 --> 00:02:24,660
However, notice something here.

35
00:02:25,110 --> 00:02:27,330
There are five sections in this model.

36
00:02:27,570 --> 00:02:29,040
Is this lightning model?

37
00:02:29,850 --> 00:02:31,030
There's a competition here.

38
00:02:31,050 --> 00:02:32,130
This is our model here.

39
00:02:32,910 --> 00:02:35,730
It is a training loop which is down here, the training step.

40
00:02:36,330 --> 00:02:41,460
There's a validation step which isn't pictured in this diagram, unfortunately, but it's not another

41
00:02:41,460 --> 00:02:44,130
function that does the validation forward step.

42
00:02:44,730 --> 00:02:50,430
There's a test loop also from this diagram and then it optimizes, which are theme here and you can

43
00:02:50,430 --> 00:02:57,570
see the death in it, which we'll talk about in the next slide is quite similar to the existing PI itself.

44
00:02:57,810 --> 00:02:59,970
And you can see that here more clearly.

45
00:03:00,630 --> 00:03:05,910
This here is a pocket watch model, a simple, linear model.

46
00:03:05,910 --> 00:03:07,560
Here you can see it's just a neural network.

47
00:03:07,560 --> 00:03:09,390
There's no CNN's in this network.

48
00:03:09,930 --> 00:03:16,350
This layer one layer, two layer treed or all declared here in the deaf init function in this class.

49
00:03:17,250 --> 00:03:22,560
Remember, the soup of the super function here basically means you're inheriting everything from this

50
00:03:22,560 --> 00:03:23,790
class that you input here.

51
00:03:24,150 --> 00:03:29,520
So all the functions that are available in this class and then module class are inherited in this class

52
00:03:29,520 --> 00:03:33,300
now, which is a very cool, object oriented style programming.

53
00:03:34,200 --> 00:03:39,710
So now you can see we have two fold parts where we just run our model through the steps of pipeline,

54
00:03:39,720 --> 00:03:45,060
basically, if you will, the sequential steps that generate output to the end function coming from

55
00:03:45,060 --> 00:03:45,750
the South Max.

56
00:03:46,410 --> 00:03:48,450
And you can see the pie torch lighting module.

57
00:03:48,450 --> 00:03:51,300
It's pretty much exactly the same this part of the network, isn't it?

58
00:03:51,750 --> 00:03:56,630
The only difference here is besides the name is that we're inheriting now from the the pipe down the

59
00:03:56,640 --> 00:03:57,510
lightning module.

60
00:03:57,780 --> 00:04:00,150
So let's get started with this now.

61
00:04:01,050 --> 00:04:03,220
Let's take a look at the trainers.

62
00:04:03,240 --> 00:04:04,740
This is where the big changes are.

63
00:04:05,160 --> 00:04:09,540
And this is where lightning makes your life so much easier to remember.

64
00:04:09,720 --> 00:04:16,080
In PyTorch, we had to manually run our training loop using the actually calling the epochs the fourth

65
00:04:16,620 --> 00:04:21,840
epoch in number of epochs and then running to batch getting the data out of it and then doing the same

66
00:04:21,840 --> 00:04:23,010
thing for the validation loop.

67
00:04:23,570 --> 00:04:25,920
Well, lightning cleaned it up quite a bit.

68
00:04:26,520 --> 00:04:31,170
Lightning by using this def trading setup and validation step, which I would have shown you in the

69
00:04:31,170 --> 00:04:32,800
previous diagram, but there was some space.

70
00:04:33,420 --> 00:04:35,190
You can actually see how easy it is.

71
00:04:35,190 --> 00:04:39,420
You just have the training batch to batch it forward propagated here.

72
00:04:39,780 --> 00:04:42,570
Use across cross entropy loss logos.

73
00:04:42,570 --> 00:04:46,350
Things returned to lost functions, similar for the validation step.

74
00:04:46,890 --> 00:04:53,040
It's quite simple and quite easy, and it saves you so much coding the time and space here and it just

75
00:04:53,040 --> 00:04:54,150
works out of the box.

76
00:04:54,150 --> 00:04:59,850
Just cool your network like this, and then you can run a double model that fit and lightning.

77
00:05:00,370 --> 00:05:01,220
And it works.

78
00:05:01,240 --> 00:05:03,130
So here's some more changes as well.

79
00:05:04,090 --> 00:05:09,520
You can see this is the pie to its tail where we get the pie torch model that we created.

80
00:05:10,150 --> 00:05:14,330
Then you can set the optimizes like this here in pie torch lighting.

81
00:05:14,350 --> 00:05:19,210
You actually create a optimizer function inside of the class, and it's basically the same thing, the

82
00:05:19,210 --> 00:05:20,050
same line of code.

83
00:05:21,010 --> 00:05:23,980
Similarly, for the cross entropy loss, you do the same thing as well.

84
00:05:25,690 --> 00:05:27,880
So now this is the big, big deal here.

85
00:05:28,360 --> 00:05:33,850
So, no, you can see how we can condense all of this trading stuff here that's going on in the existing

86
00:05:33,850 --> 00:05:34,030
pie.

87
00:05:34,030 --> 00:05:37,060
Too much good and compress it basically into this here.

88
00:05:37,450 --> 00:05:41,470
So we have all pie torch model, lightning or pie torch lightning.

89
00:05:41,500 --> 00:05:43,930
So that's that's what this play is, Trina.

90
00:05:44,590 --> 00:05:48,310
And then we could do Trina that fit using our model.

91
00:05:48,550 --> 00:05:51,730
We created model class, and that's it.

92
00:05:52,180 --> 00:05:53,320
It's actually quite simple.

93
00:05:54,220 --> 00:05:55,900
So you can take a look at this here.

94
00:05:55,930 --> 00:05:59,710
This is also with the data modules, because you may have been wondering.

95
00:06:00,040 --> 00:06:01,960
So we've cleaned up the trading steps.

96
00:06:01,960 --> 00:06:06,070
We've made it much simpler and more, I guess module modular.

97
00:06:06,670 --> 00:06:11,350
However, what about the data loading part of it, despite all its lightning clean that up to?

98
00:06:11,350 --> 00:06:12,220
And yes, it does.

99
00:06:12,370 --> 00:06:19,060
So you can see before we had a transforms, we had to get amnesty to set, split and then get the test

100
00:06:19,060 --> 00:06:20,260
data, then create our data.

101
00:06:20,270 --> 00:06:20,730
Lueders.

102
00:06:21,280 --> 00:06:24,070
Now to us, lightning has made it much simpler as well.

103
00:06:24,400 --> 00:06:29,080
So don't worry if I'm not going into too much detail with this code, we'll do this in the coding lesson

104
00:06:29,080 --> 00:06:29,620
and colab.

105
00:06:30,250 --> 00:06:31,500
But let's take a look at this.

106
00:06:31,500 --> 00:06:37,010
So now there's a function that basically prepares the transforms to the standard, and this just gets

107
00:06:37,010 --> 00:06:37,780
to the data here.

108
00:06:38,410 --> 00:06:43,960
Then there's a trimmed data loader here where you have to transform, so we have to get this again.

109
00:06:44,470 --> 00:06:50,100
Then we have the random splits here and then the data loader here, which we create similarly for this

110
00:06:50,110 --> 00:06:51,760
file and to test as well.

111
00:06:52,330 --> 00:06:58,390
So it's this one isn't as simple as maybe the others, but it's it's quite it's a bit more organized.

112
00:06:59,380 --> 00:07:05,350
Also, PyTorch towards lightning offers something called lightning bolts, which is basically similar

113
00:07:05,350 --> 00:07:07,660
to torture vision, where it's a bunch of other ones.

114
00:07:08,080 --> 00:07:14,770
Things like state of the art models and communities and pre-trained models predict with more advanced

115
00:07:14,980 --> 00:07:17,590
callback functions, data sets, data modules.

116
00:07:18,070 --> 00:07:20,850
So it's quite high level and quite expensive.

117
00:07:20,860 --> 00:07:23,190
There's a lot more being added to this.

118
00:07:23,260 --> 00:07:28,620
So in the next lesson, we'll take a look at applied towards lightning exercises.

119
00:07:28,760 --> 00:07:35,470
We will convert our cats versus dogs by touch code into a lightning module will use the automatic size

120
00:07:35,470 --> 00:07:38,140
finder will also used auto learning.

121
00:07:38,140 --> 00:07:43,900
Read Selector will implement callbacks which are really stopping model, check pointing, logging as

122
00:07:43,900 --> 00:07:44,170
well.

123
00:07:44,410 --> 00:07:49,120
So I'll see you in the next section where we implement those things in code.

124
00:07:49,510 --> 00:07:49,930
Thank you.