1
00:00:00,120 --> 00:00:06,390
Hi and welcome to our lesson on deep learning libraries, deep learning libraries are the tools we use

2
00:00:06,390 --> 00:00:08,760
to actually create our deep cleaning networks.

3
00:00:09,210 --> 00:00:15,750
They're effectively APIs which stand for application programming interfaces, which allow us to access

4
00:00:15,750 --> 00:00:21,270
all of the low level code with high level functions, which you'll see shortly and make deep learning

5
00:00:21,270 --> 00:00:23,700
much easier, much more accessible for us.

6
00:00:24,450 --> 00:00:30,570
So deep learning to implement this when a computer is actually quite difficult, there's a lot of moving

7
00:00:30,570 --> 00:00:33,000
parts and there's a massive amount of calculations.

8
00:00:33,510 --> 00:00:37,860
It just takes a lot of time and it can be prone to breaking because there's so many things to think

9
00:00:37,860 --> 00:00:38,190
about.

10
00:00:38,790 --> 00:00:39,950
So it's a lot.

11
00:00:40,020 --> 00:00:44,790
And then there's a lot of complexity in implementing special features like special lost functions and

12
00:00:44,790 --> 00:00:45,570
optimizes.

13
00:00:46,170 --> 00:00:49,170
So basically, don't try this at all.

14
00:00:50,220 --> 00:00:51,510
However, actually, you should.

15
00:00:52,230 --> 00:00:52,470
What a.

16
00:00:53,100 --> 00:00:59,820
Don't try making your own deep learning library with low level code because it's already been done.

17
00:01:00,510 --> 00:01:01,110
That's what I need.

18
00:01:01,620 --> 00:01:06,450
You can actually, when I said you can try at home, it's that you can actually use Python a number

19
00:01:06,450 --> 00:01:11,370
and try to create your own deep learning library purely as a learning exercise.

20
00:01:11,970 --> 00:01:17,160
You don't actually, you're not going to actually use this in real life because these depleting libraries

21
00:01:17,160 --> 00:01:22,620
like PyTorch, TensorFlow and Keras will us as part of TensorFlow effectively, which we'll talk about

22
00:01:22,620 --> 00:01:23,070
soon.

23
00:01:23,550 --> 00:01:25,120
They're so well developed.

24
00:01:25,140 --> 00:01:29,550
We have so many engineers, probably thousands of engineers working on these libraries.

25
00:01:30,000 --> 00:01:36,060
So the reason it's a very high standard in efficiency and accessibility and features.

26
00:01:36,750 --> 00:01:44,790
So the two main ones that we're going to talk about and discuss is TensorFlow and PyTorch know TensorFlow

27
00:01:44,820 --> 00:01:51,180
was developed by Google, actually, and it's been released maybe since 2014 to maybe around then maybe

28
00:01:51,180 --> 00:01:51,510
before.

29
00:01:51,550 --> 00:01:55,050
I'm not entirely sure which was more recently released.

30
00:01:55,050 --> 00:02:00,930
I think it's in 2016 by Facebook, and these guys have made so many updates and revisions.

31
00:02:00,930 --> 00:02:05,790
TensorFlow is on 2.0, 2.0, 2.5 or 2.6 right now.

32
00:02:06,090 --> 00:02:12,690
PI touches on one point nine one revisions to so many visions and so many good features have added their

33
00:02:12,690 --> 00:02:14,360
amazing, amazingly fast.

34
00:02:14,370 --> 00:02:19,620
They're quite mature and reliable and stable and is very, very good support for both of them.

35
00:02:19,620 --> 00:02:21,180
Online does good documentation.

36
00:02:21,420 --> 00:02:28,020
There's good forums, communities that help each other, researchers and generally I would stick to

37
00:02:28,020 --> 00:02:28,830
using these two.

38
00:02:28,990 --> 00:02:31,650
Now you may have heard me mention Keros.

39
00:02:31,900 --> 00:02:33,600
Well, what exactly is Keros?

40
00:02:34,050 --> 00:02:40,680
Well, Keras is I'll talk about it a bit more detail, but Caris is basically an abstraction for TensorFlow,

41
00:02:41,160 --> 00:02:47,640
and it actually has been incorporated into TensorFlow 2.0, and it makes using TensorFlow Library much

42
00:02:47,640 --> 00:02:49,980
easier and adds a lot of features as well.

43
00:02:50,430 --> 00:02:56,280
So the TensorFlow guys and Google decided, let's just merge these two libraries together and get to

44
00:02:56,310 --> 00:02:57,420
get the best of both worlds.

45
00:02:58,380 --> 00:03:01,620
So let's talk a bit about why we're using these two libraries.

46
00:03:02,040 --> 00:03:03,600
Look at this graph on the top right here.

47
00:03:04,020 --> 00:03:12,690
You can see basically from maybe in the mid 2000s, there was almost like a drastic increase in interest

48
00:03:12,810 --> 00:03:13,650
in TensorFlow.

49
00:03:14,070 --> 00:03:19,060
And that's because TensorFlow was the first major deep learning library that was open source.

50
00:03:19,080 --> 00:03:23,970
There were others like Emacs in that Cafe Tejano, which I used to use.

51
00:03:23,970 --> 00:03:27,180
T.A. was quite nice and had some very good documentation, too.

52
00:03:27,810 --> 00:03:33,540
However, they were still never fully really well developed, and then TensorFlow came along, which

53
00:03:33,810 --> 00:03:38,220
was backed by Google, and basically interest in it skyrocketed.

54
00:03:38,610 --> 00:03:44,700
However, Keros, which was an abstraction that made TensorFlow much easier, also got popular simultaneously

55
00:03:44,700 --> 00:03:45,630
with TensorFlow.

56
00:03:46,200 --> 00:03:50,160
And then we also had PyTorch, which was released looks like 2016.

57
00:03:51,150 --> 00:03:56,640
This one became immensely popular as well, and as many reasons why PI Torch became so popular.

58
00:03:57,090 --> 00:04:00,720
It it basically it's a lot more python it condemns of.

59
00:04:01,050 --> 00:04:06,480
If you're experience Python developer, you would tend to prefer how PyTorch works, whereas TensorFlow

60
00:04:06,930 --> 00:04:14,700
kind of has its own function, coding methodology and style, which isn't as well suited for Python

61
00:04:14,700 --> 00:04:15,240
programmers.

62
00:04:15,240 --> 00:04:16,940
But I mean, I've gotten used to both.

63
00:04:16,950 --> 00:04:18,480
I don't have any issues with using either.

64
00:04:19,020 --> 00:04:23,880
I do prefer Pi Torch now, but initially I always did prefer Carris and TensorFlow because it's just

65
00:04:23,880 --> 00:04:28,020
it's just so easy to get up and running and there are others, too.

66
00:04:28,050 --> 00:04:34,740
Like I mentioned, Yano, which is no longer supported as deprecated, though Caffey, which was a research

67
00:04:34,770 --> 00:04:41,460
language coming out of the University of Berkeley that was basically it's for fodder for my thoughts

68
00:04:42,090 --> 00:04:46,020
and then is a mixed nut and paddle paddle, which is a Chinese deep learning library.

69
00:04:46,380 --> 00:04:50,220
And there are a couple of the Chinese deep learning libraries, which we don't hear too much about in

70
00:04:50,220 --> 00:04:53,490
the West, but they're actually quite good as well.

71
00:04:54,390 --> 00:04:58,870
And you can see in this little graph fire, these are the many, many different tools.

72
00:04:58,890 --> 00:04:59,750
Is it a deep learning one?

73
00:04:59,850 --> 00:05:03,630
And painkillers that make up it easy to get essentially.

74
00:05:04,230 --> 00:05:06,930
I mean, there's so many, these are just the most popular ones.

75
00:05:07,440 --> 00:05:11,820
But you can see in terms of the deplaning libraries, there are all of these here and there are more

76
00:05:11,910 --> 00:05:12,780
right now as well.

77
00:05:13,200 --> 00:05:18,280
And rapper libraries, which you've seen Chris mentioned here, Chris is a rapper.

78
00:05:18,300 --> 00:05:23,840
It's basically a rapper or obstruction over high level of obstruction over TensorFlow.

79
00:05:24,150 --> 00:05:26,690
So let's talk about TensorFlow now.

80
00:05:26,730 --> 00:05:28,170
TensorFlow is free and open source.

81
00:05:28,170 --> 00:05:33,870
Like all of the as we just mentioned, it's developed internally by Google Brain and released in 2015.

82
00:05:34,380 --> 00:05:37,090
It's written in C++ and CUDA could.

83
00:05:37,260 --> 00:05:41,940
It was also written in C++, too, which means that it's can perform.

84
00:05:41,940 --> 00:05:48,870
It can take advantage of Nvidia's GPUs, and C++ is quite efficient language, so it's very fast.

85
00:05:49,530 --> 00:05:52,080
And yes, it's primarily used in Python.

86
00:05:52,110 --> 00:05:56,860
However, there are other API wrappers that exist for these popular languages as well.

87
00:05:56,880 --> 00:06:02,280
However, pretty much like ninety five percent of the work done into TensorFlow is going to be done

88
00:06:02,280 --> 00:06:09,750
in Python and Version 2.0, which basically made a library be kind of the kind of consolidated a lot

89
00:06:09,750 --> 00:06:10,890
of things in a library.

90
00:06:11,310 --> 00:06:13,590
They standardize things, so a lot of features.

91
00:06:13,590 --> 00:06:17,640
A lot of things were dropped, kind of, but it were incorporated in a new way.

92
00:06:18,270 --> 00:06:23,670
There was also the inclusion of tens of Keros in it, which was quite good.

93
00:06:24,870 --> 00:06:31,770
So let's talk about PyTorch now, also free and open sourced, developed by Facebook's research team.

94
00:06:32,130 --> 00:06:37,020
Similarly, sea based Tensor Library, which could have capabilities as well.

95
00:06:37,030 --> 00:06:43,440
So it's just as fast as TensorFlow, although in my experience I do find it sometimes a tad slower.

96
00:06:43,890 --> 00:06:45,960
But then again, it depends.

97
00:06:45,960 --> 00:06:50,040
There's many factors that affect speed, so it's not just about deep learning library that you're using,

98
00:06:51,120 --> 00:06:53,490
it is primarily meant again for Python.

99
00:06:53,910 --> 00:07:00,120
However, C++ is pretty well supported, though I don't know anyone personally who uses Python and C++.

100
00:07:00,690 --> 00:07:02,330
So Python as a python.

101
00:07:02,340 --> 00:07:04,020
Basically, this is what the definition is.

102
00:07:04,470 --> 00:07:05,190
It's a python.

103
00:07:05,190 --> 00:07:11,130
Be a scientific computing package in the replacement for you pay using the power of GPUs Nampai as a

104
00:07:11,130 --> 00:07:13,200
numerical processing library in Python.

105
00:07:13,200 --> 00:07:17,280
So it's it's basically a GPU successor in a way of it.

106
00:07:17,820 --> 00:07:23,340
And basically, it's become very, very popular with the deep learning research community, mainly because

107
00:07:23,340 --> 00:07:27,330
it is very, very flexible and easy to tweak and debug.

108
00:07:27,840 --> 00:07:29,250
And it's quite fast as well.

109
00:07:29,970 --> 00:07:31,710
So let's talk about Carousel.

110
00:07:33,000 --> 00:07:34,650
Curse was developed by Francois.

111
00:07:34,650 --> 00:07:39,980
Actually, it's a free, open source library, and initially it was supported to beckons.

112
00:07:40,050 --> 00:07:43,470
The back back in a basic, low level is under Keros.

113
00:07:43,890 --> 00:07:45,750
TensorFlow was one and antigeno.

114
00:07:45,990 --> 00:07:51,060
I used to use Dano's, as I said quite a bit before TensorFlow and Tianna was quite nice.

115
00:07:52,380 --> 00:07:58,080
Chris has written in Python for Python, which means that it takes about Python type approach to things.

116
00:07:58,950 --> 00:08:05,040
It's primarily meant to make TensorFlow more accessible, so it obstructs a lot of the low level work

117
00:08:05,040 --> 00:08:09,630
that you would do in TensorFlow and kind of like build them in the background, but you don't see it.

118
00:08:09,810 --> 00:08:16,170
It's all done magically by carers, and it's it's contained several limitations.

119
00:08:16,180 --> 00:08:17,400
It's very advanced, actually.

120
00:08:17,400 --> 00:08:22,140
Read now, it's very customizable, not as customizable as PyTorch, but it's quite good.

121
00:08:23,010 --> 00:08:27,950
And like I said, it's now bundled with TensorFlow and the two point over two point excretions.

122
00:08:29,040 --> 00:08:34,320
So just to quickly summarize both TensorFlow and potpie touch a fairly low level.

123
00:08:34,800 --> 00:08:39,480
However, in my opinion, PyTorch is not as low level as TensorFlow.

124
00:08:39,810 --> 00:08:46,560
In some ways, and it's actually quite quite nice to use Carrera's is quite simple and quick to get

125
00:08:46,560 --> 00:08:47,500
to get up and running.

126
00:08:47,520 --> 00:08:48,540
It's quite fast.

127
00:08:49,290 --> 00:08:55,620
You can build fairly complex models with much less code sometimes, and it's actually easier for beginners

128
00:08:55,620 --> 00:09:00,060
to grasp it carries than by Python, which you will see in the next section, actually.

129
00:09:00,750 --> 00:09:06,420
TensorFlow, though, has wider adoption in the industry, mainly because it was the first to market

130
00:09:06,420 --> 00:09:06,870
in the way it was.

131
00:09:07,020 --> 00:09:12,360
It was the first big deep learning library that came out, so everyone who has all the companies that

132
00:09:12,360 --> 00:09:17,130
were in AI, maybe in the late 2020s, certainly the 2010s.

133
00:09:17,400 --> 00:09:22,620
Whatever you call that era, they used TensorFlow and a lot of them are switching right now.

134
00:09:22,660 --> 00:09:29,010
Like the company I work with actually uses PyTorch primarily, although we're not adverse to using TensorFlow.

135
00:09:29,460 --> 00:09:36,060
It's just that we're sort of standardize on PyTorch and Python, which has gained its popularity mainly

136
00:09:36,060 --> 00:09:40,460
due to research in academia because Cafe was a model of choice for them.

137
00:09:40,470 --> 00:09:46,800
The library of choice for them previously and PyTorch took a lot of things from Cafe, developed it

138
00:09:46,800 --> 00:09:49,080
and made it much more accessible now.

139
00:09:50,250 --> 00:09:55,770
So if you're interested in comparing these deep learning libraries, Wikipedia has a little link here.

140
00:09:55,770 --> 00:09:59,580
You can see comparisons probably more to date than the screenshot to.

141
00:09:59,710 --> 00:10:04,240
I know, but you can take a look and compare them with features, see what interface they were written

142
00:10:04,240 --> 00:10:06,310
and see what they support.

143
00:10:06,990 --> 00:10:08,260
See who's backing them.

144
00:10:08,860 --> 00:10:12,010
And you Typekit give you leverage support as well.

145
00:10:12,670 --> 00:10:16,390
So it's it's good to keep track of what's going on in the world because you never know.

146
00:10:16,720 --> 00:10:20,530
There can be another groundbreaking deep learning library that changes everything.

147
00:10:20,620 --> 00:10:21,130
Who knows?

148
00:10:22,300 --> 00:10:26,200
So we'll stop there, and now we'll get into the practical part of it.

149
00:10:26,620 --> 00:10:30,670
We're now going to cut our first CNN Impi.

150
00:10:31,390 --> 00:10:32,240
So stay tuned.

151
00:10:32,590 --> 00:10:39,460
And also, I'll show that there's a reason we start with PyTorch, mainly because it's hotter, but

152
00:10:39,460 --> 00:10:44,740
also because it follows the sort of methodology and design of CNN.

153
00:10:44,740 --> 00:10:51,760
So you can actually see it in a more granular way as opposed to tariffs, which will obstruct and make

154
00:10:51,760 --> 00:10:53,200
it seem almost too easy.

155
00:10:53,770 --> 00:10:55,210
So stay tuned for that lesson.

156
00:10:55,420 --> 00:10:55,870
Thank you.