1 00:00:00,750 --> 00:00:06,240 So you've trained a PyTorch convolutional neural network and you've trained a Keros with TensorFlow 2 00:00:06,570 --> 00:00:09,030 back end convolutional neural network. 3 00:00:09,750 --> 00:00:12,440 So you should have a good idea of the differences. 4 00:00:12,450 --> 00:00:18,660 However, let's do a quick recap and I'll explain the differences between PyTorch and Keras in a bit 5 00:00:18,660 --> 00:00:24,390 more detail and hopefully know that you actually have some hands on understanding of both libraries. 6 00:00:24,810 --> 00:00:27,390 This lesson will feel a bit better. 7 00:00:27,810 --> 00:00:31,350 It will reinforce what we've we've talked about before. 8 00:00:31,950 --> 00:00:33,450 So let's begin with TensorFlow. 9 00:00:33,630 --> 00:00:36,900 So when we used Keros, what did we do? 10 00:00:37,500 --> 00:00:45,390 Well, Keros, think of Keros as an API that builds upon TensorFlow, so it allows you to access and 11 00:00:45,390 --> 00:00:49,530 do and implement complex TensorFlow functions and scripting. 12 00:00:50,070 --> 00:00:54,210 However, without actually knowing the TensorFlow could, so it does it behind the scenes. 13 00:00:54,750 --> 00:00:59,610 So that should tell you that TensorFlow is a fairly low level library, and it is. 14 00:01:00,060 --> 00:01:05,340 When it first came out in 2015, I remember sitting down and going to the library, comparing it to 15 00:01:05,340 --> 00:01:09,900 the know, and I did find it a bit tricky to grasp all the concepts. 16 00:01:10,470 --> 00:01:15,660 However, Keros was a godsend at that time in 2016 when I started using it. 17 00:01:16,080 --> 00:01:22,290 It allowed you to build neural networks, so convolutional neural networks so much faster and create 18 00:01:22,290 --> 00:01:25,860 such powerful deep networks almost effortlessly. 19 00:01:26,700 --> 00:01:29,820 So you've seen this before, but I'll talk about it again. 20 00:01:30,210 --> 00:01:37,800 TensorFlow was written in C++ and uses Nvidia's CUDA Compute Package, which stands for Compute Unify 21 00:01:37,800 --> 00:01:44,400 Device Architecture API allows you to implement very complicated computational calculations on the GPU 22 00:01:44,430 --> 00:01:46,170 quite quickly and scalable. 23 00:01:47,010 --> 00:01:51,260 Also, it's primarily used in Python that is TensorFlow, not CUDA. 24 00:01:51,300 --> 00:01:56,010 I'm talking about and the other APIs or app is exist in other languages. 25 00:01:56,460 --> 00:02:00,070 But obviously C++, which is a direct sort of like API. 26 00:02:00,070 --> 00:02:08,640 It's a direct coding core Java, JavaScript and Swift and TensorFlow 2.0, which incorporated Kerberos, 27 00:02:09,090 --> 00:02:10,560 was released in 2019. 28 00:02:10,560 --> 00:02:13,050 Late 2019, if I'm if I'm not mistaken. 29 00:02:14,130 --> 00:02:22,390 So you've seen PyTorch and Page two, which basically came out a little bit after TensorFlow, maybe 30 00:02:22,470 --> 00:02:29,160 in about 2018 or so it became popular, and it's free and open source apps from Facebook iResearch research 31 00:02:29,160 --> 00:02:29,490 team. 32 00:02:29,820 --> 00:02:36,090 It's also Cbus Tensor Library Tensor, meaning that that's what the matrices are called. 33 00:02:36,090 --> 00:02:40,770 And when we're doing deep learning, we describe it as a tensor because that's what it is mathematically, 34 00:02:41,370 --> 00:02:45,300 and it's also written with CUDA capabilities, which makes it super fast as well. 35 00:02:45,990 --> 00:02:52,000 It's primarily meant for Python, although it's the C++ as well supported, and it's basically it's 36 00:02:52,050 --> 00:02:52,590 Python. 37 00:02:52,890 --> 00:02:59,010 This scientific computing package aimed at replacement for a nampai when using by using, allowing you 38 00:02:59,010 --> 00:03:04,980 to use the power of GPUs and also provides you with a very good deep learning research platform and 39 00:03:04,980 --> 00:03:07,070 PyTorch is actually very customizable. 40 00:03:07,080 --> 00:03:11,060 It's one of the key advantages in why it's taken over the research world. 41 00:03:11,070 --> 00:03:13,710 So much so. 42 00:03:14,040 --> 00:03:20,970 Keros Keros was developed by Francois Shuli, and basically he made a proposal explained to you before. 43 00:03:21,420 --> 00:03:26,020 That basically allows you to access to low level TensorFlow operations. 44 00:03:26,040 --> 00:03:28,530 Initially, it was TensorFlow A. A. They supported. 45 00:03:28,530 --> 00:03:34,110 The T.A. has been deprecated now, and it allows you to build deep networks so easily. 46 00:03:34,620 --> 00:03:42,930 It's written in Python for Python, and it's basically primarily meant to make TensorFlow easy to use. 47 00:03:43,500 --> 00:03:48,930 So and that's why in 2019, Keros is no bundled with TensorFlow 2.0. 48 00:03:50,700 --> 00:03:52,560 So let's compare them, though. 49 00:03:53,010 --> 00:04:00,920 So TensorFlow and Python, it's about fairly low level, meaning that they actually go into the nitty 50 00:04:00,990 --> 00:04:03,810 gritty detail of implementing each layer. 51 00:04:03,810 --> 00:04:06,900 You have specified input output sizes, activation functions. 52 00:04:07,200 --> 00:04:13,620 I mean, you do all of that in Keros, but they do take care a lot of a lot of things for you and Carrera's 53 00:04:13,620 --> 00:04:16,650 when comparing it to Pi Touch or TensorFlow. 54 00:04:17,520 --> 00:04:23,370 So Chris is simpler and definitely quicker to implement and also less prone to user errors, which is 55 00:04:23,370 --> 00:04:26,340 a good thing, especially when you're prototyping code. 56 00:04:27,480 --> 00:04:33,870 TensorFlow has definitely being first to market and supported by Google has gained wider adoption in 57 00:04:33,870 --> 00:04:35,550 the industry at large. 58 00:04:36,060 --> 00:04:42,090 However, I will tell you in 2021 right now, almost all companies I'm seeing switching are switching 59 00:04:42,090 --> 00:04:42,810 to by touch. 60 00:04:42,870 --> 00:04:50,130 It's remarkable how fast touch is gaining popularity, and the reason for that was because many state 61 00:04:50,130 --> 00:04:53,820 of the art networks that are done in research are built using PI Watch. 62 00:04:53,820 --> 00:04:58,800 And the only way to implement those on your production side is by using PI torture as well. 63 00:04:59,220 --> 00:04:59,970 I mean, you can. 64 00:05:00,030 --> 00:05:06,150 Could the same thing in TensorFlow, but it's often challenging when why do that, when the code is 65 00:05:06,150 --> 00:05:08,010 available on GitHub in Python already? 66 00:05:08,400 --> 00:05:11,400 So does this apply to watch in that case? 67 00:05:12,570 --> 00:05:18,240 So the myriad of differences between Keras and PyTorch is that this is a high level API that can use 68 00:05:18,240 --> 00:05:19,080 different buckets. 69 00:05:19,080 --> 00:05:23,010 However, right now it's almost exclusively going to use TensorFlow 2.0. 70 00:05:23,040 --> 00:05:28,350 No one is going to use the old divisions of care has to do with anyone else to have some legacy thing 71 00:05:28,350 --> 00:05:30,240 to support, which is unlikely. 72 00:05:31,020 --> 00:05:36,330 So Python to do is also fairly low level, and because of that, it requires a lot more. 73 00:05:36,330 --> 00:05:42,510 Could coding knowledge of Python to implement the same things that Karas does quite easily, which you've 74 00:05:42,510 --> 00:05:48,570 seen before, you've seen how easy it is to implement model that fits in Keros, whereas the actual 75 00:05:48,570 --> 00:05:52,830 training loop code in Python, which can become quite complicated. 76 00:05:53,880 --> 00:05:56,020 So but Python much to offer something here. 77 00:05:56,670 --> 00:06:00,540 It's a very python like language and scripting API. 78 00:06:01,080 --> 00:06:04,920 So if you're if you're a good Python developer, you're going to find PI to. 79 00:06:04,920 --> 00:06:07,050 It's very easy to work with and I do as well. 80 00:06:07,050 --> 00:06:07,830 It's quite nice. 81 00:06:08,340 --> 00:06:10,590 So let's take a look at some of the code here. 82 00:06:11,250 --> 00:06:17,430 You can see when recruiting a PI PyTorch model how you have to enforce and inherit from the super, 83 00:06:17,570 --> 00:06:24,360 using super from the fighters in that class, then you have to define your filters here or your convolutional 84 00:06:24,360 --> 00:06:25,950 is your max boot. 85 00:06:25,980 --> 00:06:31,890 All of these things here specified input and output sizes, while the input sizes for each layer see 86 00:06:32,250 --> 00:06:33,360 the output sizes. 87 00:06:33,930 --> 00:06:37,170 So it is a bit trickier to do in Python. 88 00:06:37,830 --> 00:06:43,890 It's not as intuitive as carers and carers because you just you can model that ad using the sequential 89 00:06:43,890 --> 00:06:44,280 model. 90 00:06:44,850 --> 00:06:49,530 It's quite easy just to link layers together, and you don't even have to know about the output size. 91 00:06:49,890 --> 00:06:55,070 It carries off automatically passes that information onto the next layer, whereas PyTorch doesn't. 92 00:06:56,250 --> 00:06:58,200 So that's basically what I just said. 93 00:06:58,200 --> 00:07:05,160 Here it is in point form, you can definitely see carers obstructs us away from the specifics and quitting 94 00:07:05,160 --> 00:07:10,740 a lot, so it's quite easy to add more layers deeper and easier, but less good. 95 00:07:11,460 --> 00:07:16,860 Whereas carers is simpler and modular, PyTorch, you can see, has a lot more stuff going on. 96 00:07:18,300 --> 00:07:20,730 So why would you use Python watching? 97 00:07:21,670 --> 00:07:25,080 Well, to me, PyTorch has definitely been more flexible. 98 00:07:25,410 --> 00:07:30,870 So if you're doing some intricate state of the art deep learning, using implementing some deep learning 99 00:07:30,870 --> 00:07:34,290 techniques, PyTorch will almost always be the way to go. 100 00:07:34,830 --> 00:07:39,420 But it also does encourage you to have a stronger understanding of deep learning, which is a very good 101 00:07:39,420 --> 00:07:39,810 thing. 102 00:07:39,810 --> 00:07:46,410 And as I said, the popularity of PyTorch is growing very, very quickly, especially in the academia 103 00:07:46,410 --> 00:07:47,520 and research world. 104 00:07:48,540 --> 00:07:52,130 So here's a comparison table of carers first PyTorch. 105 00:07:52,770 --> 00:07:55,950 I'm not going to read this for you if you want, you can pause this right now. 106 00:07:56,280 --> 00:07:58,470 However, I will just discuss some things here. 107 00:07:58,710 --> 00:08:03,210 Firstly, as you've seen, carers is easier to use in speed. 108 00:08:03,240 --> 00:08:04,130 They're both equal. 109 00:08:04,140 --> 00:08:10,850 However, PyTorch has easier multi-GPU support, so I tend to use it more often when treating it works 110 00:08:10,860 --> 00:08:11,250 right now. 111 00:08:11,760 --> 00:08:17,970 Because of that, debugging is quite good, and they have both very good documentation as well. 112 00:08:18,690 --> 00:08:24,600 Customisation is less obviously what carers buy, which is quite good and you're constantly adding new 113 00:08:24,600 --> 00:08:25,140 features. 114 00:08:25,470 --> 00:08:32,910 Facebook's I-Team does a very good job of keeping Python up to date features by which probably wins 115 00:08:32,910 --> 00:08:33,210 there. 116 00:08:33,600 --> 00:08:36,630 However, carers is quite good, so don't take it for granted. 117 00:08:37,090 --> 00:08:40,590 Carers is very, very well supported with a lot of features. 118 00:08:40,590 --> 00:08:46,560 So even though I say Python, which allows you to do more experimental state of the art stuff, although 119 00:08:46,800 --> 00:08:53,450 a lot of these things end up in carers just a little bit later in terms of reach, this is hard to see 120 00:08:53,460 --> 00:08:53,940 right now. 121 00:08:54,530 --> 00:08:59,220 I would say they're about neck and neck in equal input production and academia. 122 00:08:59,640 --> 00:09:08,010 Although I do think by two which has the advantage in academia right now and support, there is tons 123 00:09:08,010 --> 00:09:15,000 of libraries that support other high level frameworks that support what I touch and a lot do in the 124 00:09:15,000 --> 00:09:18,690 TensorFlow as well care as being one of them deployment. 125 00:09:18,710 --> 00:09:26,340 I do realise that deploying models been built in TensorFlow tends to be easier, but again, that is 126 00:09:26,340 --> 00:09:29,040 changing quickly as more people are using PyTorch. 127 00:09:30,660 --> 00:09:32,400 So when would you use PyTorch? 128 00:09:32,940 --> 00:09:38,490 Well, if you want more visibility over the training aspects of your network, Python, which gives 129 00:09:38,490 --> 00:09:45,150 you that, so a lot of researchers will train custom exotic models using Python precisely for that reason. 130 00:09:45,750 --> 00:09:53,810 So if you want to also alter an existing model and add some tweaks, experiment with some of little 131 00:09:53,820 --> 00:09:59,220 new blocks of CNN offer deep learning layers PEIDA, which is quite good for that. 132 00:09:59,850 --> 00:10:04,140 And if you're quite good at Python, then pretty much would be pretty easy for you to use. 133 00:10:04,650 --> 00:10:06,630 So when would you use Keros? 134 00:10:07,020 --> 00:10:13,020 Keros is great for quick prototyping when I just want to get out a quick classifier of something. 135 00:10:13,290 --> 00:10:19,860 I mean, I do use my torch now to be fair, but Keros has also always been so easy just to train some 136 00:10:20,430 --> 00:10:23,430 simple classifier or something simple as well. 137 00:10:23,430 --> 00:10:29,640 So Keros has is definitely quicker, and it's definitely easier to for beginners. 138 00:10:30,120 --> 00:10:35,370 So because you don't have to have as deep as in-depth Python knowledge as you do with Byddwch. 139 00:10:35,850 --> 00:10:38,250 So Kerry's definitely wins in that sense. 140 00:10:38,820 --> 00:10:41,400 It also because it supports TensorFlow. 141 00:10:41,730 --> 00:10:43,200 It's going to be quite fast. 142 00:10:43,200 --> 00:10:46,680 It's going to be so many external tools that support TensorFlow functions. 143 00:10:47,010 --> 00:10:52,200 Deployment is going to be quite easy as well because so many things support TensorFlow models. 144 00:10:52,200 --> 00:10:54,320 So Keros is quite good. 145 00:10:54,330 --> 00:10:57,660 You're not going to go wrong using either of these two libraries, basically. 146 00:10:59,070 --> 00:11:02,340 So in terms of popularity, these are the Google Trends analysis. 147 00:11:02,340 --> 00:11:05,160 This is up to 2020 and late 2020. 148 00:11:05,760 --> 00:11:08,580 You can see Keros is definitely popular. 149 00:11:08,970 --> 00:11:14,040 TensorFlow has been kind of going down slightly, but Python, which has been remarkably popular just 150 00:11:14,040 --> 00:11:15,240 a little bit in here. 151 00:11:15,630 --> 00:11:17,070 But they're both neck and neck. 152 00:11:18,330 --> 00:11:26,190 And in terms of research publications, you can see who is using what in different research conferences 153 00:11:26,190 --> 00:11:26,610 right now. 154 00:11:27,030 --> 00:11:34,530 So A.B. and I see a Lawrence Clipper would be the top computer vision conferences, and you can see 155 00:11:34,620 --> 00:11:35,740 who's using PI torch. 156 00:11:36,210 --> 00:11:37,950 Most people are using PI to watch right now. 157 00:11:38,400 --> 00:11:39,000 Definitely. 158 00:11:39,270 --> 00:11:41,490 It's definitely a good sign. 159 00:11:41,790 --> 00:11:43,830 Solid lines, I thought so. 160 00:11:43,830 --> 00:11:46,560 Pi Dodge, as I said, is quite strong in the academia. 161 00:11:46,600 --> 00:11:52,170 Will all of these solid lines are pointing up, whereas all the dotted lines are sort of remaining constant? 162 00:11:53,730 --> 00:11:56,130 So why aren't we learning pure TensorFlow? 163 00:11:56,730 --> 00:11:58,680 Well, it's even more low level and pi touch. 164 00:11:59,040 --> 00:12:05,940 And to be honest, it's so it's so rare to see in that were acquitted in pure TensorFlow right now. 165 00:12:06,330 --> 00:12:08,910 Maybe only in academia, academia or research papers. 166 00:12:08,910 --> 00:12:09,510 You would see it. 167 00:12:09,930 --> 00:12:12,720 Everyone is using cameras because it's effectively the same thing. 168 00:12:13,680 --> 00:12:21,660 It does require a bit more hands-on, working around than PyTorch as well, and others have cameras 169 00:12:21,660 --> 00:12:26,880 provides 95 percent of the deep learning capabilities of TensorFlow, especially in computer vision. 170 00:12:27,300 --> 00:12:33,060 They have implemented so many complex algorithms with cameras and computer vision, so there's no need 171 00:12:33,060 --> 00:12:35,580 to actually code these things with TensorFlow. 172 00:12:36,570 --> 00:12:41,370 And learning Keras and PyTorch is a very good idea. 173 00:12:41,970 --> 00:12:42,990 It's not going to. 174 00:12:43,410 --> 00:12:48,330 It's not only going to open up your job prospects, but it's going to enable you to learn so much quicker. 175 00:12:48,360 --> 00:12:56,340 Because, as I said, half of the market right now in both academia and production is split between 176 00:12:56,550 --> 00:13:01,500 cameras and TensorFlow carries TensorFlow and PyTorch so little. 177 00:13:01,500 --> 00:13:06,870 But you're going to it's not going to be you're not going to go wrong learning, but definitely not. 178 00:13:08,610 --> 00:13:12,990 And many companies simply just use cameras with fatal logic in my company. 179 00:13:13,350 --> 00:13:17,820 Actually, the past two or three companies have looked when no one really cares too much what library 180 00:13:17,820 --> 00:13:18,120 use. 181 00:13:18,120 --> 00:13:22,470 But a lot of companies are standardizing on PyTorch like my company as well. 182 00:13:22,920 --> 00:13:27,230 So it's keep us in mind, so we'll stop there. 183 00:13:27,240 --> 00:13:33,870 And in the next section, we'll take a look at assessing model performance, which is a very important 184 00:13:33,870 --> 00:13:34,350 thing to do. 185 00:13:34,350 --> 00:13:39,210 So after you train your model, you need to understand how to analyze its performance metrics. 186 00:13:39,390 --> 00:13:41,160 So I'll see you in the next lesson. 187 00:13:41,490 --> 00:13:41,880 Thank you.