1 00:00:00,240 --> 00:00:06,750 Hello, everyone, and welcome to another amazing session in which we are going to see how to work with 2 00:00:06,750 --> 00:00:12,270 weights and biases and integrated with our already existing TensorFlow code. 3 00:00:12,270 --> 00:00:18,930 Base weights and biases help practitioners in experiment tracking collaborative reports, dataset and 4 00:00:18,930 --> 00:00:26,040 model versioning, Interactive data visualization and hyper parameter optimization is trusted by over 5 00:00:26,040 --> 00:00:30,690 100,000 plus machine learning practitioners around the world. 6 00:00:30,720 --> 00:00:34,310 In this session, we are going to focus on experiment tracking. 7 00:00:34,320 --> 00:00:40,290 It's one thing to build a model, train this model and evaluate it on a given dataset. 8 00:00:40,290 --> 00:00:45,090 And at least as we've seen throughout this course, this is pretty easy with TensorFlow. 9 00:00:45,090 --> 00:00:52,050 But when we work in large teams and we have to collaborate, we have to produce reproducible results, 10 00:00:52,050 --> 00:00:58,950 we need to debug this ML models as a team and we also need to enforce transparency. 11 00:00:58,950 --> 00:01:06,210 Then a machine learning operations platform like weights and biases becomes indispensable weights and 12 00:01:06,210 --> 00:01:07,020 biases. 13 00:01:07,020 --> 00:01:14,880 Parameters build better models faster with experiment tracking that revisioning and model management. 14 00:01:14,880 --> 00:01:22,800 As of now, the different products which weights and biases offers to us our experiments tracking reporting 15 00:01:22,800 --> 00:01:28,560 that is producing collaborative dashboards, artifacts, dataset and model versioning. 16 00:01:28,560 --> 00:01:37,530 Just like how you would do code versioning in Git, our interactive data visualization and hyper parameter 17 00:01:37,530 --> 00:01:38,610 optimization. 18 00:01:38,610 --> 00:01:45,780 Now, also, you could see that this is being used by modern 100,000 machine learning practitioners 19 00:01:45,780 --> 00:01:46,650 around the world. 20 00:01:46,650 --> 00:01:52,440 Some key aspects of this weights and bias tool are the fact that you could integrate very quickly. 21 00:01:52,440 --> 00:01:54,180 You see that with Keras. 22 00:01:54,180 --> 00:02:00,480 So here we suppose, and we are building a Keras model, a TensorFlow Keras model, and that all you 23 00:02:00,480 --> 00:02:08,550 need to do is to impart this one DB call back right here, start a new run as is given your if you have 24 00:02:08,550 --> 00:02:15,240 some configurations, you set that those configurations and then in the place of the callbacks you simply 25 00:02:15,240 --> 00:02:19,650 pass this weights in bias callback which we import it right here. 26 00:02:19,650 --> 00:02:24,630 So it's quite easy to integrate with already existing all frameworks. 27 00:02:24,630 --> 00:02:31,760 Now you see with any framework you just need to do one DB log and you could log any information you 28 00:02:31,770 --> 00:02:32,400 want to log. 29 00:02:32,400 --> 00:02:39,120 We could also visualize useful information very seamlessly and then we could collaborate in real time. 30 00:02:39,120 --> 00:02:46,890 So your if you're working on a project, you could all as part of the team, discuss the project's progression 31 00:02:46,890 --> 00:02:51,480 and see how to eliminate any bugs or any problems. 32 00:02:51,480 --> 00:02:54,660 Now, weights and biases is designed for all use cases. 33 00:02:54,660 --> 00:02:59,250 So here we have a practitioner that's we suppose not just a single person. 34 00:02:59,250 --> 00:03:06,210 You have this dashboard central dashboard you can look at, you see this hyper parameter sweeps artifacts. 35 00:03:06,210 --> 00:03:11,220 Your you could do data set and model versioning just like you would do it. 36 00:03:11,220 --> 00:03:18,510 GitHub code reports to share updates very transparently throughout this course will look at these different 37 00:03:18,510 --> 00:03:19,200 products. 38 00:03:19,200 --> 00:03:23,100 And in this section we'll focus on experiments tracking. 39 00:03:23,100 --> 00:03:25,860 We're now going to go straight forward into signing up. 40 00:03:25,860 --> 00:03:33,780 So we click right here, want to sign up with GitHub, click here, and then we authorize one DB to 41 00:03:33,780 --> 00:03:36,120 get access to our neural learning account. 42 00:03:36,120 --> 00:03:44,130 So authorize one to be we have the full name organization and you will learn and that's fine. 43 00:03:44,190 --> 00:03:46,650 Okay, So I agree to the terms and conditions. 44 00:03:46,650 --> 00:03:51,960 You could always read out the terms and conditions very carefully. 45 00:03:51,960 --> 00:03:56,220 And then from here we also have to put in this username. 46 00:03:56,220 --> 00:04:03,840 So let's say neural learn and okay, so we continue how often do you train models, let's say every 47 00:04:03,840 --> 00:04:04,500 week. 48 00:04:04,500 --> 00:04:07,080 So we have that and then get started. 49 00:04:07,110 --> 00:04:08,550 Here is now homepage. 50 00:04:08,630 --> 00:04:15,900 You could create a new project, you could modify a profile, invite your team, you have this documentation, 51 00:04:15,900 --> 00:04:18,390 you click, write your docs. 52 00:04:18,390 --> 00:04:24,660 The one DB that I with the different guides references which you could always make use of in case you 53 00:04:24,660 --> 00:04:30,780 are having any difficulties or as a starter, you want to master how all of this work. 54 00:04:30,780 --> 00:04:36,320 Then you also have this fully connected right here which brings ML practitioners together. 55 00:04:36,330 --> 00:04:44,730 Now you will have curated tutorials, conversations with industry leaders, deep dives into new ML research 56 00:04:44,730 --> 00:04:46,170 and a whole lot more. 57 00:04:46,170 --> 00:04:49,710 So you would always have this information. 58 00:04:49,710 --> 00:04:52,920 Others curated information at your disposal. 59 00:04:52,950 --> 00:04:58,530 Then we also have the community and then we have this quick stat for the different framework so we could 60 00:04:58,530 --> 00:04:59,970 view all frameworks here. 61 00:05:00,600 --> 00:05:04,780 And getting back, you see, we have PyTorch, Keras, we're working with Keras. 62 00:05:04,800 --> 00:05:06,050 Click on Keras. 63 00:05:06,060 --> 00:05:10,230 You would have this Quickstart for Keras users, which we actually. 64 00:05:10,230 --> 00:05:16,680 So here you see you see how easy it is to get started with one DB in Keras. 65 00:05:16,710 --> 00:05:23,850 The very first step we will take would be to install and log into one DB, so we'll simply copy this 66 00:05:23,850 --> 00:05:26,870 and then get back to our lab notebook. 67 00:05:26,880 --> 00:05:29,760 We paste this all right here and then run this. 68 00:05:29,770 --> 00:05:32,250 So we have the syntax error. 69 00:05:32,280 --> 00:05:34,720 Let's have this and then we run. 70 00:05:34,740 --> 00:05:35,410 There we go. 71 00:05:35,430 --> 00:05:38,670 As you can see, one DB has been installed from here. 72 00:05:38,670 --> 00:05:40,420 We are going to get into the logging. 73 00:05:40,440 --> 00:05:47,280 Now you see, you can find your API key in your browser here, so we could click on this link, get 74 00:05:47,280 --> 00:05:49,290 the API key and then paste it out here. 75 00:05:49,320 --> 00:05:53,590 Now let's get back to this and then copy out this API key. 76 00:05:53,610 --> 00:05:54,780 There we go. 77 00:05:54,780 --> 00:05:57,420 We have that and then we paste it out here. 78 00:05:57,420 --> 00:05:58,920 So we paste it out there. 79 00:05:58,950 --> 00:06:05,850 Or you could click on this link and then still get the key which we can paste in here and then simply 80 00:06:05,850 --> 00:06:06,870 press and enter. 81 00:06:06,870 --> 00:06:12,300 So we hit enter and we should be able to log in since we have this key put in right here. 82 00:06:12,330 --> 00:06:17,100 Now moving on to the next step at the top of your training script, start a new run. 83 00:06:17,100 --> 00:06:21,390 And to start this new one, we are going to make use of this init method. 84 00:06:21,390 --> 00:06:28,410 Getting back to the documentation you have, your run is a unit of computation logged by one DB. 85 00:06:28,440 --> 00:06:36,470 Typically this is an an ML experiment, so we create a run with the one DB in it. 86 00:06:36,480 --> 00:06:43,830 So before moving on, you should note that you could create a project and in this project you have several 87 00:06:43,830 --> 00:06:44,430 runs. 88 00:06:44,460 --> 00:06:50,970 Now one run could be for training, so we could have a training run like some sort of ML experiment 89 00:06:50,970 --> 00:06:52,140 as defined here. 90 00:06:52,140 --> 00:07:01,440 And then we have evaluation and you could have other different runs or all the different training processes 91 00:07:01,440 --> 00:07:03,600 which will act as different runs here. 92 00:07:03,600 --> 00:07:11,960 As you can see, when you just import one DB, the run is known and now you have one DB in it. 93 00:07:11,970 --> 00:07:19,620 Once you have once you make a call with this you on this method, you have a run which is automatically 94 00:07:19,620 --> 00:07:20,280 created. 95 00:07:20,280 --> 00:07:28,020 And so everything you're going to lock in into one DB will be sent to that particular run. 96 00:07:28,020 --> 00:07:33,870 So if you have, say, this project, let's call this project malaria or Prediction project, if you 97 00:07:33,870 --> 00:07:39,960 have our malaria prediction project and then we have this training run everything like once we create 98 00:07:39,960 --> 00:07:45,540 this run, everything we log in will be sent into this particular run right here. 99 00:07:45,540 --> 00:07:52,980 And then if we create another run that's evaluation, run, everything we log in will be stored in this 100 00:07:52,980 --> 00:07:54,660 particular run here. 101 00:07:54,660 --> 00:07:59,670 So getting back here, you see, you could create this run and you could finish or you could stop that 102 00:07:59,670 --> 00:08:01,770 run. 103 00:08:01,770 --> 00:08:07,050 And that's why, yeah, once you have this in it, you've created a run, you stop the run and then 104 00:08:07,050 --> 00:08:13,610 you're normally this should be known that there should be no run going, no run created. 105 00:08:13,620 --> 00:08:20,340 Now, after doing this, you could also create this like here's like one DB that you need. 106 00:08:20,340 --> 00:08:28,230 You could put this in a width block so we have width one db in it as run. 107 00:08:28,230 --> 00:08:36,990 Then you now have all the data to be locked in your such that out of this width block, you have no 108 00:08:36,990 --> 00:08:39,540 run from here. 109 00:08:39,540 --> 00:08:44,950 You could check out on the different attributes and then the information related to this attributes. 110 00:08:44,970 --> 00:08:46,320 Now let's get into this one. 111 00:08:46,320 --> 00:08:47,430 DB Neat one. 112 00:08:47,430 --> 00:08:48,030 DB Neat. 113 00:08:48,030 --> 00:08:56,190 You have this definition with all the different arguments it takes and then you have this information 114 00:08:56,190 --> 00:08:58,020 concerning all these different arguments. 115 00:08:58,020 --> 00:09:04,710 So here we could define like we have this run, which you could create by simply doing want to be in 116 00:09:04,740 --> 00:09:05,130 it. 117 00:09:05,130 --> 00:09:13,350 And then we'll specify the project, specify the entity, I would say neural learn the project, malaria 118 00:09:13,350 --> 00:09:15,660 detection, the configuration. 119 00:09:15,660 --> 00:09:18,750 All information will be needed, let's say for training. 120 00:09:18,840 --> 00:09:24,750 We specify a safe code to permit one DB, save our code by default. 121 00:09:24,750 --> 00:09:25,890 This is actually false. 122 00:09:25,890 --> 00:09:30,150 So by default your code is not going to be saved to one DB. 123 00:09:30,180 --> 00:09:31,950 You could check this out here. 124 00:09:31,980 --> 00:09:33,570 Let's have safe code. 125 00:09:33,570 --> 00:09:38,910 Let's search that, save code, go up and there we go. 126 00:09:38,910 --> 00:09:43,050 So by default, we don't allow this. 127 00:09:43,830 --> 00:09:47,370 And you could flip this behavior by going to settings page. 128 00:09:47,370 --> 00:09:50,250 Now, you could check on all this other arguments. 129 00:09:50,250 --> 00:09:54,840 You have this job type argument to, which is also very, very important. 130 00:09:54,840 --> 00:09:59,700 And this is very useful when you group in runs together into larger. 131 00:10:00,020 --> 00:10:03,190 Erm it's using this group argument right here. 132 00:10:03,650 --> 00:10:09,140 Now that said, let's copy this part of the code and then get back to our notebook. 133 00:10:09,560 --> 00:10:17,150 We will notice that we have this in parts so we would have to finish or put out this imports here. 134 00:10:17,150 --> 00:10:22,850 Let's have this, We have import one to be from one to be cross, import one to be called back. 135 00:10:22,850 --> 00:10:24,260 Let's take this off. 136 00:10:24,260 --> 00:10:29,660 So we put those two here and then we, we run the cell. 137 00:10:29,990 --> 00:10:31,280 That should be fine. 138 00:10:31,280 --> 00:10:32,450 Now that's run. 139 00:10:32,450 --> 00:10:40,970 We scroll down, we have the model, let's run this again and then let's actually get back here and 140 00:10:40,970 --> 00:10:41,900 create this run. 141 00:10:41,900 --> 00:10:43,460 So let's have this here. 142 00:10:43,460 --> 00:10:46,820 So one to be in needs, we're going to specify the project. 143 00:10:46,820 --> 00:10:47,780 We're going to create this. 144 00:10:47,780 --> 00:10:57,140 So this is one DB install logging and install logging and initialization. 145 00:10:57,140 --> 00:11:01,700 So we have that now, as we've said already, this is going to permit us create a run. 146 00:11:01,700 --> 00:11:10,910 So here let's have malaria detection, malaria detection and one way the like to write this out like 147 00:11:10,910 --> 00:11:14,150 the preferred to put this hyphen instead of the space. 148 00:11:14,150 --> 00:11:15,710 So let's have that that way. 149 00:11:15,710 --> 00:11:22,160 So we have here malaria detection entity in your learn and let's have that for now. 150 00:11:22,160 --> 00:11:27,350 So let's run the cell and then let's add this code below. 151 00:11:27,350 --> 00:11:31,730 We're currently locked in this new learn and that's fine from here. 152 00:11:31,730 --> 00:11:36,290 Let's now do one TB the run and see what we get from here. 153 00:11:36,290 --> 00:11:40,790 We see that we have no metrics locked yet, so that's fine. 154 00:11:40,790 --> 00:11:49,460 We could now take this off and then go straight away to add our one DB callback to TensorFlow. 155 00:11:50,000 --> 00:11:53,960 Before that we have saved model inputs and hyper parameters. 156 00:11:53,960 --> 00:12:00,800 So one TB config, let's have this copied and then we have that before defining the model, just copy 157 00:12:00,800 --> 00:12:06,680 that and then just here we'll add this code cell again. 158 00:12:06,680 --> 00:12:12,020 And then let's have your initialization configuration and there we go. 159 00:12:12,020 --> 00:12:19,100 Okay, so we have that when the B config learning rate specified number of epochs batch size would include 160 00:12:19,100 --> 00:12:25,880 the dropout rate, the image size, regularization rate, number of filters can all size, number of 161 00:12:25,880 --> 00:12:32,570 strides, pool size number of outputs for the first dense layer number of outputs for the second dense 162 00:12:32,570 --> 00:12:33,710 layer was as fine. 163 00:12:33,710 --> 00:12:35,020 We now around the cell. 164 00:12:35,030 --> 00:12:37,880 So we've now started this configurations. 165 00:12:37,880 --> 00:12:45,440 Then the next and last step is to simply have this callback in this fit method. 166 00:12:45,440 --> 00:12:48,410 So let's copy this out and then get back. 167 00:12:48,410 --> 00:12:49,940 So we have that. 168 00:12:49,940 --> 00:12:52,010 We've run our model. 169 00:12:52,010 --> 00:13:00,950 Let's be sure we run this model and just here, let's have this configuration. 170 00:13:00,950 --> 00:13:09,620 Let's say we have our configuration configuration equal one TB dot config, and then right here we have 171 00:13:09,620 --> 00:13:19,310 configuration and in size and we have same for the other hyper parameters like the dropout rate, regularization 172 00:13:19,310 --> 00:13:24,730 rate, number of filters, kernel size, pool size and number of strides. 173 00:13:24,740 --> 00:13:32,090 We now go ahead to the training process and so we copy this out and paste just right here. 174 00:13:32,090 --> 00:13:35,300 So here would have this callbacks. 175 00:13:35,300 --> 00:13:41,180 Let's take this off instead of this TensorFlow tensor board callback we had, we now have this one DB 176 00:13:41,180 --> 00:13:46,760 callback, so we have this one to be callback and you could always even include the tensor board callback. 177 00:13:46,760 --> 00:13:50,360 So you could also have the tensor board callback we had previously. 178 00:13:50,360 --> 00:13:54,410 Now there we go, we have this one DB callback, let's have that. 179 00:13:54,410 --> 00:13:57,380 And then we run the self metrics. 180 00:13:57,380 --> 00:13:59,510 We compile the model. 181 00:13:59,510 --> 00:14:01,460 One of the models compiled. 182 00:14:01,460 --> 00:14:03,770 Let's include this learning rate. 183 00:14:03,770 --> 00:14:10,040 So let's have this configuration and then we'll specify the learning rates. 184 00:14:10,040 --> 00:14:16,630 So we have that learning right there and then we compile the model and start with the training. 185 00:14:16,640 --> 00:14:18,170 Now the training is complete. 186 00:14:18,170 --> 00:14:25,970 We could go to our weights and bias dashboard right here and see exactly what went on during the training 187 00:14:25,970 --> 00:14:26,690 process. 188 00:14:26,690 --> 00:14:28,340 So here we have this projects. 189 00:14:28,340 --> 00:14:29,840 You could see your neural learn. 190 00:14:29,840 --> 00:14:36,560 We have the projects, we click on this malaria prediction project and then we select this run, the 191 00:14:36,560 --> 00:14:39,620 run we select and here is this sandy water run. 192 00:14:39,620 --> 00:14:40,970 So there we go. 193 00:14:40,970 --> 00:14:43,400 Now you see we have 19 different charts. 194 00:14:43,430 --> 00:14:49,640 AUC validation, LOS false, positive, false positive. 195 00:14:50,690 --> 00:14:53,000 Well, this is Epoc versus Epoc. 196 00:14:53,000 --> 00:14:59,660 So that's why you have this kind of straight line and then you have this precision recall. 197 00:14:59,840 --> 00:15:02,270 Loss through negative. 198 00:15:02,300 --> 00:15:04,000 To positive accuracy. 199 00:15:04,010 --> 00:15:10,220 In fact, all this is basically gardened from all this metrics we had here. 200 00:15:10,220 --> 00:15:18,920 So this means that with this simple callback we have put right here, waiting by us is able to get all 201 00:15:18,920 --> 00:15:24,350 this or capture all of this information during the training process and give it to us or presented to 202 00:15:24,350 --> 00:15:26,280 us after we're done with the training. 203 00:15:26,300 --> 00:15:30,380 So here we can see those different charts. 204 00:15:30,380 --> 00:15:36,020 We've already used to seeing this already, so you should be already familiar with this different charts. 205 00:15:36,890 --> 00:15:41,150 Now, apart from this charts, you have the system information. 206 00:15:41,150 --> 00:15:48,170 So here we have the CPU utilization, we have the system memory utilization process, memory and use 207 00:15:48,170 --> 00:15:49,790 process, memory and use. 208 00:15:50,690 --> 00:15:52,550 While this is in megabytes. 209 00:15:52,550 --> 00:15:59,660 Yeah, we have this information in the percentage with respect to the total memory available. 210 00:15:59,660 --> 00:16:07,160 Then we have process memory available process CPU tracks and use disk utilization network traffic, 211 00:16:07,730 --> 00:16:10,220 GPU utilization, GPU temperature. 212 00:16:10,220 --> 00:16:17,150 So you could have this information, your GPU time spent, GPU memory allocated, GPU power usage. 213 00:16:17,720 --> 00:16:25,360 And so you see how easy it is to get all this system information without writing any extra line of code. 214 00:16:25,370 --> 00:16:27,650 Now from here we could go on to the model. 215 00:16:27,650 --> 00:16:31,820 You see, we have this table right here which shows our model. 216 00:16:31,820 --> 00:16:35,690 It's kind of similar to the model summary we had put out previously. 217 00:16:35,690 --> 00:16:39,260 So yeah, it's kind of similar to what we had here. 218 00:16:39,260 --> 00:16:44,420 So now we have instead of that, we have this beautiful table right here, output shape, number of 219 00:16:44,420 --> 00:16:47,210 parameters type and the name. 220 00:16:47,210 --> 00:16:48,200 So that's it. 221 00:16:48,200 --> 00:16:51,290 And then from here we have this logs. 222 00:16:51,290 --> 00:16:57,020 So we have everything that was locked out like all we had here. 223 00:16:57,020 --> 00:17:00,140 So you see everything locked out. 224 00:17:00,140 --> 00:17:01,520 We have it in this run. 225 00:17:01,520 --> 00:17:09,980 Now recall that when the when doing this one DB in need last year, once we do this one to be in need, 226 00:17:09,980 --> 00:17:11,900 we actually create a run. 227 00:17:11,900 --> 00:17:18,050 And once the run is created, everything we do after that is going to be stored in that run. 228 00:17:18,050 --> 00:17:24,440 And that's why, if you notice here, like you don't only have this information like not only this training 229 00:17:24,440 --> 00:17:30,590 information, but even this data which was locked, even this model because we ran this several times. 230 00:17:30,590 --> 00:17:39,440 So you see here that even this information which was locked is actually start by one DB. 231 00:17:39,440 --> 00:17:45,880 And so you see that experiment tracking here is done very easily and actually seamlessly. 232 00:17:45,890 --> 00:17:54,320 Now you look at the files we start, you have the model best see, you have automatically this Keras 233 00:17:54,680 --> 00:17:56,660 model file will just start. 234 00:17:56,660 --> 00:18:01,820 You have this metadata, you could open this up and you have this information start. 235 00:18:01,820 --> 00:18:06,560 You see that we're using a Tesla P 100 GPU. 236 00:18:06,590 --> 00:18:11,570 So that's it, Python version, operating system version. 237 00:18:11,570 --> 00:18:17,200 And then other information like the GPU count or CPU count and so on and so forth. 238 00:18:17,210 --> 00:18:18,230 So that's it. 239 00:18:18,860 --> 00:18:20,570 Let's get back to this overview. 240 00:18:20,570 --> 00:18:21,890 You see it's still running. 241 00:18:21,890 --> 00:18:26,270 That's because we have our lab notebook still running right here. 242 00:18:26,480 --> 00:18:31,130 Now, one thing you could always count on is this documentation right here. 243 00:18:31,130 --> 00:18:37,670 So you just come straight to those integration and then you pick out Keras, you would have your cursor, 244 00:18:37,670 --> 00:18:41,360 and then you have this one to be Keras one DB callback. 245 00:18:41,360 --> 00:18:43,540 Then right here you have this. 246 00:18:43,550 --> 00:18:51,140 Now you could see these arguments, all the different arguments we have here, and you have the explicit 247 00:18:51,140 --> 00:18:52,250 definitions here. 248 00:18:52,250 --> 00:18:54,620 So let's check out on these arguments. 249 00:18:54,620 --> 00:19:04,160 You see, we have monitor the validation loss, which plays a similar role to this model checkpoint 250 00:19:04,160 --> 00:19:05,600 callback right here. 251 00:19:05,990 --> 00:19:08,300 And also we could specify the mode. 252 00:19:08,300 --> 00:19:12,470 So let's get back to the documentation where we are. 253 00:19:12,470 --> 00:19:17,990 Okay, So we get back to this mode so you could pick out the mode by default. 254 00:19:17,990 --> 00:19:19,100 It's automatic. 255 00:19:20,780 --> 00:19:23,540 We could also select mean or max. 256 00:19:23,660 --> 00:19:29,390 In the case of validation loss, obviously we would select a mean and the case of if we are dealing 257 00:19:29,390 --> 00:19:35,060 with a validation, accuracy or validation position or recall, then we'll select the mode to max such 258 00:19:35,060 --> 00:19:40,460 that we're saving our model when we have the maximum precision or maximum recall. 259 00:19:40,460 --> 00:19:46,820 For example, here we have the safe model save graph, save weights only lock weights, gradients, 260 00:19:46,820 --> 00:19:48,860 training data, validation data. 261 00:19:48,890 --> 00:19:56,990 Now you should know that with this we're able to pass in our data set to when TB and the fact that we 262 00:19:56,990 --> 00:19:59,360 pass in this data to one DB. 263 00:19:59,590 --> 00:20:07,150 And permit us come up with some visualizations of what the model is predicting since now we have the 264 00:20:07,150 --> 00:20:07,670 data. 265 00:20:07,690 --> 00:20:13,960 Then we also have the generator validation steps, levels, predictions, input type and so on and so 266 00:20:13,960 --> 00:20:14,320 forth. 267 00:20:14,320 --> 00:20:16,510 So you could always check out on this. 268 00:20:16,540 --> 00:20:25,840 Now the next thing we'll be doing is take the validation data pass into this one DB callback right here 269 00:20:25,840 --> 00:20:31,990 and then be able to visualize the different predictions as we go through the training process. 270 00:20:31,990 --> 00:20:33,610 So let's break this up a little. 271 00:20:33,610 --> 00:20:38,680 We have this training and validation number of epochs. 272 00:20:38,680 --> 00:20:47,770 This should actually be configuration configuration, number of epochs, number of epochs. 273 00:20:47,770 --> 00:20:48,370 Okay. 274 00:20:48,400 --> 00:20:54,210 So we have that and then we have the verbosity callbacks. 275 00:20:54,220 --> 00:21:04,210 Now in this one to be callback right here, we have validation validation data, which is of our data 276 00:21:04,210 --> 00:21:04,600 set. 277 00:21:05,200 --> 00:21:13,920 And then from here, let's check out the documentation and then we have the labels and the data type. 278 00:21:13,930 --> 00:21:17,380 So let's specify the labels from here. 279 00:21:17,380 --> 00:21:29,140 We have labels and the levels we have here we have Parasitized and Uninfected. 280 00:21:29,500 --> 00:21:37,630 Anyway, we could create this, like we could have the classes, so we could have your labels and then 281 00:21:37,630 --> 00:21:38,330 specify this. 282 00:21:38,350 --> 00:21:44,410 So let's just take this off from your cut that and then piece it out your. 283 00:21:44,800 --> 00:21:47,620 So we have that and then let's have this levels. 284 00:21:48,070 --> 00:21:50,050 Now we have the label. 285 00:21:50,380 --> 00:21:52,510 We could also specify the data type. 286 00:21:52,510 --> 00:21:56,050 So that's how we specify this to be image. 287 00:21:56,050 --> 00:21:59,620 Okay, we have that number of epochs and all of that. 288 00:21:59,620 --> 00:22:03,940 Let's go ahead and modify this number of books here. 289 00:22:03,940 --> 00:22:06,580 So number of epochs, 100. 290 00:22:06,580 --> 00:22:07,720 Let's change this. 291 00:22:07,720 --> 00:22:09,490 Let's just have three epochs for now. 292 00:22:09,490 --> 00:22:10,780 So let's get back. 293 00:22:11,600 --> 00:22:15,430 Want to be config, We should have this up here. 294 00:22:15,430 --> 00:22:19,840 So once we make this, we get this new configuration. 295 00:22:19,840 --> 00:22:20,680 So we run. 296 00:22:20,680 --> 00:22:21,940 That looks fine. 297 00:22:21,940 --> 00:22:23,740 We get back to the training from here. 298 00:22:23,740 --> 00:22:26,170 Now we could run this training again. 299 00:22:26,470 --> 00:22:27,430 Training complete. 300 00:22:27,430 --> 00:22:32,020 Let's go ahead and check out what has been locked in our dashboard. 301 00:22:32,020 --> 00:22:33,850 So let's have this here. 302 00:22:33,850 --> 00:22:35,770 We pick out this run. 303 00:22:35,770 --> 00:22:36,580 There we go. 304 00:22:36,580 --> 00:22:39,070 We see we have still our 19 different charts. 305 00:22:39,070 --> 00:22:42,580 And then we have this media which has been added here. 306 00:22:42,580 --> 00:22:45,640 We click on this and you see what we get. 307 00:22:46,090 --> 00:22:50,260 As we could see here, we have the different images and the predictions. 308 00:22:50,260 --> 00:22:53,860 You could view full screen mode by doing this. 309 00:22:55,030 --> 00:22:58,270 And then just here you could select the step. 310 00:22:58,270 --> 00:23:05,770 So if you pull this to the end, you see we have 36 steps, meaning we have trained for over 36 different 311 00:23:05,770 --> 00:23:14,650 epochs now selecting this malaria detection project and having all this to read different runs right 312 00:23:14,650 --> 00:23:15,130 here. 313 00:23:15,130 --> 00:23:19,510 You see, you could view all the runs simultaneously. 314 00:23:19,510 --> 00:23:21,010 So I could click on this one. 315 00:23:21,010 --> 00:23:24,040 You have this is one run, actually, this is another run. 316 00:23:24,040 --> 00:23:27,010 So I could view all this now simultaneously right here. 317 00:23:28,360 --> 00:23:29,790 See we have that. 318 00:23:29,800 --> 00:23:35,440 You see the difference in our GPU power usage with the two different runs. 319 00:23:35,440 --> 00:23:39,850 That's the sandy water to run and this exalted night to run. 320 00:23:40,390 --> 00:23:46,000 And then from here, if you want to stop around us, if we want to stop this current exalted night we 321 00:23:46,000 --> 00:23:49,600 run, what we could do is come right here. 322 00:23:49,600 --> 00:23:53,740 That is, let's say we let's have this code here. 323 00:23:53,740 --> 00:23:59,530 We could come right here and just simply put out one DB dot finish. 324 00:23:59,530 --> 00:24:03,160 So we call this and we should be able to stop this. 325 00:24:03,550 --> 00:24:10,510 The current run, you can always check out on this one to be the run documentation right here to get 326 00:24:10,510 --> 00:24:11,800 more information about this. 327 00:24:11,800 --> 00:24:20,590 You see for this example, given where the create the run and then the stop the run and then this shows 328 00:24:20,590 --> 00:24:22,510 that there is no current run. 329 00:24:22,510 --> 00:24:27,490 And then after creating the recreating the run, you see that there is now a run. 330 00:24:27,490 --> 00:24:31,300 So this is simply how we stop this run. 331 00:24:31,300 --> 00:24:33,100 And that's what we've just done here. 332 00:24:33,100 --> 00:24:40,360 We could simply run the cell and after running the cell, you have this run summary, you see the accuracy 333 00:24:40,360 --> 00:24:45,070 and the other different metrics and lost values. 334 00:24:45,070 --> 00:24:52,060 Notice how the different metrics and losses have been put out in this command line formatting. 335 00:24:52,060 --> 00:24:56,230 So you could see how those loss, for example, your drops. 336 00:24:56,230 --> 00:25:00,550 And then getting back to our dashboard, we see that there is no current run. 337 00:25:00,550 --> 00:25:05,800 Then we've again recreated a new run with a want to be neat right here. 338 00:25:05,800 --> 00:25:09,220 So we've created this new run on this elevator. 339 00:25:09,220 --> 00:25:11,020 We could check it out here. 340 00:25:11,050 --> 00:25:16,210 You see, we created this new run and you can see this little green cycle right here. 341 00:25:16,210 --> 00:25:17,800 Coming back to another book. 342 00:25:17,800 --> 00:25:26,020 We have this one DB callback, which will give you so far in logging information into weights and biases. 343 00:25:26,020 --> 00:25:34,300 But then this is limited because we are not yet able to define our own custom callbacks. 344 00:25:34,480 --> 00:25:37,930 Now, that said, let's get back to this callbacks. 345 00:25:37,930 --> 00:25:38,860 We are defined here. 346 00:25:38,860 --> 00:25:41,110 Yeah, we have this log images callback. 347 00:25:41,110 --> 00:25:47,640 We're going to actually try to log images into weights and biases using this callback which we had created. 348 00:25:47,650 --> 00:25:55,000 Now recall that to create this kind of custom callbacks you inherit from the callback class in TensorFlow, 349 00:25:55,000 --> 00:26:03,340 and then you get to put in some code in here which defines that custom callback. 350 00:26:03,610 --> 00:26:06,610 Now, we've just copied this and paste it right here. 351 00:26:06,610 --> 00:26:13,690 But the difference here is this is one DB, so this is what we had for Tensor Bard and this one now 352 00:26:13,690 --> 00:26:14,590 for one to be. 353 00:26:14,590 --> 00:26:20,110 You're going to see how easy it is even though we're building this custom callbacks. 354 00:26:20,110 --> 00:26:21,700 So right here. 355 00:26:22,540 --> 00:26:29,950 So let's suppose we want to predict or rather we want to log this conversion matrices and want to log 356 00:26:29,950 --> 00:26:31,610 this to weights and biases. 357 00:26:31,630 --> 00:26:35,740 Now what we'll do is we are going to take off all this year. 358 00:26:35,740 --> 00:26:36,760 So let's get back. 359 00:26:36,760 --> 00:26:42,940 This is what we did with ten so bored we saw that we had to make use of map plot, leap and then log 360 00:26:42,940 --> 00:26:45,040 this information into tensor board. 361 00:26:45,040 --> 00:26:54,570 But here's what we have or what we need to do to log the conversion metrics is simply just this part, 362 00:26:54,580 --> 00:26:55,990 this code right here. 363 00:26:55,990 --> 00:27:00,400 So here's what we have is we'll take all this off. 364 00:27:00,400 --> 00:27:06,160 You see, we take all this code, write your off, take this off. 365 00:27:06,160 --> 00:27:07,690 Oh, let's have that. 366 00:27:07,690 --> 00:27:10,060 So we have this code right here. 367 00:27:10,060 --> 00:27:11,080 Take an off. 368 00:27:11,080 --> 00:27:13,210 And then we just left with this. 369 00:27:13,210 --> 00:27:20,710 So we see that instead of having to write all this code, all we need now to write is just this one. 370 00:27:20,800 --> 00:27:28,270 Now, if you're wondering why or how to get the auto plots or what other plots we could get automatically 371 00:27:28,270 --> 00:27:34,540 like this with weights and bias, we get into these weights and bias GitHub repository, and here we 372 00:27:34,540 --> 00:27:41,770 have we have this link or one DB client tree master, one DB plot. 373 00:27:41,800 --> 00:27:44,470 You see all the different plots we have now. 374 00:27:44,470 --> 00:27:51,910 And since it's constantly under development, surely in some time to come we will have many more of 375 00:27:51,910 --> 00:27:54,460 these different plots we could do very easily. 376 00:27:54,460 --> 00:27:58,630 So here we have bar the pie confusion matrix. 377 00:27:58,630 --> 00:28:05,490 You see we have the convention matrix, histogram, line lion series, precision recall curve, rosy 378 00:28:05,500 --> 00:28:08,430 curve, and Qatar pie pie. 379 00:28:08,440 --> 00:28:15,880 So this means that already we could plot out this coefficient matrix and the RC curve very easily with 380 00:28:15,880 --> 00:28:17,170 Wilson bias. 381 00:28:17,170 --> 00:28:20,000 So let's get back to this or check this out. 382 00:28:20,020 --> 00:28:22,810 The first thing we have noticed, this log one to be the log. 383 00:28:22,810 --> 00:28:27,730 So whenever you're using this one DB callback here was actually happen. 384 00:28:28,230 --> 00:28:32,910 Or what happens under the hood is this information has been logged like this. 385 00:28:33,240 --> 00:28:35,790 So we have make use of this one to be logged. 386 00:28:35,790 --> 00:28:44,330 And then here we are going to have let's change this, let's say confusion, confusion matrix, and 387 00:28:44,340 --> 00:28:45,720 we'll just write like this. 388 00:28:45,750 --> 00:28:50,820 Okay, Yeah, we have the convention matrix and then you have one DB plot confusion matrix. 389 00:28:50,820 --> 00:28:57,030 You could modify this and put power curve that expression we recall curve or RC curve and that's it. 390 00:28:57,030 --> 00:28:59,220 And then this props. 391 00:28:59,220 --> 00:29:05,250 Then we have y true projects, the predictions and then the class names. 392 00:29:05,250 --> 00:29:13,860 So your we should change this class names and have parasitized parasitized. 393 00:29:13,860 --> 00:29:23,220 And what we have here we have uninfected and then you can either pass the probabilities or the predicted 394 00:29:23,220 --> 00:29:23,730 score. 395 00:29:23,730 --> 00:29:30,780 So here we are going to have props equal predicted since what our model predicts or what our model outputs 396 00:29:30,780 --> 00:29:34,830 are the probabilities and then the Y true is equal levels. 397 00:29:34,830 --> 00:29:38,520 From here we're going to take out the spreads because we've done that already. 398 00:29:38,520 --> 00:29:40,680 So let's have this year. 399 00:29:40,710 --> 00:29:41,610 There we go. 400 00:29:41,610 --> 00:29:44,190 This doesn't take into consideration the threshold. 401 00:29:44,190 --> 00:29:51,750 So we take off this threshold that it now we have this callback one DB and then we're ready to train 402 00:29:51,750 --> 00:29:52,650 our model. 403 00:29:52,890 --> 00:29:59,880 After the training process, we have this results which look great, and we could now go ahead to look 404 00:29:59,880 --> 00:30:03,830 at the convention matrix plotted in the dashboard. 405 00:30:03,840 --> 00:30:09,990 So let's get to our dashboard and this is where we should have, let's get back to runs and then click 406 00:30:09,990 --> 00:30:12,030 on this call and run. 407 00:30:12,030 --> 00:30:13,770 And this is what we get now. 408 00:30:13,770 --> 00:30:22,020 We see that we have this coefficient matrix which doesn't actually show us what we expect to get. 409 00:30:22,020 --> 00:30:30,330 And this is simply because the way this was conceived or the way this callback here, this login was 410 00:30:30,330 --> 00:30:36,030 conceived, was such that we have a multi class problem. 411 00:30:36,030 --> 00:30:44,370 So even in the case where we have a binary classification problem, like in this case we expect to have 412 00:30:44,370 --> 00:30:46,170 an output of two values. 413 00:30:46,170 --> 00:30:55,230 So that's why when you have this output, let's get down here, let's click on this, let's add this 414 00:30:55,230 --> 00:30:55,590 code. 415 00:30:55,590 --> 00:31:04,800 So that's why when you have a value like say and the output 0.9, it's considered to be parasitized, 416 00:31:04,800 --> 00:31:12,270 although we defined already, that parasitized is meant to be zero and infected equal one. 417 00:31:12,420 --> 00:31:19,050 And so when we have 0.9, this is greater than the midpoint of this tool, which is 0.5. 418 00:31:19,050 --> 00:31:21,210 So we should consider this as uninfected. 419 00:31:21,330 --> 00:31:27,480 But the way, as we said already, the way this has been conceived is such that even in the binary classification 420 00:31:27,480 --> 00:31:34,560 problem, we shouldn't or we don't have a single output, but two outputs, that is, if we have a value 421 00:31:34,560 --> 00:31:43,410 of 0.9, what one DB expects to see something like this, something like zero and 0.9. 422 00:31:43,410 --> 00:31:52,380 So showing that your oh, let's, let's have input zero one even zero points less than 0.8 and 0.2. 423 00:31:52,770 --> 00:31:54,210 Okay, so let's have this. 424 00:31:54,210 --> 00:31:58,260 So even in this case it comes back to the same output. 425 00:31:58,590 --> 00:32:04,590 So this simply means that we are having an unaffected. 426 00:32:04,590 --> 00:32:08,760 So since this first index here has a higher value. 427 00:32:09,570 --> 00:32:14,640 Now, to solve this issue, what we're going to do is we are going to take each and every output we 428 00:32:14,640 --> 00:32:16,770 have and convert it to this format. 429 00:32:16,770 --> 00:32:24,360 So in the case where we have, for example, 0.1 as output, we are going to convert it into one zero 430 00:32:24,360 --> 00:32:32,580 because this means that the higher the this one means that it's parasitized cell and this tool means 431 00:32:32,580 --> 00:32:40,470 the same because here we're seeing that this zero index, which is the prioritized index, has a higher 432 00:32:40,470 --> 00:32:41,040 value. 433 00:32:41,040 --> 00:32:43,680 Hence this is a prioritized cell. 434 00:32:44,580 --> 00:32:50,490 So that said, what we're going to do now is for all values, less than 0.5, we could say less than 435 00:32:50,700 --> 00:32:55,080 0.5, we are going to convert it into zero. 436 00:32:55,080 --> 00:32:57,390 Sorry, we're going to convert this one zero. 437 00:32:57,390 --> 00:33:05,100 And then for all values greater than 0.5, we're going to convert them into zero one. 438 00:33:05,100 --> 00:33:12,360 So this the transformation we are going to make in order for this one to be locked method to correctly 439 00:33:12,360 --> 00:33:14,040 log our values. 440 00:33:14,310 --> 00:33:20,640 But note that if we're having a multi classification problem, it will be needless doing this transformation. 441 00:33:20,640 --> 00:33:26,490 So let's get straight away and see how we're going to transform this into this required format right 442 00:33:26,490 --> 00:33:27,090 here. 443 00:33:27,120 --> 00:33:27,660 Let's. 444 00:33:27,770 --> 00:33:30,110 Started by copying out this first part. 445 00:33:30,110 --> 00:33:32,810 Let's copy this into this other cell here. 446 00:33:32,810 --> 00:33:36,090 And once we run this, we have this output right here. 447 00:33:36,110 --> 00:33:39,050 Then we could go ahead and modify this predicted. 448 00:33:39,050 --> 00:33:48,980 So what we now have is we will define the spread, this auto list and then four iron range range, the 449 00:33:48,980 --> 00:33:50,510 line of predicted. 450 00:33:50,510 --> 00:34:01,250 We are going to make sure that if the value pre that I we're going to take this zero index. 451 00:34:01,250 --> 00:34:12,290 If it's less than 0.5, then what we have to do is append this to this list spread right here. 452 00:34:12,290 --> 00:34:17,210 So what we're going to append this one zero because this is less than 0.5. 453 00:34:17,210 --> 00:34:25,790 So if it was a multiclass problem, we would have the highest or the higher value in this case, since 454 00:34:25,790 --> 00:34:32,030 we just have in two classes, we have the higher value representing this output which was predicted, 455 00:34:32,030 --> 00:34:34,010 which happens to be less than 0.5. 456 00:34:34,010 --> 00:34:38,660 So we have your one and then yeah, we'll have else. 457 00:34:38,930 --> 00:34:43,700 Brad append what do we have next? 458 00:34:43,700 --> 00:34:45,590 We have zero one. 459 00:34:45,590 --> 00:34:47,330 So yeah, we have zero one. 460 00:34:47,330 --> 00:34:50,150 Let's take this off zero one and that's fine. 461 00:34:50,150 --> 00:34:55,430 So yeah, we've modified the spread and then we could then print out the spread. 462 00:34:55,910 --> 00:35:00,140 So with this now we should be able to have the expected output. 463 00:35:00,320 --> 00:35:01,130 There we go. 464 00:35:01,130 --> 00:35:08,810 We have this output, let's add this code cell and then print out our plate shape. 465 00:35:09,110 --> 00:35:11,330 We run this and what do we get? 466 00:35:11,330 --> 00:35:13,190 You see, we have exactly what we expect. 467 00:35:13,190 --> 00:35:20,420 So that said, let's copy out this part from your and then we're going to add this here. 468 00:35:20,690 --> 00:35:22,270 So we have the spread. 469 00:35:22,280 --> 00:35:31,460 Now let's have this forward and then we're going to have Brad equal input array of print. 470 00:35:31,460 --> 00:35:37,370 So we'll convert it into a non pi array before passing into this lock method console. 471 00:35:37,580 --> 00:35:38,450 That should be fine. 472 00:35:38,480 --> 00:35:39,550 Okay, so that's it. 473 00:35:39,560 --> 00:35:41,750 Now we have this set. 474 00:35:41,750 --> 00:35:42,740 Everything looks fine. 475 00:35:42,740 --> 00:35:45,680 Let's take this off and have that spread. 476 00:35:45,680 --> 00:35:52,820 So we have the spread now everything looks okay and we could now rerun our restart our training. 477 00:35:52,820 --> 00:35:56,840 So let's have this year, let's take this off from here. 478 00:35:56,840 --> 00:36:05,030 We train again for just about two epochs and then the results we get so we can click on this run here. 479 00:36:05,030 --> 00:36:08,930 And then you could have the tables and the custom chats. 480 00:36:08,930 --> 00:36:10,640 Let's start with the custom chats. 481 00:36:10,640 --> 00:36:13,160 You could expand this this way. 482 00:36:13,160 --> 00:36:20,330 So you see that we have the predicted and the actual so we have the model. 483 00:36:20,360 --> 00:36:26,870 You see now that we have this number of true, let's say true positives, which increases the number 484 00:36:26,870 --> 00:36:34,840 of true negatives, which also increase while we have this number of false negatives to be 112, while 485 00:36:34,850 --> 00:36:36,380 here we have 62. 486 00:36:36,380 --> 00:36:41,150 This is 1278 1305. 487 00:36:41,360 --> 00:36:47,090 And when you compare this with this previous run, you see that now we are having reasonable outputs. 488 00:36:47,090 --> 00:36:54,440 As with this previous run, we had that error where the coefficient matrix method of one db considers 489 00:36:54,440 --> 00:36:57,770 all the outputs to be parasitized. 490 00:36:58,160 --> 00:37:00,590 Let's have this back and take this off. 491 00:37:01,220 --> 00:37:06,140 Then from here we could go ahead to look at how to plot the rosy plots. 492 00:37:06,410 --> 00:37:07,850 Let's get back here. 493 00:37:07,880 --> 00:37:09,020 Scroll up. 494 00:37:09,500 --> 00:37:17,390 What we'll do is here we just have to let's commend this here and then now do the same for the row plots. 495 00:37:17,840 --> 00:37:22,520 Let's get back to the GitHub repo and click on this rosy curve here. 496 00:37:22,610 --> 00:37:27,250 In this year, we have now those arguments which have been described. 497 00:37:27,260 --> 00:37:30,740 So here we have why true why probabilities. 498 00:37:30,740 --> 00:37:38,240 Obviously this is the level predicted and then the classes, the class names, classes to plot that 499 00:37:38,240 --> 00:37:42,260 is a set of classes which we are going to plot. 500 00:37:42,620 --> 00:37:48,650 We're going to exclude all classes which aren't specified in this classes to plot argument. 501 00:37:48,650 --> 00:37:52,850 Now this is actually optional, so you don't necessarily need to have this. 502 00:37:52,850 --> 00:37:56,540 We also have this title, so let's copy this out. 503 00:37:56,570 --> 00:37:59,930 Let's copy out this year. 504 00:38:00,230 --> 00:38:05,660 Let's copy this out and paste just right here. 505 00:38:05,660 --> 00:38:07,490 We then specify the levels. 506 00:38:07,490 --> 00:38:09,680 Y True levels. 507 00:38:10,280 --> 00:38:16,160 Yeah, we have Pred and then right here we have this class names. 508 00:38:16,160 --> 00:38:20,240 So let's copy this out and paste here. 509 00:38:20,720 --> 00:38:23,030 Okay, so that's what we have. 510 00:38:23,030 --> 00:38:25,190 Let's run this cell. 511 00:38:25,220 --> 00:38:27,580 Everything looks fine and then restore. 512 00:38:27,990 --> 00:38:29,040 With the training. 513 00:38:29,190 --> 00:38:30,600 Training now complete. 514 00:38:30,630 --> 00:38:31,980 Here are the results we obtain. 515 00:38:31,980 --> 00:38:34,560 We get back to our dashboard and this is what we have. 516 00:38:34,560 --> 00:38:39,880 You could go back to the runs and you'll see that you click on this custom charts right here. 517 00:38:39,900 --> 00:38:40,950 Now let's click on this. 518 00:38:40,950 --> 00:38:44,910 Run your first, and then we have this two custom charts. 519 00:38:44,910 --> 00:38:51,150 So we have those previous conviction metrics which we logged in, and then now we have this ROIC curve, 520 00:38:51,150 --> 00:38:52,650 which we just plotted out. 521 00:38:52,650 --> 00:38:57,930 We could also check out the tables for the C curve and the conviction metrics. 522 00:38:58,110 --> 00:38:59,190 So here we go. 523 00:38:59,190 --> 00:39:06,900 We have this rosy plot and then we have this those two plots here where one is for the uninfected and 524 00:39:06,900 --> 00:39:10,010 the other is for the parasitized class. 525 00:39:10,020 --> 00:39:16,560 I haven't seen how to plot the coefficient metrics and the rosy curve we could check out in the documentation 526 00:39:16,560 --> 00:39:18,270 and look at the order plots. 527 00:39:18,270 --> 00:39:27,360 So here we have basic charts, line scatter bar chart, histogram, model line, and the next model 528 00:39:27,360 --> 00:39:33,030 evaluation charts like what we've just done, the conversion metrics I see curves, PR curves, that's 529 00:39:33,030 --> 00:39:34,440 precision recall curves. 530 00:39:34,440 --> 00:39:39,510 You have them your interactive custom charts, plot leap and plot plot. 531 00:39:39,510 --> 00:39:47,280 So this means you could actually come up with your own plot with map plot lib, and then log that to 532 00:39:47,280 --> 00:39:48,450 one DB. 533 00:39:48,630 --> 00:39:56,070 And then apart from this method in which we use the one DB callback here, Oh no, not this one, the 534 00:39:56,070 --> 00:39:58,170 one that we call callback we had defined previously. 535 00:39:58,350 --> 00:40:06,300 Apart from using that one DB callback to log the loss metrics and other information, we could directly 536 00:40:06,300 --> 00:40:12,870 just do one DB log and then we say for example, want to log the loss? 537 00:40:12,870 --> 00:40:14,490 So we just put our loss. 538 00:40:14,490 --> 00:40:22,170 And then since we got the loss from this logs, we have this logs here, we scroll down and then we 539 00:40:22,170 --> 00:40:26,340 put out your logs logs and then we specify the loss. 540 00:40:26,340 --> 00:40:31,680 So this all what suffices to log this lost values from here. 541 00:40:31,680 --> 00:40:35,280 You could also log the accuracy and all the metrics. 542 00:40:36,360 --> 00:40:37,590 It's actually a dictionary. 543 00:40:37,590 --> 00:40:38,580 So let's have that. 544 00:40:39,180 --> 00:40:40,530 And there we go. 545 00:40:40,530 --> 00:40:43,070 So we've seen how to log these values. 546 00:40:43,080 --> 00:40:51,960 Now, let's also see how the log images, like with this, we had this image and then we log this to 547 00:40:51,960 --> 00:40:52,170 ten. 548 00:40:52,170 --> 00:41:00,060 So board now we'll see how to get this image and then log it to one DB to log Those images will then 549 00:41:00,060 --> 00:41:04,650 check out here in the documentation and you'll see that you can log rich media files. 550 00:41:04,650 --> 00:41:14,670 That's really any kind of major file, three rd point clouds, molecules, HTML and histograms. 551 00:41:14,880 --> 00:41:15,770 So that's that. 552 00:41:15,810 --> 00:41:22,620 You scroll down and you have this code here which permits us log image data. 553 00:41:22,620 --> 00:41:32,100 So yes, logging erases images, log in pill images, log in, images from files, image overlays, 554 00:41:32,100 --> 00:41:35,970 segmentation masks, bounding boxes and so on and so forth. 555 00:41:35,970 --> 00:41:37,200 So you have this. 556 00:41:37,200 --> 00:41:39,060 You also have this histograms. 557 00:41:39,060 --> 00:41:40,890 You have three D visualizations. 558 00:41:40,920 --> 00:41:43,830 So you could see point cloud and molecules. 559 00:41:43,830 --> 00:41:51,630 So you see that one DB actually permits just any practitioner be able to log any data or any kind of 560 00:41:51,630 --> 00:41:54,960 data they really want to log, hence making them more efficient. 561 00:41:54,960 --> 00:41:57,420 So that said, let's get back to this year. 562 00:41:57,420 --> 00:42:03,150 We log in arrays, let's copy this out, let's click here, you click, just simply copy that way and 563 00:42:03,150 --> 00:42:04,470 then you get back to the code. 564 00:42:04,470 --> 00:42:10,080 So here's what we're saying is we want to log we want to be able to log this images as this level. 565 00:42:10,080 --> 00:42:15,180 So let's paste this out somewhere here and then copy out this part. 566 00:42:15,180 --> 00:42:17,220 Again, we're going to copy this part. 567 00:42:17,370 --> 00:42:25,110 Recall that with this one to we are defined here, We we kind of like got we ended at this level where 568 00:42:25,110 --> 00:42:32,690 we got the labels and the predicted and automatically we got this rosy curve and coefficient metrics. 569 00:42:32,700 --> 00:42:40,950 Now what we want to do is get right up to this point where we actually have the image that is our own 570 00:42:40,950 --> 00:42:42,780 image which we've created. 571 00:42:42,780 --> 00:42:51,090 And then from this image, log E to one to B, And so right here we have this piece of code here and 572 00:42:51,090 --> 00:42:55,950 we're going to integrate this one thing we could do again, let's copy this again. 573 00:42:55,950 --> 00:42:59,760 Now let's copy this other one, because here we have the image, so we'll copy this. 574 00:42:59,760 --> 00:43:01,890 Again, we consider that this is 4/10 of board. 575 00:43:02,310 --> 00:43:03,780 Let's write your ten. 576 00:43:03,780 --> 00:43:04,770 So bar ten. 577 00:43:04,770 --> 00:43:06,360 So board. 578 00:43:06,360 --> 00:43:08,130 Okay, we have that for tensor board. 579 00:43:08,130 --> 00:43:12,300 We have this for one DB now one DB plot. 580 00:43:12,300 --> 00:43:14,880 And then this next one is just one DB. 581 00:43:15,060 --> 00:43:16,410 Let's add this here. 582 00:43:16,530 --> 00:43:17,520 There we go. 583 00:43:17,520 --> 00:43:22,470 Okay, so yeah, we have this 1db, that's fine. 584 00:43:22,470 --> 00:43:27,060 So what we're saying is we are going to take off all this here. 585 00:43:27,060 --> 00:43:27,450 So we. 586 00:43:27,920 --> 00:43:32,450 All this off, and then we copy this out. 587 00:43:33,080 --> 00:43:36,140 Oh, let's cut that out and then paste it here. 588 00:43:36,590 --> 00:43:42,260 So there's how are we're going to lock this to one to be as usual, is going to always be simpler than 589 00:43:42,260 --> 00:43:43,910 what we have with the tensor board. 590 00:43:43,910 --> 00:43:49,060 So we just all we need to do is just to take our image and then lock as simple as that. 591 00:43:49,070 --> 00:43:54,470 Now, our image array here, let's have let's see this image array, our image arrays, this output, 592 00:43:54,470 --> 00:43:59,870 which we have, and then we pass this image array into this caption. 593 00:43:59,870 --> 00:44:09,950 Let's put a caption, let's say confusion, confusion matrix for epoch, and then we get the epoch from 594 00:44:09,950 --> 00:44:11,600 here, like we have this epoch. 595 00:44:11,600 --> 00:44:14,720 So we're going to be logging this completion matrix for each and every epoch. 596 00:44:14,720 --> 00:44:18,050 So we just put out this epoch here and that's fine. 597 00:44:18,050 --> 00:44:20,390 So that's our caption. 598 00:44:20,390 --> 00:44:24,650 We have one tab image and we lock it in here. 599 00:44:24,650 --> 00:44:33,590 Let's have this, let's say confusion, confusion matrix or yeah, confusion matrix. 600 00:44:33,590 --> 00:44:34,700 Okay, let's have that. 601 00:44:35,060 --> 00:44:37,700 Now we run this, we copy. 602 00:44:37,700 --> 00:44:39,110 Anyway, we had that already. 603 00:44:39,110 --> 00:44:40,520 Let's copy this out. 604 00:44:40,520 --> 00:44:43,580 We have an error here. 605 00:44:43,580 --> 00:44:44,900 They should be closed. 606 00:44:45,050 --> 00:44:46,100 Okay, so that's fine. 607 00:44:46,100 --> 00:44:49,970 We have this one DB log right here and everything looks okay. 608 00:44:50,000 --> 00:44:53,420 Okay, so let's run that again and check this out. 609 00:44:53,420 --> 00:44:54,380 Looks fine. 610 00:44:54,380 --> 00:44:57,290 Now, we could go ahead and start with a training. 611 00:44:57,290 --> 00:44:58,670 Let's run this. 612 00:44:58,670 --> 00:45:02,750 They should be okay now and compile the model. 613 00:45:02,750 --> 00:45:06,140 After compiling the model, we now go ahead and train the model. 614 00:45:06,290 --> 00:45:08,960 Let's say we want to have three here. 615 00:45:09,080 --> 00:45:11,000 Let's keep that aside for now. 616 00:45:11,000 --> 00:45:12,980 And then here we have this. 617 00:45:12,980 --> 00:45:14,060 Let's paste it out. 618 00:45:14,060 --> 00:45:14,720 It should be the same. 619 00:45:14,720 --> 00:45:15,140 Okay. 620 00:45:15,140 --> 00:45:16,370 Yeah, we have it now. 621 00:45:16,370 --> 00:45:19,340 Let's run the training process and see what we get. 622 00:45:19,340 --> 00:45:20,750 Training now complete. 623 00:45:20,750 --> 00:45:26,480 We can get back to our dashboard and see what we have as plots. 624 00:45:26,480 --> 00:45:31,310 We'll click on this here, this run our car and run. 625 00:45:31,310 --> 00:45:35,000 We have this media we locked previously. 626 00:45:35,000 --> 00:45:37,940 Now let's check in this hidden panels and see what we get. 627 00:45:38,030 --> 00:45:42,620 And it happens that we have this confusion matrix locked in hidden panels. 628 00:45:42,620 --> 00:45:44,270 Anyways, we have that. 629 00:45:44,270 --> 00:45:45,560 And what do you notice? 630 00:45:45,560 --> 00:45:50,620 You see for the three different epochs, you see for the three, you could go to the second and the 631 00:45:50,660 --> 00:45:51,320 third epoch. 632 00:45:51,350 --> 00:45:52,100 Start with the first. 633 00:45:52,130 --> 00:45:55,790 You see we have the condition matrices which should be logged per epoch. 634 00:45:55,790 --> 00:46:03,620 And now let's do this so it appears clearer and we see we have this number of true positives, true 635 00:46:03,620 --> 00:46:07,430 negatives, false positives, false negatives and so on and so forth. 636 00:46:07,430 --> 00:46:08,210 So that's it. 637 00:46:08,210 --> 00:46:14,210 From here you can notice how this value gives some leaves from 68 to 52. 638 00:46:14,210 --> 00:46:16,720 And then finally we have 53. 639 00:46:16,730 --> 00:46:23,270 Now another interesting functionality is you could simply have that C, you could download it and now 640 00:46:23,270 --> 00:46:26,450 you have this full screen and you see that clarity. 641 00:46:26,690 --> 00:46:30,800 So let's press escape close and then get back. 642 00:46:30,800 --> 00:46:31,490 So that's it. 643 00:46:31,490 --> 00:46:35,000 That's how we lock this image very easily with one. 644 00:46:35,000 --> 00:46:35,500 DB. 645 00:46:35,510 --> 00:46:40,370 Now getting back to the documentation, you might have noticed that you have your Keras and then you 646 00:46:40,370 --> 00:46:42,290 have TensorFlow. 647 00:46:42,290 --> 00:46:44,750 Somewhere around you have TensorFlow. 648 00:46:44,750 --> 00:46:49,160 You see that this kind of like considered to be two separate libraries. 649 00:46:49,790 --> 00:46:55,490 But then just note that if you build an Keras model, that's if you're building this kind of model where 650 00:46:55,490 --> 00:47:02,810 you have you're making use of this method, this feed method to train your model. 651 00:47:02,810 --> 00:47:11,900 Then making use of this Keras documentation right here is appropriate, but then sometimes you want 652 00:47:11,900 --> 00:47:19,160 to have control of full control over what you're doing, and then you want to be able to do custom training 653 00:47:19,370 --> 00:47:20,990 like we had seen previously. 654 00:47:20,990 --> 00:47:23,330 Let's scroll down to cause some training loop. 655 00:47:23,330 --> 00:47:30,350 So if you are having a custom training loop, like with this year, you would find that you would not 656 00:47:30,350 --> 00:47:37,190 be able to use the one DB callback as easily as you had done with the Keras code. 657 00:47:37,190 --> 00:47:43,010 So in that case you'll see that instead of having like for example, like year we have training block 658 00:47:43,010 --> 00:47:50,900 and then we have this last year, all you need to do is come and put in the one DB you log and then 659 00:47:50,900 --> 00:47:51,920 you log the loss. 660 00:47:51,920 --> 00:47:57,500 So you just say loss and then you simply lock this last, let's say last non pi and that's all. 661 00:47:57,500 --> 00:48:04,000 So that's why it takes now to log this last values, the different metrics and so on and so forth. 662 00:48:04,010 --> 00:48:07,400 The integration with tensor board has been made quite easy too. 663 00:48:07,400 --> 00:48:11,060 So if you're already using tensor board, it's easy to integrate with one DB. 664 00:48:11,090 --> 00:48:19,400 Now you're they say how one DB is different from tensor barred the ability to reproduce models, automatic 665 00:48:19,400 --> 00:48:27,350 organization fast flexible integration persistence centralized dashboard powerful tables and tools for. 666 00:48:27,440 --> 00:48:32,330 Collaboration that said, let's copy this out, let's copy this again. 667 00:48:32,330 --> 00:48:37,120 And now when we'll be creating our run with a init method, we're going to instead have this. 668 00:48:37,130 --> 00:48:39,680 So let's get back to our code. 669 00:48:40,880 --> 00:48:44,780 Let's reduce this year and then stop this run. 670 00:48:44,780 --> 00:48:47,410 So we have this one DB finish. 671 00:48:47,420 --> 00:48:52,280 We stopped the run, the run stopped, and then we're going to create this other new run which would 672 00:48:52,280 --> 00:48:55,720 take into consideration tens of barred logs. 673 00:48:55,730 --> 00:49:01,270 So let's get back to the top and just write your pieces out. 674 00:49:01,280 --> 00:49:04,130 We've already had this so we could take this off. 675 00:49:04,130 --> 00:49:08,870 Now let's comment this and then let's have Project Entity. 676 00:49:08,900 --> 00:49:11,380 Just copy that and paste out here. 677 00:49:11,420 --> 00:49:14,060 That's fine configuration. 678 00:49:14,060 --> 00:49:16,640 We have that syntax of BAH, that's fine. 679 00:49:16,670 --> 00:49:17,930 Okay, let's run this. 680 00:49:17,930 --> 00:49:22,610 Now we get this error, so let's take this off here. 681 00:49:22,910 --> 00:49:28,250 Now, this isn't compatible with version two of TensorFlow, which is what we're using. 682 00:49:28,250 --> 00:49:29,870 So let's run this again. 683 00:49:29,870 --> 00:49:31,580 And this now should be fine. 684 00:49:31,580 --> 00:49:34,010 We now have created this new run. 685 00:49:34,010 --> 00:49:38,480 Let's scroll down and we get to our custom training loop. 686 00:49:38,480 --> 00:49:40,520 So we're going to run this. 687 00:49:40,700 --> 00:49:46,610 We have the optimizer metric metric validation hyperx number specified. 688 00:49:46,610 --> 00:49:56,510 And then let's say we have this configuration and then we have an E box and epochs. 689 00:49:56,510 --> 00:49:58,130 Okay, so that's it. 690 00:49:58,610 --> 00:50:02,870 We have all this set and then we run this. 691 00:50:02,900 --> 00:50:10,790 We've seen this already under the Section ten, so we get this one in when using several event log directories 692 00:50:10,790 --> 00:50:17,570 like what we're doing right here, please call tensor when db tensor board patch and then specify the 693 00:50:17,570 --> 00:50:20,870 root log directory before the one to be in need. 694 00:50:20,870 --> 00:50:23,570 So let's take this from here. 695 00:50:23,960 --> 00:50:24,950 Copy this out. 696 00:50:24,950 --> 00:50:28,490 And even if we're checking the documentation, you should have that. 697 00:50:28,490 --> 00:50:30,260 So here you have this. 698 00:50:30,260 --> 00:50:38,120 You're the one to be one to be ten support patch, and then you have this root directory right here. 699 00:50:38,510 --> 00:50:43,100 So this means again, we are going to stop the current run. 700 00:50:43,100 --> 00:50:44,990 Where are we exactly? 701 00:50:44,990 --> 00:50:46,850 Let's stop this current run. 702 00:50:46,850 --> 00:50:48,590 It should be at the level of training. 703 00:50:48,590 --> 00:50:51,920 We're going to stop this current run here. 704 00:50:51,920 --> 00:50:53,570 We run this again. 705 00:50:53,570 --> 00:50:54,890 We stop that run. 706 00:50:54,890 --> 00:51:02,570 And then what we do now is we have this year and then we specify logs. 707 00:51:02,720 --> 00:51:06,080 So we have that root directory specified. 708 00:51:06,080 --> 00:51:13,700 Now we have the root directory specified that's logs, get back documentation, we have that specified 709 00:51:13,700 --> 00:51:14,630 before the init. 710 00:51:14,630 --> 00:51:18,110 So we have to run that before the init looks fine. 711 00:51:18,110 --> 00:51:19,250 Let's run this now. 712 00:51:21,230 --> 00:51:22,890 Let's run this cell. 713 00:51:23,270 --> 00:51:24,230 And there we go. 714 00:51:25,370 --> 00:51:27,380 Yeah, we told tensor board already patched. 715 00:51:27,380 --> 00:51:30,110 Remove the symptoms about true from one to be needs. 716 00:51:30,110 --> 00:51:36,370 So since we've already done this patch we should remove this from want to be in needs OC. 717 00:51:36,380 --> 00:51:38,390 Let's take that off and run again. 718 00:51:38,390 --> 00:51:39,860 We're still getting the error. 719 00:51:39,860 --> 00:51:46,610 So what we'll do is we just get back and then have that and then we just go as we started initially. 720 00:51:46,610 --> 00:51:47,840 So let's run this. 721 00:51:47,960 --> 00:51:55,490 They should work now from here we are going to continue with what we were about to do with the custom 722 00:51:55,490 --> 00:51:56,750 training process. 723 00:51:57,050 --> 00:51:58,820 So let's scroll down. 724 00:51:58,820 --> 00:51:59,330 Okay. 725 00:51:59,330 --> 00:52:01,520 So we are going to this point. 726 00:52:01,520 --> 00:52:05,840 We have to anyway, we run this already, so that's fine. 727 00:52:05,840 --> 00:52:14,930 Now let's command this one and then run the cells, run this neural and method, and then start the 728 00:52:14,930 --> 00:52:15,980 training process. 729 00:52:15,980 --> 00:52:17,860 As you can see the training now complete. 730 00:52:17,870 --> 00:52:19,700 Let's go ahead and launch Tensor board. 731 00:52:19,700 --> 00:52:24,050 We have the last values and accuracy values which have been locked in. 732 00:52:24,080 --> 00:52:29,870 You could see them here just reduce this validation, accuracy and validation loss. 733 00:52:29,960 --> 00:52:36,050 Now, from this, we now going to look at our one DB logs. 734 00:52:36,050 --> 00:52:42,680 So we click on this run and you see we have this log data in your you see we have the train, we have 735 00:52:42,680 --> 00:52:49,010 the plot for the loss, we have the plot for the accuracy and we have the plot for the global step. 736 00:52:49,010 --> 00:52:50,180 Then for the validation. 737 00:52:50,180 --> 00:52:55,430 You see, we have this here, we have the global stab validation, loss and validation. 738 00:52:55,430 --> 00:53:02,330 Accuracy is actually what we have with tensor board without adding any extra line of code for the one 739 00:53:02,330 --> 00:53:02,870 DB. 740 00:53:02,900 --> 00:53:10,760 That is the only thing we actually did was at this point here, at this point here we said we wanted 741 00:53:10,760 --> 00:53:12,110 to sync tensor board. 742 00:53:12,110 --> 00:53:16,550 So this all we need to sync tensor support with one DB. 743 00:53:16,550 --> 00:53:22,550 So if you have been using tensor board or if you are using tensor bar on a particular project, that's 744 00:53:22,550 --> 00:53:27,290 all you need to log your information now to one to be apart from this you'll. 745 00:53:27,450 --> 00:53:31,140 Notice this tensile board here, so you can click on that. 746 00:53:31,140 --> 00:53:33,060 We're spinning up your pencil, for instance. 747 00:53:33,060 --> 00:53:38,760 Hang tight, take about 30 seconds and we'll keep it on line as long before even completing the sentence. 748 00:53:38,760 --> 00:53:39,810 You have this already. 749 00:53:39,810 --> 00:53:42,360 So you see, we have the stencil bar, which has been locked. 750 00:53:42,390 --> 00:53:45,120 Here we have the spy torture profiler. 751 00:53:46,670 --> 00:53:49,620 And then time series killers. 752 00:53:49,640 --> 00:53:51,690 Just exactly what we have with ten so bored. 753 00:53:51,710 --> 00:53:52,850 So that sounds cool. 754 00:53:52,850 --> 00:53:56,930 We have the locks, we have the different files which have been saved. 755 00:53:57,020 --> 00:53:59,210 System information. 756 00:53:59,210 --> 00:54:07,280 And we have this charts and then overview where we could see all this information and then get back 757 00:54:07,280 --> 00:54:08,120 to our runs. 758 00:54:08,120 --> 00:54:08,930 So that's it. 759 00:54:08,930 --> 00:54:16,430 We've seen how to sync Tensor bar with one to B, and so now you're ready to track your experiments 760 00:54:16,430 --> 00:54:17,960 with one to be. 761 00:54:17,990 --> 00:54:21,110 Thank you for getting round up to this point and see you next time.