WEBVTT 00:00.750 --> 00:08.970 High in past few sessions, we have discussed about depression and logistic regression in logistic regression 00:08.970 --> 00:14.460 and regression, we have used the most used metrics. 00:15.030 --> 00:23.160 The metrics are basically the quality, quantifying or quality of prediction, how we can actually make 00:23.160 --> 00:25.420 sure that bad models are performing well. 00:25.680 --> 00:28.290 These are what the metrics have been designed. 00:28.980 --> 00:35.040 Now, there are different metrics available for regression problems and different metrics which are 00:35.040 --> 00:37.620 present for the classification problem. 00:38.690 --> 00:47.000 So Ascalon is the library, which we have been using and Escalon on Metrick is actually the module which 00:47.000 --> 00:53.060 helps us to select certain metrics which we will be using for these different algorithms. 00:54.580 --> 01:00.710 Now, let us have a look at this particular library and see what other metrics that are available. 01:01.090 --> 01:09.400 So this is the link, which is cyclone dot org slash stable slash module's slash Mauduit evaluation. 01:10.280 --> 01:17.870 From here, you can actually get the details of different evaluation metrics, so here we have certain 01:17.870 --> 01:21.650 sporting barometers like cross-validation. 01:22.850 --> 01:29.810 Great TV cross-validation score, so these are different validation scores, which we can use, then 01:29.810 --> 01:37.940 we have metrics which are the values such as you see a mean absolute score and all different scores 01:37.940 --> 01:39.290 which we have now. 01:39.290 --> 01:43.400 This link itself provides different metrics like this. 01:43.400 --> 01:51.230 If we want to have a classification metric here, each class is individual class and that is no scenario 01:51.230 --> 01:57.980 where one class would be or multiple classes could be do that does not present in the classification 01:57.980 --> 01:58.390 metric. 01:58.700 --> 02:06.590 So classification metric only helps when we want to find out something like either something is a cat 02:06.590 --> 02:15.650 or dog if it is or a sees a problem where we have either two sides, either it would be yes or it would 02:15.650 --> 02:20.220 be an either someone will pass the exam or someone will fail the exam. 02:20.360 --> 02:25.580 So these type of problems will be considered in of classification metrics. 02:29.020 --> 02:36.100 Next set of metrics is being multilabel ranking metric, these type of metrics are something which are 02:36.100 --> 02:39.820 used when we have multiple levels which are applicable. 02:40.210 --> 02:51.360 For example, we have classes like or labels like Don and Chubbie, and we have something like labels 02:51.370 --> 02:56.930 like short, chubby and not chubby or they're not fat. 02:56.980 --> 03:02.620 So these kind of metrics are present and we want to find out which labels are attached to a particular 03:02.620 --> 03:11.130 person so a person can be either a boy or chubby and the person can be chubby or not chubby. 03:11.290 --> 03:14.950 So something will be a combination of multiple things. 03:15.100 --> 03:17.590 So it cannot select only one type of food. 03:18.100 --> 03:25.630 So it will be there can be multiple labels which can be selected that we can have someone else job boot 03:26.290 --> 03:31.330 so they can label selected out of all the five labels or their neighbors, which we have. 03:32.430 --> 03:39.540 So that is the multilabel metric, then we have regression metric, which is where we have the regression 03:39.540 --> 03:40.140 problems. 03:40.440 --> 03:44.640 These are the metrics which are used for continuous value evaluation. 03:44.910 --> 03:48.880 So this is something like some of squared error out of just your data square. 03:48.900 --> 03:51.270 So we will discuss about those next. 03:51.270 --> 03:52.870 We have the clustering metric. 03:52.890 --> 03:57.560 This is something which is used in case of unsupervised learning. 03:57.750 --> 04:03.420 So we will discuss about it separately during the unsupervised learning module. 04:05.080 --> 04:09.860 So let us discuss about classification metric, multiband metric and regression metrics. 04:10.150 --> 04:13.760 So here you can see the definition of these. 04:14.020 --> 04:16.960 So these are different pre defined metrics. 04:17.290 --> 04:22.360 So scoring accuracy could be accuracy, balanced accuracy, average precision. 04:22.630 --> 04:30.930 If one if one micro, if one macro of some negative log loss, then for clustering. 04:30.940 --> 04:32.530 This is what classification metrics. 04:32.540 --> 04:34.840 See, these are different classification metrics. 04:36.300 --> 04:38.550 Then these are different clustering matrix. 04:39.820 --> 04:46.900 Here we have the different regression matrix, so regression metrics are explained variance, maximum 04:46.900 --> 04:53.620 error, negative, mean absolute error, mean negative means squared error, negative root mean squared 04:53.620 --> 05:00.280 error, negative means squared log error, negative media, an absolute error of scored. 05:00.490 --> 05:08.710 This R-squared score is a very majorly used to score because it is having one speciality. 05:09.010 --> 05:17.680 That is when we use our square score and we try to find out different type of columns. 05:17.690 --> 05:18.520 So let us see. 05:18.520 --> 05:22.840 I'm working with a specific let me discuss about it separately. 05:25.600 --> 05:31.450 So this is the R-squared Coalition, this is sufficient of determination, so this is regression. 05:32.350 --> 05:40.660 So the best possible score is won and it can be negative also because the model can be arbitrarily was 05:40.960 --> 05:48.790 now a constant model that always predicts the expected value by will have the value one disregarding 05:48.790 --> 05:52.630 the input feature would get our squared value of. 05:56.000 --> 05:57.590 So the best model will be. 05:58.740 --> 06:07.410 Having value one now, what we will be doing here is that is in case of R-squared, what happens is 06:07.410 --> 06:09.830 legacy say we want to find out. 06:10.200 --> 06:12.720 We have been very busy. 06:12.990 --> 06:21.330 And so out of those 10 variables, let us say I want to find out which we are actually helping me in 06:21.330 --> 06:24.870 finding out the predicting the value which I want to predict. 06:25.290 --> 06:30.390 So what I can do is I can keep my scoring mechanism as our squit. 06:31.540 --> 06:41.290 And then I can try the try finding out the R-squared value by draining my model on the first feature. 06:41.890 --> 06:49.870 Now from the first feature I will see what is the R-squared value if the R-squared value is improved 06:49.960 --> 06:54.950 by introducing the second feature. 06:54.970 --> 06:59.320 Now, then it means that second feature is a useful feature. 07:01.030 --> 07:08.240 Similarly, we can test for the third feature if introducing the third feature actually improves Darmody 07:08.350 --> 07:09.310 performance or not. 07:09.520 --> 07:15.820 So this is how we can actually decide upon multiple features, because if the feature does not have 07:15.820 --> 07:22.460 any impact on the R-squared value, then it means that the feature has no impact on the model. 07:23.650 --> 07:29.950 So this is how our squared value is very important and it is a very useful metric in finding out the 07:30.040 --> 07:31.120 feature importance. 07:31.120 --> 07:38.140 Also, although we have different methods which we have discussed about like feature importance, which 07:38.140 --> 07:46.120 we will be discussing in case of random forest or the opportunity we are using L2 evaluation or evaluation, 07:46.390 --> 07:53.470 which we have discussed in case of meaning and logistic mortgage, so we can use any of those methods. 07:53.470 --> 08:01.930 And other than that, we have different methods as bonders, profiling and ViiV method and the correlation 08:01.930 --> 08:02.850 coefficient methods. 08:02.950 --> 08:09.820 So there are several methods and you can choose any method which you like and maybe use a combination 08:09.820 --> 08:11.090 of all of these methods. 08:11.320 --> 08:13.330 So it is completely your choice. 08:13.510 --> 08:22.770 But I would suggest try all the different methods one by one, maybe in one velzy or one model. 08:22.780 --> 08:28.690 You can try one method under the model, try another method so that you will get to know how to actually 08:28.690 --> 08:29.470 use this. 08:29.740 --> 08:37.780 Because, I mean, unless you try and use all the different methods of finding out the best coefficients, 08:37.780 --> 08:41.890 you won't really get to unless you will try different methods. 08:41.890 --> 08:48.520 You will not get to know which method you are most comfortable with and which methods you are not comfortable 08:48.520 --> 08:54.010 with so that you can work on the methods which you are not really comfortable with and improve your 08:54.010 --> 08:54.910 concepts on that. 08:56.330 --> 08:59.550 So let us go back to the Matrix, which we have. 08:59.720 --> 09:02.810 So these are all different metrics which are available. 09:03.050 --> 09:10.220 So from all of these metrics, the the of using these metrics is just the same. 09:10.430 --> 09:19.220 You will simply import the metric scale on dot, matrix and dot the name of the metric that you want 09:19.220 --> 09:19.780 to use. 09:20.030 --> 09:21.680 That is how you will it. 09:21.890 --> 09:30.260 And then you will simply use it like a cross-validation score for the model for the of the particular 09:30.680 --> 09:34.460 X value and the Y value and then check which one will with. 09:35.490 --> 09:37.240 So this is how you will use them. 09:37.650 --> 09:40.830 So let us go to the classification matrix. 09:43.310 --> 09:51.680 So these are different classification metrics, so you can see we have precision of accuracy, of balance, 09:51.680 --> 09:53.960 accuracy score and cupper score. 09:54.200 --> 09:56.510 So there are a lot of metrics. 09:56.540 --> 10:01.100 The most frequently used metrics are accuracy of. 10:02.320 --> 10:04.630 Then we have confusion, matrix. 10:05.500 --> 10:12.620 Then we have a U.S. school, so here you have the definition of the methods also. 10:12.880 --> 10:19.120 So what you can do is you can read these descriptions and then decide upon that which method you want 10:19.120 --> 10:20.090 to actually use. 10:20.380 --> 10:29.660 So there are a lot of metrics and different methods provide different kind of benefit or under the method. 10:29.920 --> 10:32.880 So that is how you can actually use them. 10:34.070 --> 10:41.050 Then we have this recall score you use, you score, then we have average precision school love loss, 10:41.210 --> 10:44.000 so there are different kind of scores which are available. 10:44.330 --> 10:47.840 So there are scores coming score. 10:47.990 --> 10:53.770 So these having score is basically use when you have something like the actual data so you can decide 10:53.790 --> 10:58.000 upon what kind of data you have and then use that kind of metric. 10:59.660 --> 11:09.580 Then in case we want to have a multiclass kind of a problem, so then there is a different kind of matrix. 11:09.580 --> 11:15.790 So see if you have by any metric, then you can improvise. 11:15.800 --> 11:21.530 And in case you have multiclass or multilabel problem, then you can use a different version of The 11:21.530 --> 11:22.040 Matrix. 11:22.580 --> 11:28.880 So there are different versions of Matrix, such as macro rated micro, which have different kind of 11:28.880 --> 11:29.400 benefits. 11:29.630 --> 11:37.770 So let's say Micro will simply calculate the mean of the binary matrix giving equal weight to each class. 11:38.420 --> 11:45.230 So if we have, let's say, a class which is balanced in nature, then we can use macro. 11:45.650 --> 11:49.520 Now let's say we have some kind of imbalanced classes. 11:49.850 --> 11:56.270 Then we can use the weighted version of the particular metric which we have. 11:56.630 --> 12:05.120 Then for micro, it gives each sample last year an equal contribution to the overall metric except as 12:05.120 --> 12:08.630 a result of some fluid rather than some make the metric. 12:08.750 --> 12:12.870 It will simply put some calculation on top of it. 12:13.130 --> 12:20.490 Now, micro averaging, maybe preffered in multilabel setting, including the multiclass classification. 12:20.510 --> 12:23.710 So in case we have multilabel setting. 12:24.170 --> 12:26.810 So in that case, we can use this microstamping. 12:27.110 --> 12:31.310 So there are these different versions which are available to your matrix. 12:32.470 --> 12:38.920 Then we have accuracy score, which you can use, and violence accuracy score, these are different 12:38.920 --> 12:41.820 versions of the scores which we have here. 12:41.830 --> 12:45.820 We have the confusion matrix, which is, again, very highly used. 12:46.030 --> 12:49.660 And if you want to see the further documentation, you can just click on this. 12:49.900 --> 12:53.980 And the more detailed definition would come up. 12:54.220 --> 13:01.030 Like you can see here, I have clicked on confusion matrix, so it simply tells me the method of using 13:01.030 --> 13:01.130 it. 13:01.720 --> 13:09.240 So I can simply see that, see that it needs the true value, the predicted value. 13:09.460 --> 13:13.330 And after that, all these things are actually are not really useful. 13:13.340 --> 13:15.880 If you want to use them, then you can use otherwise. 13:15.880 --> 13:22.990 These are optional in nature so you can provide the labels or the sample rate or if you want to normalize 13:22.990 --> 13:23.920 the date or not. 13:24.190 --> 13:29.140 So these are different things which you can use in the confusion matrix and it gives the details of 13:29.140 --> 13:30.010 the parameters. 13:30.010 --> 13:34.210 Also what I present here, and it also gives the examples of those. 13:35.110 --> 13:42.370 So this is how you can actually check how you want to use a particular metric and how to actually implement 13:42.370 --> 13:42.650 that. 13:42.880 --> 13:48.520 So in case there is a metric which you want to explore further, you can simply go to the documentation 13:48.520 --> 13:49.520 and have a look at it. 13:49.940 --> 13:55.170 Similarly, let's say you want to use any different version of any algorithm. 13:55.390 --> 14:02.280 So it is not a sure thing that you will always use the same implementation of Decision three or same 14:02.290 --> 14:04.030 implementation of linear model. 14:04.150 --> 14:07.730 So you might be interested in a different type of implementation. 14:07.930 --> 14:11.670 So in that case, simply go to the documentation and have a read. 14:12.310 --> 14:19.210 This is the best prescribed way, which will not just help you now, but also in the future when you 14:19.210 --> 14:22.300 will have to explore different models. 14:22.960 --> 14:30.400 So this is one practice which is very useful and it actually helps and saves a lot of things because 14:30.400 --> 14:36.130 it will not happen that you will always remember how you want to write the code for the confusion matrix. 14:36.520 --> 14:38.890 So you don't need to worry about that. 14:39.100 --> 14:40.770 You will learn that by time. 14:40.960 --> 14:46.410 But for their timing, you can simply go to the documentation and read it out, go to the example, 14:46.450 --> 14:50.050 see how it is being solved, and then do the implementation. 14:51.890 --> 14:59.450 OK, so this is how you will learn slowly and gradually, it is like a best practice, what you can 14:59.450 --> 14:59.840 follow. 15:00.200 --> 15:06.410 So here we have a different classification matrix, which I was talking about from the classification 15:06.410 --> 15:06.890 matrix. 15:06.890 --> 15:10.610 You can see this classification report, which is provided. 15:10.880 --> 15:15.320 This is a very useful tool because it gives the precision detail. 15:15.320 --> 15:18.230 Also recall also if one score also. 15:18.440 --> 15:23.330 So here you can actually compare and how much precision is then how much recall is that? 15:23.540 --> 15:29.000 And then based on the problem that you have, if you want to improve the precision, then you can look 15:29.000 --> 15:30.590 at the precision of the glasses. 15:30.710 --> 15:35.930 If you want to improve the recall, then you can look at all of the glasses and then decide which one 15:35.930 --> 15:36.890 do you want to use. 15:37.900 --> 15:38.240 Right. 15:38.530 --> 15:46.750 So this is what you can do then, apart from that, you can see the details of the autopsy also being 15:46.750 --> 15:47.410 given here. 15:47.620 --> 15:48.930 So you can use that. 15:49.240 --> 15:55.780 But these are the major methods which you will be using other methods you might not really use that 15:55.780 --> 15:56.080 often. 15:56.320 --> 15:59.340 So here you have the autopsy documentation. 15:59.560 --> 16:01.920 So this is the documentation of Orosco. 16:03.440 --> 16:09.380 Here you can see you just need to provide the zip code, which has been imported from on dot matrix 16:09.680 --> 16:17.660 and give the value and divide predicted value and the label details, and then it will give you the. 16:18.720 --> 16:19.950 Oh, itself. 16:22.620 --> 16:30.210 So you can bring the code for yourself or compare the values and then you will get to know which value 16:30.210 --> 16:31.950 you want to pick for this particular. 16:33.570 --> 16:39.050 And you can generate different goals for all the models which you have prepared. 16:39.330 --> 16:40.020 So let us see. 16:40.020 --> 16:45.360 You are applying a classification problem, then you can implement the classification problem using 16:45.360 --> 16:50.570 logistic regression, also decision, but also random forest also exposed also. 16:50.760 --> 16:56.910 And then create Orosco for all of these modules and see which one has the maximum. 16:57.110 --> 17:02.520 You see the maximum area and the goal and decide which one you want to pick up. 17:02.940 --> 17:05.050 So these are different things which you can use. 17:05.070 --> 17:07.350 There are a lot of tools which are present. 17:07.650 --> 17:10.140 You just need to learn and explore those. 17:11.950 --> 17:18.040 So here you can see here we have all the models which have been come back and you can clearly see that 17:18.040 --> 17:22.570 the blue light blue one is the winner, so which is for the plus one. 17:22.600 --> 17:23.530 So this is the winner. 17:23.560 --> 17:26.170 This is the best model which has been created. 17:27.660 --> 17:34.700 So similarly, you can go for the multilabel problem then, in case you want to find out the regressions. 17:34.700 --> 17:38.720 So here you can see what a different regression matrix. 17:39.000 --> 17:46.590 So we have this explained variance metric by which you can see how much variance is explained by a particular 17:47.520 --> 17:48.010 column. 17:48.330 --> 17:56.370 So this helps in using this helps in finding out if a particular variable is actually important or not, 17:56.400 --> 18:05.880 or is it actually having some impact on finding out the target or not how useful the variable is? 18:05.880 --> 18:08.880 That is what this explained variance code will tell you. 18:10.760 --> 18:16.460 Similarly, you can find out the mean absolute error and compare different mean absolute errors, the 18:16.460 --> 18:23.230 mean absolute obscure, the mean square logarithmic error, all these values need to be closer to zero. 18:23.480 --> 18:26.450 So if the value is close to zero, then we're good to go. 18:26.780 --> 18:31.040 And regarding the school, the school has to be close to one. 18:31.640 --> 18:36.570 Apart from that, all the other regression metrics need to be close to zero. 18:36.830 --> 18:41.030 So that is all you can actually find, though, then use different metrics on here. 18:41.070 --> 18:48.140 We have all the implementations so you can try these implementations and check the implementations, 18:48.140 --> 18:55.220 which I would be giving in my code so that you will have enough documentation and you can learn a lot 18:55.220 --> 18:55.850 from this. 18:57.150 --> 19:04.470 So this is what this particular session in the next session, we will go ahead and learn about the decision 19:04.470 --> 19:04.560 to. 19:06.330 --> 19:06.930 Thank you.