In this video tutorial, we will look at how we can fine tune the YOLO model on a custom dataset for
personal protective equipment detection.
We will be using the Personal Protective Equipment data set available on Roboflow.
We will we will download the Personal Protective Equipment data set from Roboflow into this Google Colab
notebook.
And then we will fine tune the YOLO v9 model on this Personal Protective Equipment data set.
As now you can see the demo of our output videos, which we have got after doing Personal Protective
equipment detection.
On these videos, we will also see how we can do personal protective equipment detection on images and
on videos.
I have already prepared a complete notebook for this tutorial, and we will be using YOLO v9 Extended
model in this tutorial.
So let's get started.
So now here you can see the complete notebook over here.
So I have written all the code and all the added all the required explanation as well.
And this will provide you a step by step guide how you can fine tune the YOLO v9 model on any custom
data set.
But in this tutorial we will be fine tuning the V9 model on Personal Protective Equipment data set,
which is available on Roboflow.
You can get that, uh, you can get the data from any other sources as well, but I will be using the
personal protective data.
Roboflow and I will show you the dataset as well.
So, uh, the dataset consists of seven different classes.
As we go on we will discuss that as well.
So let's get started with it.
But before you run the script, please make sure that we have selected a runtime asset for GPU.
Google Colab offers free GPU and we can use this as well.
So here you can see we have the yolo v9 GitHub repo.
And over here you just the first step.
You need to clone this GitHub repo.
So you can just copy the link from here.
Okay.
And you can simply add this link over here.
After editing this link, you will.
We will clone this GitHub repo or in this Google Colab notebook over here.
So now.
You can see here.
If I just refresh this now, you can see the, uh, this completed GitHub repo, which we have cloned,
and we have all the required files over here as well.
So now I will just start this, uh, yellow benign folder as my current directory.
So to set any folder as your current directory you will write CD.
And now if I just run this cell this yellow benign folder is being set as my current directory.
So if you want to do object detection on images or video, or if you want to point to the yellow benign
model or any custom data set, we need to first, uh, install all the required packages.
So over here, if you just open the requirements dot txt file, you will find all the packages that
are required to train detection or run the validation script.
So you can install these packages one by one.
Or if you just write pip install minus r requirements dot txt, it will install all the packages that
are mentioned.
This requirements dot txt file.
So now I will just run this and it will install all the packages that are mentioned in the requirements
dot txt file.
So this will take two seconds before all the packages get stored.
So it takes around 1 to 2 minutes for all these packages to get installed.
And it also depends on the, uh, speed of your internet as well.
So now you can see over here I think two more packages are left.
And, uh, they will take a few more seconds to go.
Okay.
So you can see this is in progress currently.
So let's wait for a few seconds for it to get complete.
Okay.
So like you can see that these are the size of different packages.
So after we install all the required packages, then I will download a model weights from the yolo v9
GitHub repo into this Google Colab notebook.
So if you just show you the yolo v9 GitHub repo over here.
Okay, so if you just, uh, go to releases from here.
Here you will find the Yolov5 model.
So Golovnin comes with, uh, four different, uh, models.
Yolo v9, small, yolo v9 medium, YOLO v9 compact, and YOLO v9 extended.
What are currently the weights of YOLO v9 Compact and Extended are currently available, so if you want
to download this model to your Google Colab notebook, you can simply copy link address from here.
And if you just see over here, uh, we currently we only have available a YOLO v9 model weights for
Compact and extended model or the other model weights are not available.
So.
I will just add this link over here, which I have copied and.
Before we go ahead with this, uh, we will set, uh, this, uh, yolo v nine folder as our current
directory.
We have all those already set this, but we will, uh, save this, uh, directory name into our variable,
which is home.
Okay.
So.
So now inside this we will create.
So using mkdir we will create a new directory by.
The name bears inside this line.
So inside this yolo v nine folder we will create another directory by.
The name bears.
Inside this weights are folder or directory.
We will save our uh YOLO nine compact and extracted model weights.
We will not be using the compact model weight.
We will only fine tune the YOLO nine extended model.
So now if I just run this cell.
So over here you will see we have created a weird spreadsheets.
It's empty.
So now we will, uh, download the YOLO v nine compact and YOLO in an extended model weights into this
directory.
If I just run this cell over here.
So now if you can see over here I have downloaded the Yolo B9 Compact and Extended model weights you
can find over here.
Or you can simply see what different folders you have or what different files you have inside this weights
folder.
So we have Gallery nine which is Gallery nine compact model and YOLO v nine extended model.
I already told you YOLO v nine comes with four different models.
Gallery nine small Gallery nine medium Gallery nine compact and extended.
But currently the weights for Gallery nine Compact and extended model are available.
So we are just using that.
So now I will download the data set from Roboflow into the Google Colab notebook.
As I already told you that we will fine tuning the YOLO benign model on Personal Protective equipment
data set.
So if I just open this, uh, let me just open the data set over here and show you how our dataset looks
like.
And the dataset is available on roboflow.
And you can see that I have written this own dataset, and the dataset consists of 311 training images,
652 images are for the validation and 327 images are for the test set.
So we are taking 70% of our data for the training purpose, 20% of the data for the validation purpose,
and 10% of the data for the testing purpose.
And here my dataset consist of seven different classes which include dust masks, gloves, protective
boots, protective helmet, i-we're, safety vest, and shields.
So these are, uh, these are the seven different classes which we have in our dataset.
And over here, if I just open this data set, you can see over here, uh, these are the protective
ports.
This is the safety vest.
These are the, uh, safety vest.
This is the protective helmet.
Okay.
So.
And if I just see the health check over here.
So in the health check, we can find that, uh, uh, if our data is completely balanced or not.
So we can see that, uh, out of seven or the classes are balanced while three classes are not balanced.
So you can see over here.
So this is how a data set looks like.
Okay.
So now you can see a no augmentations were applied in our data set because we usually apply a, uh,
augmentation at that time when we don't have enough images in our training data.
So to increase the size of our training data, we usually apply augmentation.
But in that in this case, we have already a 2311 images in our training data.
So we don't need to apply augmentation.
So now I will just download this data set so you can simply, uh, create an account on Roboflow.
So if you just try it over here I already have an account on Roboflow.
So but you can simply, uh, sign in with Google or if you already have an account, or you can enter
the credentials.
So now you can see I've just logged in.
It's very simple to create an account on Roboflow, you can simply create.
And if you want to download this data set you can simply select the format.
I will select YOLO v9 format and click on show Download code.
And if you are using uh, Jupyter Notebook or Google Colab notebook, then you will simply copy this
code from here.
Uh, but if you are using Visual Studio or PyCharm ID, then you will just copy this code and paste
in the terminal.
It will download the data set from Roboflow into your IDE.
So I will simply copy this code from here.
And I will just add this code over here.
And uh, so now it will download a data set from Roboflow into this Google Colab notebook.
So this might take a few more seconds before we are able to download the data set.
Okay, so now you can see that warning the following packages were previously imported in the runtime.
You must restart the runtime in order to use newly installed versions, so I will just wait for the
data set to download complete.
Then I will just restart.
Okay, so now you can see the tick mark over here.
It means the data set is downloaded.
So I will just restart the session so the package installation can be completely done.
So now if I just restart a session, then I just need to select uh, this V9 folder as my current directory
again.
Okay.
So now you can see over here.
So now if now we need to train our custom model.
So if I just open the YOLO like GitHub repo over here.
So you will find all the steps for the training are written over here.
So you just see over here.
So now you can see that if you want to train a a, so if you want to train your over nine models you
will run that train, dash your.py step.
And if you want to train a grand model you can run the train.py script.
But here I am going to train the yolo v9 extended model.
So I will just run the python train that.py script.
So you can simply copy this line of code from here, and you can just add this line of code over here
and over here.
I have made few changes.
I have just uh, left made the batch size as eight and I am just train using uh GPUs website that device
zero.
The image size is being set to 640, which is by default.
And I you know that in the start I have downloaded the weights from the GitHub repo into this Google
Colab notebook.
So.
Over here in the weights folder you will find the weights over here.
So I will run YOLO Renai EPD.
So yolo B9 extended model weights.
And I have just uh, my fine tuned model weights will be saved in the folder Yolo v9.
Fine tuning.
So by this by this name of folder will be created inside this folder directory.
And inside this our fine tuned model weights will be saved as well.
So and plus I am fine tuning the YOLO v9 model on for 30 epochs.
You can fine tune the model for 80 or 100 epochs, which is well suited, but in this tutorial I am
just fine tuning the YOLO v9 model for 30 bucks and it gives me good results as well.
Thus here I will just in the data path, I will just pass the name of data dot yml file.
So if you just see this, the this is the data set which I have downloaded.
And here we have the data dot yml file.
So you can see that we have seven different classes in our data set we have which include dust mask
IVR glove, protective boots, protective helmets, safety vest and shield.
And here you can see that training valid learning data set path validation data set uh images path and
test images path.
You can see over here.
Well you just need to add this data dot yml file path over here which I've already added.
And plus.
So we just as we have restart the session.
So you just need to run this cell again over here so that we can set this as our.
Just need to run this cell once.
Okay.
That's fine.
And if you just run this cell now, this will restart the training over here.
It will start the training over here.
So it will, uh, train or fine tune the Yolo v9 model on personal protective equipment data set over
30 epochs, which you have mentioned over here.
I have already done that training, or I have already fine tuned the YOLO v9 extended model with 30
epochs on Personal Protective Equipment data set, which I have shown you.
So now you can see over here and then let me just pull down.
So the training takes a lot of time.
So I'm not going to train again.
Like previously it has taken over 30 epochs.
It has taken 2.6 to 1 hours like it has taken around one 60 minutes.
Uh, to fine tune the YOLO v9 model on personal protective equipment data set for 30 epochs.
And in the end, we got the best model weights and the last model weights model weights on the last
epoch.
And the best model weights represents, uh, the epoch on which the model gives the best mean average
precision scores.
And they did.
The last part basically are the model weights on the last epoch, which is the 30th epoch.
Okay.
And from here you can see, uh, if you just see mean, average precision, uh, like you can see here,
class wise, the mean average precision with the IOU 0.5 and the mean average precision when IOU varies
from 0.5 to 0.95.
So now you can see over here we have the mean average precision for the dust mask i-we're glove, protective
boots, protective helmet safety vest and shield class.
So from here you can see that, uh, the mean average scores are very good, like we have for all the,
uh, two classes we have mean average precision score above 90% for I use 0.5, and for only one, uh,
for only one class, two classes, we have, uh, mean average precision with I use 0.4 scores below
90%, one is 0.845 for IVR.
And for the shield class we have the mean average precision score with IOU 0.5 0.761 or 76.1%, while
the mean average for average precision score when IOU varies from 0.5 to 0.95 are pretty good as well,
which you can see over the values over here.
So the results are very promising for now.
Okay, so I have already fine tuned the okay, just let me just save these changes.
So as I already told you, I've already fine tuned the yellow benign model for the Personal Protective
equipment data set.
Uh, so I after finding running this, I have saved the best model weights into my right.
So I will directly download the best model weights from right into this Google Colab notebook.
And then we will do inference on images and on videos on this best model weights.
So here we have the results which you usually get after training.
So now you can see the mean average precision score is continuously increasing.
And it goes up till zero above 90%.
While you can see that mean average precision score when you varies from 0.5 to 0.95, is also increasing
and goes up to 65% as well, while the loss is continuously decreasing.
So this is very good.
Okay so here we have the confusion matrix.
So the confusion matrix basically tells us how our models handle our model handles different classes.
So over here like you can see this is the confusion matrix chart.
And here you can see we have all our seven different classes.
The mask I-we're glove protective goods, protective uh helmet safety vest, shield and a background.
When the thing is detected and we have the all these class names mentioned over here as well.
So if we just see from the dust mask.
So 94% of the times when there is a dust mask or when the person is wearing the dust mask, 94% of the
times our model correctly detected that the person is wearing the dust mask.
So 0.94 means, uh, 94% of the times when the person is wearing a dust mask.
Our fine tuned YOLO benign model correctly detects that the person is wearing a dust mask.
While 6% of the times when the person is wearing a dust mask, our model is unable to detect anything.
So here you can see 0.06.
So 6% of the times when the person is wearing a dust mask, our model is unable to detect anything.
So next is I-we're.
So it is 6% of the times when the person is wearing the Ibrd's.
Our model correctly detects that the person is wearing the Ivrs while one person died when the person
is wearing dust mask or model incorrectly when the person is so, one person died when the person is
wearing I-we're.
Our model incorrectly classified it as dust mask, so 86% of times when the person is wearing IVR,
our model, correctly classified, detects that the person is wearing I-we're, while 1% of the times
when the person is wearing I-we're, our model incorrectly detects it as dust mask.
While 14% of times when the person is wearing i-we're or model is unable to detect any thing.
Okay.
And again, like you can see over here, cheat code, the shield is 76% of the times when the person
is wearing shield or model correctly detects that the person is wearing a shield, while 4% of the times
when the person is wearing a shield or model correctly, uh, detect it in our model incorrectly detected
as protective helmet.
So 4% of the times when the person is wearing a shield, our model incorrectly detect it as protective
helmet, while 20% of time when the person is wearing a shield, our model, uh, does not detect anything.
Okay, so this is our confusion matrix.
So these are the model predictions for the validation batch.
We have not used these images for training.
So it's always better to take a look and see how our model performs.
So from here you can see we have i-we're shear protective helmet.
And so on so forth.
So the results from here are very promising.
So now we will do model.
Uh, so now we will uh, just evaluate our model on the validation set images and see, uh, how our
model performs.
So now we are just evaluating our model on the validation data set images.
And over here you can see that the results are quite promising.
And you can see over here, uh, we have got very good mean average precision score with IOU 0.5 and
very good mean average precision score when IOU varies from 0.5 to 0.95.
So now we will be doing inference on images and videos over here.
So you can download these two videos.
I have already placed those videos into my drives or will directly download those videos from drive
into this Google Colab notebook.
So I'm downloading these videos.
So now, first of all, uh, if you just see over here, uh, so now to do inference with your model.
So you will just Python detects that your.py.
And if you want to do inference with Glenn model you will run to detect.
So as I want to do inference with yolo v nine model on image.
So I will just run detect-dual.py.
And if I want to do inference on videos I will run the same script.
And here I've just passed the source image and I'll just pass the best model weights which I have downloaded
from drive.
And I've selected the devices GPUs and GPU.
And if you decide you want to use CPU, you can simply write CPU over here.
Okay, so now, uh, our output image is saved in this folder.
So you just see over here runs a Dag, and in this folder we have the output image.
So I've already added this path over here.
So let me just display this output image over here okay.
So I think, uh, there's an error.
So I think I've just, uh, I just copy this from here and see where I have placed my image library.
Here again?
I think I have just, uh.
Just give me a minute.
Let me see where we have.
Or we can just, uh, initialize this library again.
Okay, so here you can see we have the library.
So this, uh, image library allows us to display an image into our Google Colab notebook.
So now it will display the output image over here.
So let's plot this cell again.
So now you can see, uh, we have tested on this random image.
And we are able to detect a protective helmet and the safety vest that this person that this person
is wearing.
So now we will just do the inference on the videos, two videos which we have downloaded.
So now you can see the inference has started.
So let's see the how it goes.
So we are using the best model weights.
So our complete video is being divided into three and ten frames.
And we will be doing uh detections on each of the frame one by one.
Like you can see over here.
We are able to detect the protective helmet, safety vest, protective boots and so on.
And the results look quite good.
Like, uh, six protective boots, three protective helmets, six safety vests, five safety vests,
vests the person are wearing.
So.
Okay.
So now you can see over here and let's display the output video into this Google Colab notebook over
here as well.
So this might take a few more seconds before we have the results.
So let's wait and see.
How does it goes over here?
So after this we will be doing detections on this part two videos as we have already two videos.
So now we are doing inference on this video.
After this we will do perform inference on this second video as well.
Okay.
So this might take few more seconds before it gets completed.
Look it is our output video.
So let me download this video first and let's see how does it goes.
So over here, if I just open this video now and let me just uh, navigate my screen towards the output
video.
So over here you can see that we are able to detect a safety vest, protective helmets, protective
boots.
So now you can see the results look very promising from here.
Okay.
And if I just go again over here okay.
And now we will, uh.
It.
And just give me a second.
Now we will load inference on the second video of using the fine tuned YOLO v9 model on personal image,
we have fine tuned on the Personal Protective equipment data set.
So now we are doing inference on the second video over here.
So this might take a few more seconds before it gets, uh.
So the complete video is being divided into 267 frames.
Okay.
Over here.
And like you can see over here.
We are doing detection on each of the frame one by one.
So what's whiteboard.
And it's going quite good.
So it will be finished soon.
So now you can see the processing is done.
So let me display this output video over here.
It.
So this might take a few seconds before we have the output video over here.
Well, here is our output video and let me just download this first.
And let me just open this.
So let me first navigate my screen towards it.
So over here you can see that we are already able to detect the safety wears, protective boots, helmet
as well.
And the results look very promising as well.
So in this tutorial we have learned that how we can do personal protective equipment detection on images
and videos.
And we have also learned that how we can fine tune the YOLO v9 model or any custom data set like you
can also find on the YOLO v9 model, on helmets, data set, or on any cars data set.
Or you can also do on any other data set.
You can also fine tune the YOLO benign model on license plate data set pins and books, data set, potholes,
data set, and so on.
There are so many applications like you can fine tune the benign model on grocery data sets as well.
So there are so many applications that exist, you just need to explore that as well.
So that's all from this tutorial.
Thank you for watching.