Hi there and welcome to this new session in which we are going to dive into testing out our YOLO model,
which we trained in the previous section.
And to carry out this testing, we are going to make use of the cocoa data set.
And so now we'll pick a random some images from this dataset and test them out with our model.
Now the first thing we'll do is load the model.
So there we go.
We load the model, we're going to create this outputs directory and then we'll specify the path to
our test images.
Now let's dive into this test method.
Here we're going to take the file path as a test path, and then we have this image on which we are
going to put the band and boxes and the classes.
Then, given that we are not or we did not use OpenCV to load the image previously, we are going to
go with the exact same process we had already.
That is, we read the file, we decode and then we resize.
So this is what we had and once we have this image, we pass this into our model.
Now the output of our model will be something like this.
So we'll have this seven by seven by 30 tensor.
Now, remember that for a cell like this one, whether it's no object, we suppose that this person
here is an object.
So like this one where there's no object, we have a zero for the first position and then for the next
position.
We'll have this for bounding box positions or bounding box values, and then we'll have another zero,
and then we'll have four and then we'll have now the 20 values for the class.
Now, given that there is no object here, all this wouldn't really matter.
And so that's why we are only going to take the boxes where this two values, this value and the value
is going to be greater than or equal a certain threshold which will define to be 0.25.
So that set is so like this one, which is the center of our object.
Remember, we have this object here and we have this center.
So you would have this cellular within the center, meaning that if we take this, let's take its values.
If you take this, you would have a let's than 0.75 and then you have four values for representing this
position or its bounding box.
And then we'll have maybe another say, 0.9.
Then we have four values and then we have different values here for the class.
And so the idea here is to get all these different positions where we have values greater than 0.25.
Now, you should know that we could pick a threshold of, say, 0.5 or 0.7 or 0.2 as we've done.
And it really depends on how this threshold affects the model performance.
So we pick the 0.25 because it performs better than picking 0.5 as we 0.5.
Many objects were missed out.
So that is it.
We move to the next.
We simply just gather all this different outputs.
That is, we have the object positions from here.
And then to obtain this different outputs here, we'll take the output itself, which is all this,
and then based off the positions, we'll get this output.
So if we do here, if from here we do, we print out the object positions, objects, positions, and
then below we print out the selected output.
That's here we have this exception.
Okay, so let's run this.
You find that for this image, for example, we have this positions.
That is, let's get back to this.
So it's telling us that this is the position for three.
We have an object.
So we go zero one, two, three, four, and then zero one, two, three.
So an object is found here for this image we have.
Now, the reason why we're having this duplicate is simply because it happens that for this first position,
that's for this first core, there is an object.
And for this other score there's also an object.
You could see that from your that we actually compared the zero position and this fit position, which
is this 0.75 and this 0.9 respectively.
And now we could take a closer look at this selected output.
You see that this is 0.96.
So here is 0.96.
Well, it's for this position.
So we could we could just take all this off.
So if we take all this off, all this here of this is dirty.
You find that this which is for three, the object for this specific image is for force at this sell
for three.
And the outputs this year and you see the first position, 0.96, then we have the four for the bounding
box and then the next 0.98.
So it shows clearly that the model is shorter, there's an object there.
And then from there we have this four again and then now we followed with this 20 different classes.
Now, just by looking at this, just by looking at this here, 0.30.3, we can look for the one with
the highest value.
Okay.
It shows clearly that this is the class with the highest value.
And so from this we know that there is an object at.
This position and that object belongs to this class.
And obviously we have the bound and box surrounding the object.
So now that we've had this different values right here, the next thing to do will be to convert this
bond and boxes.
That's this into the X mean y, mean x max y max format, which we are then going to use open c v to
draw this bounding boxes on the image.
Now we're going to go through each and every object position which we've had already.
From here.
You can see here we have this object positions 043043.
Well, it's a duplicate, so let's focus on just a single or this single one.
So we have 043.
That's essentially the position for three as we've seen already.
And to obtain the output box, which is this value, this value and this value, what we'll do is we'll
get we have the output and then we'll say output position zero position is from the object positions.
Remember, the object position in this case is 043.
So when you say position zero, you're taking zero.
So here you have zero.
Position one is for position two is three, and that's how you select this specific output here.
Now, once you select the specific output, the next selection you want to make is that of the bound
and boxes.
Now when JZ equals zero, here you have zero times five is zero.
So you go from one write up to zero plus five, that's five.
So we go from one up to five, obviously one up to five minus one.
So we have one.
So we see this position one here, position two, position three and then position four.
So that's how we select this year from our output.
And then notice that given that we, we, we have two different bounding box predictions you have this
year.
So for the first one you have 1 to 5.
And for the next time we get into this loop we have since is one we will have one times five, which
is five, five plus one is six.
So we go from 6 to 10, which now is this year.
This is six, seven, eight, nine.
Well, six, seven, eight, nine, six, seven, eight, nine.
We're going from 6 to 10 minus one.
So that's it.
So this is how we obtain the output boxes.
That's how we obtain this here of this bounding boxes.
And then given that, as we said already, we need to convert this into this X mean y, mean x max y
max format.
The first thing we'll do is convert it into the x center y center format or x and so y center width
height format, which is what we do here now to obtain the X center from this year, from this 0.53,
for example.
And so let's suppose, for example, that this 0.53 is at the position of four.
Three does this well, it's four three.
So we go 012340123.
So we have this here around the center or 0.530.17.
So it's our one year we have this.
And the idea is to obtain this value with respect to this full image height and image width.
So first, since first we know that the distance from here to this position here is simply four divided
by seven times 224 times 224.
And that's simply because all this is one, two, three, four, five, six, seven.
So because the full image width is 224, it means getting right up to this position here is four and
seven times 224, which is in fact four times 32, because 224 by seven is 32.
So you take this year.
And multiply by 32 and you get this distance from your right up to this year.
Now, to to to account for the fact that we have this 0.53 year 0.53 will take 0.53 times 10 to 2 because
this full cell is 32.
So 0.5, three times 10 to 2 plus four times 32.
So to obtain this distance, we have four times 32 plus 0.5, three times 32.
Now, for the height, because for the for the Y center, because this is X center, we'll still go
zero one.
This is one, two, three.
So we still have this here.
Again, divide it by seven times 224.
So it's going to be three times two to.
Plus 0.17 times 32.
So that's it to to obtain the why center we have the this position that's three times 10 to 2 plus 0.17
times 32 to find this distance here.
So that's exactly what we do right here.
You will notice we have this post one.
This post one is actually from your this is post.
So post one is four, which is multiplied by 32 because this is this post one plus this output zero
times all of this times 32.
So you have this times 32 plus this output box zero output box zero is 0.530.53 times 32.
So all this is just like saying we want to have four plus 0.53 and then all of this times 32.
So that's what we do here for the Y center is the same.
We have post two.
Now this is a year three times 32, plus this output box one, output box one is this 0.170.17 times
32.
Remember we got output box from year and it coincides with 0.17.
So that's it.
We obtain X enter and we obtain y center and the next thing we want to do is obtain the width and the
height for width.
And the height is going to be easier because when encoding this we simply divide it by the complete
width and the complete height.
So now we'll simply multiply by the height and then multiply by the width to obtain the width and the
height.
So that's how we obtain this from here.
We could now live from X and to y, center x with y width to x mean y, mean x, max y max.
Now, if we have a bounding box like this, let's say we have the center and we know the width and the
height to obtain the x mean we could simply take this center minus half of the width because this distance,
let's say this is the origin, this distance here, here is x center.
If we subtract half of this width, then we'll have this distance which will take us to the X mean,
and then we do the same for the Y.
That is, we take this distance right up to the center and then we subtract the y, the, the Y height
does the height.
We subtract half of the height, not not the height, but half of the height.
Then we'll get to this position.
That's why I mean, that's what we do here.
We have X enter minus half of the width and then we have Y center minus half of the height.
And then for the max we have X center plus half of the width.
So if we want to get this position here, we'll take this plus half of this width, which would take
us to this point here.
And then if we don't have this for the for the Y, then we'll take this distance, this distance plus
half of the height, which will add up to just this point right here.
So we have X mean y, mean x, max, y max.
Now be careful in the case where the X mean happens to be less than zero one to fix this to zero.
If it's less than if the women is less than zero, we fix that to zero so we don't have negative values.
If this is greater than the width, we fixed that to the true width.
If the Y max is greater than the height, we fix that to the height.
And so once we have this, now we opt in our final boxes.
Does x mean y mean x, max, y max?
And then not to forget the fact that we have some classes, so we are going to simply get the class
with the highest probability score.
So we just do this at Max and we make use of the selected output.
Remember, the selected output is what we had already seen here.
So based on this, we are going to take the last 20 values this year.
We get the max, which happens to be this year, and then we get a class which corresponds to this position.
So that is essentially what we do right here.
We we have that position and then we, we have its corresponding class.
We make sure that this a string and then we add that to our final box.
So that's it for our final box.
We also need our final scores.
We understand why we need this final scores later.
For now, just take the final scores.
We make sure that we have again, you see, if JZ equals zero, then yeah, we have zero.
So we have the selected output I and we pick zero mean that we pick in this.
And then if J is equal, if JZ equals one, then year would have one times five, that's five.
So that's a fit position which is going to be this probability score.
So essentially we get in the probability scores for the two.
Predictions.
Remember, we had actually we actually have two predictions.
So we get the probability scores and then we print them out.
So we see what this looks like.
There we go.
As you could see, we have 0.965, which makes sense.
This is 0.98.
Here we have 0.965 and we have 0.985.
Well, this is because there is some duplicates here.
So that's it.
Then we also see the final boxes.
You see the class person, C person, person, person.
Now we're going to see how to eliminate this duplicates shortly.
And that's it.
So for now, we have understood how we could get from this model's outputs to then be able to obtain
this final boxes and the final scores.
And now the next step will be to get into this non max operation.
So we have maybe seen already the we looked at the normal suppression already in theory.
Now we'll see that with TensorFlow, it's actually very easy to implement this.
But before implementing, let's take a look at what is all about.
Let's suppose we have an image like this and then we have this object.
Let's say we have this object here and then we have some bounding box.
We have this bounding box.
Remember, for each cell we have two predictions.
So let's suppose that our cell predicts this and that same cell predicts again another bounding box
like this.
All this for this same object.
Now what we'll do is with a normal suppression algorithm, we are going to compare this to probabilities
and say, okay, which one has the highest probability?
If it turns out that it's this one with the highest probability.
So let's say if this is 0.98, oops, let's say if this is 0.98, and then this one year is 0.96, and
then this two are produced in this same object, then we are going to discard this box.
So hence the term non max suppression.
So we'll suppress this box and we'll be left only with this.
So that's how we are going to also discard those duplications.
Now going back to the implementation, all we need here is just this normal suppression method we have
here.
So we have this normal suppression from TensorFlow image.
We specify the boxes.
So we have this boxes here.
Now know that our boxes from here included the classes, but we do not need that here.
So we just, as you see, we pick the first four elements as essentially x means y min, x, max, y
max.
We pick this first four boxes.
Then we also make sure we present in the scores.
Remember, in the normal suppression algorithm, we need this class to be able to discard certain boxes
which have or which are not the max scores, which we do not have the max scores and which are producing
an object which has already been predicted by another box of higher score.
So that's why we need to pass in the score here.
So essentially we pass in the boxes, passing the scores.
We want to specify the total, the maximum output size.
Here we just pick 100.
We we do not expect to have more than 100, but depending on your task, like you could have a task
where you generally have maybe say, 150 objects to be detected at once.
In that case, you need to increase this max output size to maybe say 1000.
Now we have this IQ threshold right here to understand this concept of the IOU threshold.
Let's take back our example we had here.
If we have this here, if you have this example, in order for this algorithm to know that these two
boxes are trying to predict the same object and we want to actually discard this one, what we'll make
use of is this IOU threshold.
So remember, we are seeing the IOU already.
So if you have two boxes like this, these two boxes will compute the IOU score.
That's essentially we'll look for the intersection between these two boxes, which is this area, and
then divide it by this total area occupied by this two boxes.
So in this case, it's all this area right here.
So let's let's let's have it back.
We have this here is the intersection.
And then this year totally is the union.
So we take that intersection divided by the union to obtain the IOU score.
Now, if that IOU score is greater than the IOU threshold, like in this case, let's make let's specify
an IOU.
Treasury of 0.5.
Is greater than 0.5, then we are going to discard this box.
So we're going to discard this.
If that ice core happens to be greater than 0.5.
Now, if it is less than 0.5 minute, if we have a box like this, let's say we have a box like this
where this area, this area here.
Is divided by all.
This area is less than 0.5, then we are not going to discard this box.
So we consider that this box is trying to predict is for a different object and not this other object.
So this this tree I trash whole year permits also determine whether two boxes are trying to predict
the same object or not, essentially.
So that's it.
And then here we have this score threshold which is set to negative infinity.
Now, the documentation is said that the score threshold actually is as already flow tensor, representing
the threshold for deciding when to remove boxes based on score.
So if you have a score threshold of zero points, for example, one, then what you are saying is all
boxes which are which have a score of less than 0.1 are going to be discarded straight away, mindless
of or regardless of whether they overlap with a box of higher score or not.
So that's it.
We get back here and then now we could print out let's print out our non max suppression output output.
Let's run that.
Now, one thing you notice here in this output is the fact that we have a single element.
Now, what the single element actually means is between all these four options we had, that is here
we have this person, we have this person, we have this person and this person.
Only this one year at this position, one is going to be left.
All the rest will be discarded.
And to understand why they're discarded, you could look at this course.
This is 0.96.
This is 0.85.
This is 0.96.
This is 0.980.
This is 0.985, not eight five.
So what we're saying here is because this one has the highest probability and because it overlaps with
the others like you see here, this one and this one will overlap because actually represent the same
person.
Then this others will be discarded.
Now, in the case like this, where we have this exact same box with exact same probability, one is
going to be left out and the other one left.
So that's it.
We have our output.
Now we know that we only have a single box instead of all this four boxes.
So we have our non max separation output.
The next step will be to show visually what our predictions look like.
Now here you'll note the fact that we are going to write this in this image.
So we will draw the bounding boxes and put the text on our image only for AI in our normal expression
output.
So in this example, we're going to do that only once.
Unlike the case where if we do not have non max pressure would have had to do that four times.
So now we're doing this only once because after normal expression, we're left with only a single box.
Now take a look at what we have here.
We have the X mean, we have the x max.
Remember from the final boxes here or we had year X mean y mean like you see we have the X mean we have
the Y mean sorry, not the x max, we have the x max, we have the y max and then we have the color
for the box and that's it.
So that's it.
We now put the text, now we put in this text based on certain position.
So the text itself is going to contain the class from the final box.
You see, we take the last element.
Remember from year we had the class.
And then once we obtain that class, we write that that's the text, we're going to put the text, and
then the position of that text will be based on the X mean y, mean values.
So you see we go, we go to X mean, but for Y mean we step 15 pixels downwards.
So that's it.
We define the font and the color and that's it.
So we've put out this text and then now we're ready to write this out in our new image.
So we create this new image and then we resize it.
Obviously the content of this image is this year, this image which on which we have written the all
or which we've drawn the rectangle, or that's the bounding boxes and the text.
So let's run this now completely.
And then you see again we have one and we open this up.
Okay, So we should be able to have our output.
And there we go.
You see, we have a person notice from here.
From here we had decided to go 15 steps.
Oops, Let's take this off.
We had decided to go 15 steps.
Let's go downward.
Here we just had to go 15 steps.
I see.
The text comes slightly down.
So if you if you if you don't have this and then you do this, see, it goes up and it's not very visible.
So let's have that back.
You could obviously change the color.
So let's say 225 and you could play around with all this different parameters.
So that's it.
Well, let's get back to the other color, because it's actually better.
Let's say we want to have one.
So now we could run this for all the different files and see what we get.
Well, before even checking on that, let's suppose that we do not have this num suppression output.
So let's leave out the normal suppression algorithm and see what our outputs will look like.
Let's have your four I in range.
The length of the final boxes.
The final boxes here had a length of four.
We had four outputs.
Let's take this off.
There we go.
And then run this again and see what we have.
So we have outputs.
Oops.
Well, we already had several different predictions.
Like, okay, let's let's take this one for example.
You see here we have this one.
No, let's take let's let's not have that.
Let's say we want to have year 40.
Let's get back to 40 and 40.
Okay.
So one thing you can notice here is the fact that this you see we have these two predictions for this
person and that's not what we want.
So you see that the fact that we add this and normal expression here permits us to remove some of those
boxes.
So this threshold is what you play around with to ensure that you have a single box for a single object.
So here you can see that this model does well in predicting the location of this train and knowing that
it is actually a train we have.
This here predicts that this is a person we have this predicts this person, but unfortunately doesn't
get this.
Other people here gets the airplane.
This person.
Well, doesn't get this person does quite well here.
See, this is the dining table.
This person, this person and this person.
So that's great.
Yeah.
We have the TV monitor, though.
It doesn't get sort of monitors.
Okay.
So from here, we also have this bus.
Unfortunately, we we it predicts to purchase a car also predicts two cars maybe due to this other car
being here.
Then we have this dining table and this person, we have this person and the dog.
We have this person.
This person.
Well, still a dog here, but that's not right.
Person.
Person that doesn't see this other person sees these two people here.
Your sister's persons, though the bottom box isn't quite well put out.
Then we have the cat, we have this person, we have this car and then we have this person doesn't see
the dog we have here.
See these cows?
Well, this this particular image was gotten from the paper.
So just basically crop this from the paper to test it out.
So it is a person but doesn't see that this to a dog so actually locates them quite well.
So that's it.
Yeah.
It is a cat but doesn't see this dog sees a person, sees a cow, sees the person, person sees two
people, sees this person and this person though the bounding boxes aren't quite well put out.
Unfortunately.
Yes.
This car's not quite correct then.
Yeah, we have motorbike and person, but this should be two motorbikes actually.
Okay, so that's it.
We've just tested out our model.
We'll see how it does or how it works with our images on which it has never actually seen.