Hi and welcome to Lesson 24, where we take a look after motion tracking and means we're using the means
shift and camshaft algorithms.
So go ahead and open that notebook.
So let's look at our libraries here.
You can run.
This could have already run this, so don't want to overwrite the images and have to wait.
So let me explain to you what tracking is.
So imagine you have a moving person or moving vehicle from like your CCTV video and you want to focus
in on that person you want to like basically draw a box and move that box over the person as he or she
or the car moves in the video.
That's what tracking is.
So let's take a look at two very old algorithms because many different camshaft have been around for
at least 10 years or more, and we will see how it works.
So let me explain to you what mean shift is.
So the principle behind means shift is that imagine we establish a window with is over here.
Let's wait for it to go back and establish a window appear.
And what you do.
You move this window iteratively to the the most intense parts of the frame.
But how is that done?
Well, it's done because we need to consider the histogram of of that of say, let's say it's intensities,
color intensities in the initial bounding box that we established.
And then you just set some criteria to look, move around and look for the next brightest spot around
that image.
And you just move it iteratively.
So you're moving toward the densest area of intensity in this case, and you can use anything you want.
You can use intensity of red blue green intensity of, say, saturation and HSV color space.
So that's effectively what it is.
And you can read more about this algorithm here if you click the link.
But now let's take a look at how the actual algorithm is implemented.
So since it's a video, we're doing all of our video writing back here so we can do split in club.
This is our initial location.
So we hardcoded initial location of the bounding box that we want to use and call that our truck window.
And then what we do is set up the array for tracking.
So we have that.
We have the truck windows parameters here.
So we extract the initial frame.
We converted to HSV.
We do a mass to filter on the most intense areas of that image.
You can see it's zero to 180 in each, which is so you.
And then you look at the saturation as well and the values that we're all looking at the brightest areas
in the image.
And then we calculate the histogram, which all of this you've seen before using this HSV with the mask.
And then we normalize it, normalizing it.
Just make sure that from frame to frame, it's sort of consistent in the same range.
So we don't have any big jumps.
And then we set up the termination criteria for the algorithm.
So either 10 interactions or we move at least or move it by at least one point.
So what does that mean?
That's our termination criteria to stop tracking at that point.
So, you know, we are either we stop attending commissions or where the movement is, at least by one
point, that means we stop tracking at that point.
So then we run the loop.
We implement a mean shift right here.
DSD is our image here.
I should I should add that after the frame is converted to HSV, we use Kalkan Back Project, which
is the back projection for the histogram calculation.
So we used that here.
And then this is DSD.
That's the output of this is now fed into the mean shift tracking function along with a truck window,
along with the termination criteria we established before outside of the loop.
And then we draw a tracking window, have to draw a white window in this case, and we just output it
back to a video so we can see.
So run this.
It takes about three seconds front, pretty quick.
VIDEO It's pretty short.
It's about 15 seconds.
So you can tell this is a very efficient algorithm, given that it did it in a way faster than the actual
length of the video.
Now we can with using FFmpeg to EVA to the MPEG force if you are hit and take four, because that's
what we can display in collab with the HDR mode video player.
And when that's done, you can actually.
I've already done this before, so you can you can just look at my output here.
So this is the output here, so you can see something here, you can see this is the window we establish
here initially, and it's tracking correctly.
This is another random when do we establish here that just looks for the brightest areas in the image
and you'll see let's move on with my mouse over that.
The actual window initially tracks big trucks here.
So it is just sort of working.
Not ideal.
It can see how it works.
Now let's take a look at camp shift.
Camp shift is basically almost the same as mean shift.
However, it rotates, it returns a rotated rectangle as a result.
So it's a more effective way of tracking.
So let's take a look at that.
Consider algorithm is implemented in mostly the same way.
However, instead of a mean shift for you as camp shift there, but almost everything else remains exactly
the same.
We destroy differently because no, instead of drawing using the cvg rectangle, we have to get two
points and draw the line polygon lens for the rotated rectangle.
Since the rectangle, the key to that rectangle function only gives us rectangles in 90 zero 90 degrees
orientation.
So let's run this, which I've already done and I've already completed it to and before.
So let's look at the output of camshaft.
And you can see immediately.
We have directed the triangle rectangle here.
And you can see it's focusing on the bright spot here, which is not what we want.
We wanted to focus on a car.
So let's see if it ends up doing that when the video cuts to the next scene.
No, it does not.
But that is OK here.
I should add one confusing thing is that go back up to the beginning here in this truck or this, this
truck here is actually hardcoded into the video.
It's not actually the main shift of the camera for using the camera from interfering is actually this
box here.
That previous box was just to illustrate.
Other tracking methods prior to this.
So you can take a look.
This is sorry.
This is the actual mean shift box here.
This is another this is another means of box that's in hardcoded into the video that was using that
was using a different initial criteria.
So that's it for this lesson.
The next lesson takes a look at object tracking using the more advanced method of call optical flow.
So stay tuned for that.
Thank you.