Hi and welcome back to the course in this section, we'll take a look at a very useful, deep learning
based truck called Deep Thought.
So let's get started.
So before we begin, do you remember what a truck is?
Well, don't worry, I'll refresh your memory shortly.
Now, imagine you might think a truck is just something that trucks and moving object in a scene.
And while that is correct.
And that's exactly what means shift an optical fluid did.
There's a lot more there's a lot more complicated cases where you'd want to use a much more advanced
truck.
Now let's take a look at one of those cases.
So imagine this scene here.
OK, forget about a text on top of explain it based on images right now.
So we want to truck multiple vehicles in this frame so you can easily see.
Firstly, we can have bounding boxes on these vehicles.
And isn't that trucking them?
Well, it's not technically, because what if we wanted to put a bounding box over each vehicle and
have that linked to a certain I.D. number like, say, this is car number one second of the two and
so on?
And as you can see, this is car.
The look of this car had the first big car on the left image.
Let's say that's car number one and you can see four seconds after it has gone up here.
All right.
Now what's going to happen in your little detector or most of the other detectors is that the number
of calls on the screen might change constantly from frame to frame.
You can see sometimes you might have four or five cars detected here.
Sometimes we have three cars detected.
So what that means is that whatever number the you or if you even assign a number of commanding box
one to a vehicle, there's no guarantee the bounding box one here is going to be the the box one here
on the next frame.
So essentially, I'm trying to pose the problem presented here.
The problem is when we go from frame to frame because of bounding boxes tend to be unstable, and that's
mentally unstable because of a moving scene.
I should say then you're going to have bounding boxes not aligned to the number of the bounding box,
not aligned to the vehicle or the object that it's being tracked.
Because in this case, this one could be number one.
Over here, it could be number three.
And there's no way to actually track them individually.
So if you wanted to assign, let's say, this vehicle being kind of one, you want to do all this because
number one, while it's visible in the frame, so that's essentially what tracking is.
So I'll do a quick recap on some of the previous trackers you would have seen in the open TV lesson.
So the first one is mean shift and means is quite useful, although it's quite simple to do when you're
tracking a single moving object in a scene like this.
Imagine we're tracking this tank moving along this makeshift road.
It uses a histograms, basically histograms of the color intensity or whatever metric you want to use
from the color space.
And it tries to find a likelihood of the object location over the histogram.
So it's sort of like a mode seeking type algorithm that basically senses stumbling blocks upon the object
of its being tracked.
And it works well when it's like when it's with one object.
However, it's a lot more complicated cases where you have two objects and means shifting is not going
to work.
So what about optical flow?
Well, optical flow is probably much better than mine chips in some in some cases.
However, again, it doesn't solve the problem of keeping a consistent I.D. on certain vehicle.
It's really good for tracking movement and vehicles do it because what it does and you looks at the
basically the relative motion or movement between the object and the viewer, that's you got a camera.
So now we can move on to something that is actually quite was quite groundbreaking when it was, I guess,
published.
Basically, Kalman filtering is what engineers like myself in the previous day, 10 years ago, I was
an engineer, electrical, electrical and computer engineer, and I did some robotics as well.
And Kalman filters were amazing at and cleaning up.
Since sensor data like you have a distance sensor or some sort of like light sensor, a common filter
would be ideal just to clean it up.
And how it works is that basically it takes prior knowledge of the state of the reading of that sensor.
So you can see initially it's going to be a little bit off.
The air is going to be quite big because there are no historical measurements yet, but as you can see
as we get more.
Historical measurements, the estimate using the common filter becomes narrow and narrow to the actual
values these X's here are different.
Measured temperatures in this case over time, and you can see pretty much all over the place, they're
unstable.
But using the common filtering algorithm, we get a much stable reading this way.
So now that we've discussed some of the older tracking algorithms, let's take a look at some of the
challenges faced with this.
So a lot of these are quite computationally expensive now we can get real time performance from them.
But again, they use a lot of resources, and it's probably not the most efficient way given their performance.
So they are also very susceptible to noise and fast camera movements as well as conclusions.
So generally, it's you can see here like these two people tracking here.
This is one and this is two you can see we keep them in frame here and it maintains it's correct as
the guy, as the guys walk past and believe it maintains the 80s correctly.
But you can see in a tracking scenario, this can actually cause a number of problems to those previous
trackers.
So now this brings us to deep sort.
So deep thought is a very widely used tracker right now, and it actually has been recently integrated
into India's deep stream.
Deep Stream is basically a production deployment tool for computer vision models and applications and
deep, so it was recently brought in to it.
So now we have a vision of better tracking on that in video deep stream platform.
And basically, it performs multi object tracking, as you can see in this image here.
You can see each person has a unique ID and as they move about to frame, even though this isn't a video,
but you can imagine people moving here.
They will keep the same ID consistently across the frame until the exit or leave the frame.
So just to give a quick overview of how common all of this sort of works, it basically performs common
filtering in the image space and frame by frame data association using the Hungarian method with an
association metric that measures bounding box overlap.
So what that scene, because I just read this how it would fit would here, but it's a pretty good summary
of what it's doing.
Yeah.
So what's happening is that it's looking at each frame.
So we have a frame by frame association.
So imagine we have the bomb in boxes from one frame and then the bounding boxes for the next room.
We have to find some sort of association between them.
And that's what the common filtering into image space is doing.
And next to Hungarian method is also doing a similar metric by comparing the bungling blocks overlap
with the previous frame.
So that's how the deep sort of algorithm works.
And if you want to take a look at it in deeper, you can read the paper here.
I've also compiled some notes on these slides here, so they're quite lengthy, so I'm not going to
read them out there, but difficult to explain by just reading them.
But if you want, this is basically a summary of the people above the different metrics about association
assignment algorithm talks about a lot of these things here, and it basically tells you how deep.
So it actually solves a lot of these problems that we've found, and we have a final summary of all
the deep sort of architecture and some of the more intimate, informative things about that feature
descriptor in how it uses that to match the objects.
So that's it for this deep sort lesson.
In the next section, we'll take a look at the Google Club notebook that when we actually use deep sort
with yellow.
So I'll see you in the next lesson.
Thank you.