-: Hello, and welcome to the very last step
of this part one Building the AI.
Now the only thing that we have to do left
is to make this big forward function
that will propagate the signal from the very beginning
when the brain is getting the image to the very end
when the AI plays the action.
So we're gonna make this whole function
and that's gonna be our last step
before we move on to part two
Training our AI with Deep Convolution Q Learning.
So let's do this.
We are gonna take the function call
which actually is similar to the in it function.
That is, it's an existing function,
but this time we use it to call some other functions,
the ones that we made before.
Because you know we're gonna use
the forward function from the brain
and the forward function from the body.
And so we're using this cold function now
to basically call these functions.
So call is gonna take two arguments.
The first one itself, of course, the object.
And a second argument, which according to you
what is it going to be?
Well, we are doing the whole propagation this time.
So what we want to take as input
is of course the input images,
because of course that's the storing point,
when the AI is playing the game.
It is first visualizing the images of the game
then propagates the signals in the brain
and then plays the action.
Therefore, the second argument is going to be input.
And now we are ready to make this whole propagation.
So let's do this.
Okay, so the first step, where is it?
The first step is receiving the input images from the game.
And since these images are gonna enter the neural network
where you can imagine,
that we have to format them in a special structure
and the structure is of course a torch structure.
So the first thing that will happen is that
we will convert these images into a NumPy array.
Then we will convert the NumPy array into a torch denser.
And then finally, we will put the torch denser
inside a torch variable that will contain
both the tensor and a gradient.
That's for our dynamic graphs to compute very efficiently.
The gradient later is to catch a grade in this sense.
So that's our first step.
And then once we get the right format of our images,
well they will be able to enter the neural network.
And then that's where we'll do
the whole propagation of the signals.
So let's do this first step.
Converting the image into the right format.
So our images are so far input.
So now we're going to create a new variable
which I'm calling input.
So that's the real input of the neural network.
And this input, where is it going to be?
Well, first we need to take our input,
that is our original images.
Then as we said, we want to convert these images
into NumPy arrays.
So to do this, we can simply take NumPy,
which has a shortcut, NP, then the function array.
So we put input in the parenthesis of the function array.
There we go.
Now it is converted into some NumPy arrays.
But then since the cells of the NumPy arrays
will contain the pixels, it is actually safer
to specify the type float.
It's better to make sure we have some floats right now.
And to make sure of it, we can use NP dot float 32 here.
All right, so now we still have a NumPy arrays
but with the type float.
All right, and that's also for another reason.
It's that tensors are by definition arrays of a single type.
And so we choose this single type to be a float, float 32.
All right, so now that we have our NumPy arrays,
the next step is to convert that into a torch tensor.
And to do this, we can use for example,
torch dot and then the from on this core NumPy function
so that we'll convert that into a torch tensor.
There we go.
And now the last step is to put these torch tensors
into a torch variable containing
both the tenor and the gradient.
And you know how to do it.
Of course, we take our variable class
because actually everything that is inside this variable
is actually the input of the variable class.
But I wanted to show that to you this way
because you know we start with our input images,
then we convert them into NumPy arrays,
then to torch tensors, and then to variable.
And now we're good.
They are allowed to enter the neural network.
That is first the eyes of the AI,
and then the fully connected layers
to lead to the predictions.
So speaking of the eyes of the AI,
that's exactly what we're gonna do now.
We're gonna propagate these allowed images now,
into the eyes of the AI,
that is through the three convolution layers.
And to do this, you're gonna see now how it's so simple.
That's because we already have our brain
and our body from the in it function.
We simply need to take our brain,
so self dot brain and apply this brain to the input images.
And that will propagate thanks to the forward function
here from the brain that will propagate the signals
inside the brain.
And since the forward function of the brain
returns the output signals,
that is the neurons of the output layer
containing the Q values.
Well, this self brain input here
will return this output signal.
And therefore we're gonna put here
whether it returns into a variable,
and we're gonna call it very simply output.
And this output is the output signal of the brain.
And now, now that we have the output signal of the brain,
well, we have to propagate this output signal to the body.
And to do this, we're gonna use the second forward function
from the body.
And to do this, we simply need to take our body
and apply it to of course, the output,
because the forward function of the body
takes as input, the output signals of the brain.
So that's exactly what the output is right now
and returns the actions.
And therefore, since it returns the actions,
well here we're gonna add
actions equals self dot body output.
All right, so now you can see that very simply
we propagated the signals inside the brain
and then from the brain to the body,
first by using the forward function from the brain,
which takes as input, the input images,
and then propagate them into the brain
to return the Q values.
And then we propagate this output signal into the body
with the forward function of our body
to get the action to play.
And so now the only remaining thing that we have to do
and that's the very last line of code of this part one,
Building the AI.
Well, we have to return the action to play
and that is actions.
However, right now the actions have the torch format,
and we need to convert them back into NumPy arrays.
And to do this,
we're gonna take the data structure of these actions
and then add here the NumPy function.
And there we go.
Now we have the actions returned in the right format.
So congratulations.
We are now done with this first part one.
We built the AI in three steps.
First, we made the brain, second, we made the body.
And third, we assembled the brain and the body.
And we propagated the whole signal,
from the eyes to the moment we played the action.
So that's a first step done.
That was a huge step.
But now as you understood, we build an AI,
but it is still stupid.
We need to train it to be intelligent.
So we need to train it to do what we wanted to do.
And to do this we're gonna use the reward
after doom environment, you know,
because it's learning from the reward
by being reinforced when it gets a good reward
and by being punished or weakened
when it's getting a bad reward.
So that's where the Q learning will come into play.
And so that's exactly what we'll do in this part two
Training the AI with Deep Convolution Q Learning.
I can't wait to start.
And until then, enjoy AI.