-: Hello and welcome to this Python tutorial.
All right, so we have very exciting tutorials ahead of us.
We are gonna start
by creating the architecture of the neural network.
That is, we will make the neural network
that will be at the heart of our AI,
and that will return the action to play at each time team.
So let's do this.
So since we want our neural network to be an object
we're gonna make a class,
and that's because it's much more convenient.
A class is the model of something we want to build.
We want to build a neural network
and we need to make some kind of instructions
which will all be contained in a class.
And in this class, we're gonna make two functions.
First, the init function
which is the function that comes up all the time
when making a class,
and that basically defines the variable of your object,
that is the neural network,
the variables attached to the object
as opposed to the global variables.
And so this is in this in it function
that we'll define the architecture of the neural network.
Defining the input layer,
which will be composed of five input neurons
because we have five dimensions
for the encoded vector of input state.
Then we will define some hidden layers,
maybe we'll start with one hidden layer,
and then you will be welcome
to try some other architectures of the neural network.
And then of course, we will end up with the output layer
that will contain the possible actions
that we can play at each time.
So that's exactly what we'll do in this in it function.
And then we will make another function
still inside the class,
which will be the forward function,
and that will be the function that will activate
the neurons in the neural network,
that will activate the signals.
And so we will use a rectify activation function
because of course, we're dealing
with a purely non-linear problem,
and this rectified function breaks the linearity.
But mostly we are making this forward function
to return the Q-values
which are the outputs of the neural network.
But we have one Q-value for each action,
and later on we will be returning the final action
by either taking the max of the Q-values
or using a softmax method.
We will see that afterwards.
So in this tutorial we're gonna start
by implementing the init function,
and then the next one
we will be implementing the forward function.
So let's do this.
First we need to introduce our class.
So we start with class, then we give a name to our class.
Well, we can call it network.
And then in this network class,
I'm going to use an object programming technique
which is called inheritance.
And that is just to inherit
from all the tools of a parent class.
So our network class that we're about to make
is a child class of a larger class which is an nn.Module.
So that's just to inherit from all the tools
of this module class which are of course the tools
to implement a neural network.
So that's a very powerful and efficient trick
in object oriented programming.
That's called inheritance and right now we are inheriting
from this module parent class.
All right, and now we're ready to go inside the class.
So I'm pressing, enter, twice actually
because we'll be making two functions.
And we're starting with the init function.
So the init function, we have to name it this way
with two on course, then init,
and then again, two underscores,
that's just python syntax.
That is just how we have to do it.
And then we need to input the arguments.
So we have three arguments.
The first one is a compulsory argument
that is actually self.
And self, there is no mystery about it
that refers to the object that will be created
from this class that we're about to make.
We're making this class, it's like some instructions,
some model of this neural network we want to build.
And then once the class is ready
we can make as many neural networks as we want.
And each of these neural networks
will be some object of this class.
And since we'll be using the object for some other purposes
we need to spot what are the variables of the object.
And to spot this, we are using this self here to specify
that we're referring to the object.
So whenever I want to use a variable from my object
I will use self before the variable to specify
that this is variable of the object.
All right, so that's the first argument.
And then we have two other arguments,
which are of course the number of input neurons
and the number of output neurons.
So the number of input neurons,
we're gonna call it input size.
And that's actually five
because our input vectors have five dimensions,
the three signals, plus orientation, plus minus orientation
that's our vectors of encoded values
that describe one state of the environment.
These five values are enough to describe a state
of the environment.
We could have thought of less values or more values
but that's what I tried.
And it actually makes sense
because we actually need one signal from the left
one in front of us, and one at the right.
When we're driving a car.
We could have gone for 360 signal,
like the signals at the top of the Google cars
but we can totally self drive with three centers.
And then we have this orientation and minus orientation
to keep track of the goal that we're trying to reach.
And then we have, of course,
the output neurons of our neural network,
which correspond to the actions.
And we have three possible actions,
going left going straight, ordering right,
and therefore I'm gonna call it nb-action.
And there will be three of them.
All right.
But so far we only have to give names to the input
and then we'll use these variables
to do the computations inside the neural network.
All right.
Then I'm gonna start by using another pie torch trick.
This trick is the super function.
That's a function that actually inherits from the nn.Module.
So that's why we had to use inheritance
to inherit of the module tool.
This is the first tool we use.
And so basically we are only using this super trick,
this super function, to be able to use the tools of module.
So that's much more efficient.
And inside this super function,
I just need to specify first the network.
So that's our network channel class
because this is inheriting from the module parent class
and then our object.
And then I'm just adding dot
and our init function like this,
exactly how we named it.
All right.
So that's just a trick.
That's just to use all the tools of nn.Module.
Then we can move on to the next step,
which is to specify the input layer.
So basically what I have to do is introduce a new variable
that will be attached to the object
and this variable will contain the number of input neurons.
So not to be confused with input size.
Input size is the argument of the init function
but that's not the variable that is attached
to the object yet.
The variable that is attached to the object,
well, as I just mentioned,
we need to specify that it is instead attached
to the object.
So we use a self.,
and now we give a name to this first variable
attached to the object.
And so we can simply give the same name as the input.
We can call it input size,
and we will say it is equal to the argument
of the unit function that is input size.
All right.
So each time I'm creating an object from the network class
and I'm specifying the input size.
Like for example, I'm inputting five,
there will be a five here
and therefore the input size variable
of our object will have the value of five
because this input size here will be five,
and therefore our neural network
will have five input neurons in the input layer.
All right.
And then that's the same
for the other variable that we want to attach to our object.
And as you might have guessed,
this is going to be a variable for the number
of output neurons.
And so same, we take our object self,
and then we give a name
to this second variable of the object.
We are gonna call it nb-action.
And this will be equal to this argument here
giving the number of actions.
That is the number of output neurons.
And so we set it equal to nb-action.
So actually nb-action will be equal to three.
Therefore, the variable nb-action attached to our object,
to our network, will get the value of three.
Actually, we can see a warning here.
It says undefined name nn.
Well, that's because here we use the nn shortcut
and we need to use the shortcut here, nn
for our torch.nn.Module,
and then it'll disappear.
Here we go.
Perfect.
Right now we have have no warnings.
All the warnings here are just to specify
that what we import is not yet used, but that's okay.
We'll be using them afterwards.
All right.
Then we have another two variables we want to define
for object.
And this will be the full connections,
the full connections between the different layers
of our neural network.
So since right now we want to make a neural network composed
of only one hidden layer
where there will be two full connections.
There will be one first full connection
between the input layer and the hidden layer.
And one second, full connection
between the hidden layer and the output layer.
So let's start with the first full connection.
We're gonna call it fc1.
And again, I use self here to specify
that fc1 is the variable of my object.
So self.fc1, which will be equal to,
and now we use the nn.Module,
and we're gonna use a function called linear.
And that's exactly to make this full connection
between the neurons of the input layer
to the neurons of the hidden layer.
And what do I mean by full connection?
That means that all the neurons of the input layer
will all be connected to all the neurons
of the hidden layer.
And so to make this full connection,
we use this linear function
to which we need to input some arguments.
And as you can see, these arguments are in_features.
So that is the number of neurons
of the first layer we want to connect.
Then the out_features that is the number of neurons
of the second layer we want to connect.
That is the layer at the right, that is the hidden layer.
And bias equal true.
So bias equal true, we will keep the default value,
that's in order to have a bias
and not only some weights attached to the neurons.
We'll have the weights and one bias for each layer.
And so, well, let's see what we need to input.
So the first argument in_features is the number
of input neurons in the input layer.
And so what is it?
Well, that's actually input size.
That's the argument of our init function.
Which layer will be equal to five?
The three signals, orientation and minus orientation.
So here we go.
We input the first argument,
input size.
And then the second argument is out_features.
That is the number of neurons we want to have
in the second layer.
The second layer that will be fully connected
to the first layer.
And so now the question is
how many neurons do we want in this hidden layer?
Well, I did a lot of parameter tuning.
I did a lot of experimenting.
That's what we do in AI,
or that's what we do in deep learning in general.
We do a lot of experimenting to see
what would be the best neural network
for our specific problem.
And so I tried many values
and I ended up choosing 30, 30 neurons in a hidden layer.
And you will see that with this number,
we will get some pretty good results.
But then feel free to change the architecture
of the neural network.
Feel free to play with it.
You can not only change the number of hidden neurons
in the hidden layer, but also you can add some more layers
so that maybe you get an even better car.
But 30 hidden neurons will get us a good neural network
and a good car.
So that's what we go for.
And there we go.
We have our first full connection ready.
With this linear function, we make the full connection
between the input layer and the hidden layer.
And now time to make the second full connection,
that is the full connection
between the hidden layer and the output layer.
So there we go.
We're gonna call this second full connection fc2.
There we go.
And still, this is a variable from our object.
So I'm using self here.
And then again, we use,
well, actually we can copy this
because we're gonna use the nn.Module
and then the linear function.
But then we need to change the arguments of course.
First, that's the same.
First is the number of neurons we want to have
in the first layer of the full connection.
So that's the hidden layer, and therefore that is 30.
And then second argument is the number
of neurons in the second layer of the full connection.
And that corresponds to the output layer.
And the output layer has nb-action neurons,
which later will be three
because we have three possible actions.
But so far we have to use the names we defined.
That is the names of the argument of the init function
and therefore we input here nb-action.
And there we go.
First of all, our two full connections is ready.
And second of all, our init function is ready.
So that's what will initialize our object
whenever we create an object from the network class.
And so as soon as we create an object,
well, all these variables, the four variables here,
input size, nb-action, fc1 and fc2 will be defined.
And that's how we'll get the architecture
of our neural network for each object that we create.
Each object will correspond to neural network
of five input neurons, 30 hidden neurons,
and three output neurons.
So there we go.
We are done with this first init function,
and now we can move on to the second function,
which is the forward function.
And that will be used to activate the neurons
in the neural network using rectifier activation function
and mostly to eventually return the Q-values
which are the outputs of our neural network.
So I can't wait to do this in the next tutorial.
And until then, enjoy AI.