If you could recall from the previous sections, the way the dense layer is built is such that if we
have this as our dense layer and that we have this input right here, let's call this I, let's say
it's X, we have this input x, then we have a certain M times x.
Let's see.
So this is actually the weights.
So the weights times X.
Plus the bias.
B Let's call this B and then this is now equal our output.
So here we have an output of y, y equals max plus C.
So this simply means if we want to recreate the dense layer, then we have to take this into consideration
all the definition of the layer from scratch into consideration.
That said, we could define neural learn dance.
So it's like our custom dance mural and dance is going to.
Inherit from Lear and then right here, this class.
So we have this class inherits from layer.
Then from here we have our init method in it, and then we have the super we pass in neural learn dance.
There we go.
We have that that neat.
So that's it.
Now, from here, if you notice, whenever we create an advanced layer, like let's call up here, whenever
we create an advanced layer, we generally had to specify, at least because this is by default, this
is not by default, but so here we need to pass in this year to specify the number of output units.
Now, that set, we have to take that into consideration when building our neural and dense layer.
So here we have output units.
There we go, we have our output units.
And then in here we'll define self output units to be equal output units.
So that's it.
Now, from this point, we are going to build this layer.
In order to build this layer, we have to take into consideration this definition right here.
But this definition put out a where this isn't very clear.
Now, let's make this or let's break this up.
So you suppose now we have this input of shape batch size by, let's say, number of features.
So suppose we have this input.
Now what happens here is this input is going to be multiplied by this weight, so it's going to multiply
by this weight, which happens to be a matrix.
Now we take this and multiply by that matrix, and for the multiplication to be valid, we need to ensure
that the number of columns we have here matches with the number of rows of this matrix.
But then what are the dimensions of this matrix?
We have to note that this matrix has to be defined such that we have a shape of F.
This f here must match.
By the number of output units.
So if you want a number of Apple units, for example to be one, then you should have one.
And that's why when defining the dense layer, we don't need to specify this as this.
Value is gotten automatically from the number of columns in the inputs.
Since if we don't take this, we are going to have an error.
So TensorFlow takes this automatically fits in here and then collects the input you pass in the dense.
So when you specify, when you say you have this dense like this, and then you say, for example one
and maybe some activation, then you have that.
So let's say we have some activation here.
Now, once this one gets here, this waits matrix is now defined.
So that input you pattern to this dance is going to affect the number of rows we have.
But then what you pass in as argument here is going to tell us or give us a number of columns we're
going to have for this weight matrix right here.
So that set we have F by one.
And then when we multiply this, we're going to have B by one.
So we see we now understand how we get this output and then we have plus B by one.
So that's it.
Where this one come from, the bias.
Now once we add this up, we have an output of B by one, and that's our Y, that's the shape of our
Y.
If that's understood, we will move on to building.
So here we have our build method, we have self and for now let's keep it that way.
So there we go.
We have this build method and then we're going to define our weights.
So we have self got weights, let's specify weights equal that.
And then here we're going to have the self dot add weights method.
So this actually comes with this layer class right here.
So we're able to call this because we are inheriting from the layer class.
So here we have self add weight and now we specify the shape.
So guess what?
We are going to have our number of rows, so in rows, which is going to come from the input and then
we are going to have a number of columns which is going to come from this output units you specified
which you pass in one calendar, nutrient dense layer.
So here we're going to have self dot output units, so we get a number of output units.
Now how do we obtain this number of input units?
We're going to look at that shortly.
For now we have the weights and then we have that.
And then let's go to bias.
So we have self biases and self.
This is also a weight and weights.
And then we specify does a number of output units since it's one dimensional.
So we have output units that we go.
And so at this point we've defined our weights and our bias metrics.
This actually weight just similar.
So that's it.
Now let's go into call.
So we have our call method.
We should actually get a job done and then we have our input.
So let's, let's call this input, let's add an S or let's say input data or input features.
So there we go.
So we have that.
And then what we're going to do here is we're going to return simply the matrix multiplication as we've
seen already of the weights.
So we have self that weights and this input features we call is actually the input features times the
weights.
Let's get back to this here.
If we take the input, we are the input times the weights, because if we have the weights times the
input, we will have F one times, b, f we call our weights does our weight shape and then these are
input shape our pen.
This you see that this is always the same.
So it's not going to be we're going to try an error.
And so that said, what we're going to do is we're going to just simply have input features, right
your.
So that's it.
And then we add up the bias.
So yeah, we have self dot biases, which is from this one right here.
Now the where we're going to get this number of rows is going to be easy here.
All we need to do is to specify here that we have the input feature features sheep.
We have our input features shape, which is going to come automatically from this and then to get the
number of rows, what we need to do here is have input features.
Shape, and then we get that last dimension.
So let's get back to this.
We call here we have B by F, that's B, F, And so, yeah, we need the weights needs to be F.
By the output.
And so to get this f we just need to take the input and then get this last element right here and get
is the number.
Of columns we have here.
So that's that.
We will just have that input features specify this, take this index and that would be good.
So that's how we get we obtain this value automatically the number of rows we've been looking for.
So now we've gotten this.
Everything seems fine.
Next thing to do is specify that it's trainable.
So we have to specify that these weights are trainable because in some cases we want that which shouldn't
be trainable.
So self the trainable equal.
True.
Now this most of you, this is an argument like it's one of the arguments which I've been passing to
this add weight method right here.
So we just have that.
And then here again, we have this trainable, equal, true.
Now, apart from that, we could randomly initialize our weights and biases.
So here we have random normal, random normal initialization.
That's fine.
And then we do same year.
So we have our initialize
equals random normal.
That's good.
We now run.
This is our neural learning dense layer.
And then after running this, let's get to integrated.
Let's make it quite simple.
We just use our sequential API.
So let's get back to the sequential API we had built initially.
Copy that.
And then we are going to integrate this new dense layer, this neural learn custom dense layer.
So there you go.
We have that.
And then instead of dense layer, yeah, we have neural learned, custom dense layer.
So you see that you can be able to create your own layers with TensorFlow.
So that's it.
Neural learning.
So neural and dense neural and dense.
Now, you said this is an error because we don't take into consideration activation.
And so what we could do is we could get back right here and see.
If the activation is equal value.
So if activation is equal value, we'll return this.
And then else, let's say leaf activation is equal sigmoid.
We return that.
Oops, let's get back.
We have this.
So we're going to return this, this, copy this, and then paste out right here.
And then we have else the same.
So now let's get into this and see the modifications we're going to make.
There we go.
So here we have our blue in the case of red blue.
What we wanted to have here is ttf dot and in lieu of that.
So we're going to pass this in to our blue.
And then if the activation is sigmoid, we should have to have the mad dot sigmoid.
So that's it.
There we go.
We have that.
And then this one we just maintain.
So we have modified our code so that we now integrate the activations.
There we go.
That sounds fine.
So let's run this and then.
Um, take this error.
So here we have this double equals python syntax.
And then we come down here, everything seems fine.
Um, let's take this again.
Yeah, We're supposed to have.
We're supposed to specify activation.
So you're supposed to have activation, and then self activation.
You call activation.
And then here we have self.
That activation.
Self.
That activation, we run that again looks fine.
And then now what we do is we pass.
We just simply run this.
So we run that and.
We get into error right here to solve this problem.
What we're gonna do is we're going to include the shape here.
So we're going to have shape equal that and then shape equal this as we run that.
And we should have no error again.
Oh yeah, we haven't cancelled the attributes weights likely because it conflicts with an existing read
only property of the object.
So.
Right.
Yeah.
Instead of using weights, we're just going to say w so we have the W and then.
Right.
Yeah.
We should have W now since we don't want to repeat this over or what would you say we have pre output.
So we have our pre output which is this here and pays that out and then yeah we have W right.
Yeah.
We have B does it and now we have if this then we pass the pre output and we do the same right here.
So we have just a pre output before the activation.
Now here we just have pre output actually.
So we just have pre output.
Take that off now running.
This should be fine.
So let's run that and see what we get.
We run this, we run that and there we go, We have our model, the same exact model we have been building
right from the start, but this time around we haven't.
Are we using a custom dense layer, which is our neural learning dance layer?
Now let's go ahead and compile this model and then train it.
Yeah, we've actually maintained this year, so we have to change this name.
Here is the net model.
Let's see the net custom.
Yeah, Custom.
Uh, custom model.
Yeah.
Okay.
We copied that.
We have all the net custom model.
Let's have it here.
There we go.
We run it.
That's fine, right?
Yeah.
We have the net custom.
We run that.
Yeah, we have the net custom.
We run that, and then let's check out.
As you could see with the training, there's a slight difference in the last values and accuracy we
get getting for this first epoch.
And most probably this is coming from this random normal initialization we've chosen here.
As with the standard dense layer, which you could see here is kernel initialization or the weight initialization
is using this LORO uniform and the bias initialization is the zeros.
So here we have all zeros for the biases and then we use the uniform method.
So here, as you could see, this isn't performing as well as what we had previously, but scrolling
up, there is this error.
So yeah, it was meant to be sigmoid.
So let's stop this, let's interrupt this training and then get back and run this.
So we will run this and compile and then start with the training again.
This time around it looks better.
It looks more like what we should expect.
So even though we're not using the initialization method, this training process looks quite similar
to what we would have had in the case of the initialization method, which is used with a standard dense
layer.
And here is what we get after training for over five epochs.
Thank you for getting up to this point and see you next time.