Another type of layer that you can use is a convolution,
in a very similar way to what you did with images.
The code to use a convolutional on network is here.
It's very similar to what you had before.
You specify the number of convolutions that you want to learn, their size, and
their activation function.
The effect of this will then be the same.
Now words will be grouped into the size of the filter in this case 5.
And convolutions will learned that can map the word classification to
the desired output.
If we train with the convolutions now, we will see that our accuracy does
even better than before with close to about 100% on training and
around 80% on validation.
But as before, our loss increases in the validation set,
indicating potential overfilling.
As I have a super simple network here, it's not surprising, and
it will take some experimentation with different combinations
of conversational layers to improve on this.
If we go back to the model and explore the parameters,
we'll see that we have 128 filters each for 5 words.
And an exploration of the model will show these dimensions.
As the size of the input was 120 words, and a filter that is 5 words
long will shave off 2 words from the front and back, leaving us with 116.
The 128 filters that we specified will show up here as
part of the convolutional layer.