In this section of the course, we are going to introduce the idea of artificial neural networks or
angels for short.
Specifically, we're going to talk about a certain kind of artificial neural network called a feedforward
neural network.
This is the most basic kind of neural network.
But as you'll see, the concepts involved go quite deep, and they also form the basis for other kinds
of neural networks, such as convolutional neural networks and recurrent neural networks.
To begin this discussion, let's start with why and how neural networks came to be in the first place.
This is something that a lot of teachers skip over, but it's something that I find quite interesting
and inspirational.
If you think neural networks are a cool new model to help you pick stocks or play Mario Kart, then
you are still looking for the small fish.
In fact, neural networks are way more interesting than that.
As you may have realized that the name artificial neural network means that we are trying to artificially
create our neural network in a computer.
OK, so then what's a real neural network as in a non artificial neural network?
The name neural network derived from neurons, which are the cells in your brain and extend throughout
your nervous system.
Now, this is probably too obvious for most of you, but let's state it anyway.
Just in case your brain is what you used to think, neurons in your brain are connected to each other
and can communicate with each other via electrical and chemical signals.
Amazingly, this simple physical and chemical system is what makes you you all of your thoughts and
aspirations, your emotions and every action you take throughout the day.
It's all driven by your neurons, which in turn are just sending electrical and chemical signals around
amongst themselves.
Once scientists realized what the brain was doing and what it was responsible for, the next question
was almost obvious in hindsight, can we build a brain?
I mean, if the brain is just a network of neurons and we can simulate neurons in a computer if we connect
a bunch of neurons through a computer simulation, it seems that it might be possible to create some
form of intelligence and artificial intelligence, you might say.
So let's take our model of a single neuron logistic regression.
Now let's imagine that we have multiple neurons all taking in the same inputs, but computing something
different.
Now let's do it again.
Now we have multiple logistic regression.
Now let's imagine that all these neurons are connected to more neurons.
So we just repeat the process, pretending that the new layer of neurons are actually inputs to motor
neurons.
That's basically a very tiny model of the brain neurons connected to neurons.
Of course, this is necessarily simplistic.
One side is the input and one side is the output.
Of course, the actual brain is much more complex.
There are many inputs and many outputs in the middle.
Wires can crisscross if we have a later neuron connecting back to an earlier neuron.
We call that a recurrent connection.
The neural networks we are about to discuss in this section contain no such complexities.
Instead, because the input is on one side and the output is on the other side and we go from input
to output in a layaways fashion, we call this a feedforward neural network.
In the rest of this lecture, we are going to outline what we will discuss in this section of the course.
First, we are going to start out again by discussing the model architecture.
As you know, the model will be discussing in this section is the feedforward neural network.
The next step, after discussing the model architecture, will be to go back to the geometric picture.
If you recall, my model goes.
Machine learning is nothing but a geometry problem.
So how do neural networks extend the capabilities of a basic linear model in terms of solving this geometry
problem?
Next, we're going to go more in-depth and discuss activation functions.
Activation functions are very important in neural networks.
They are what make neural networks more expressive than the simple, linear models you saw in the previous
section.
After that, we're going to discuss how to do multiclass classification using neural networks.
If you recall in the previous section, we only discussed the binary classification.
This works if we only have two classes dog or cat fraud, or no fraud, purchase or leave the store,
and so on.
But what if we have more classes?
For example, we might be working on a self-driving car that needs to be able to recognize multiple
kinds of objects on the road.
In this case, a binary classification is not good enough, and we need multiclass classification.
Now you'll notice that most of what we just discussed was our theory.
This is basically optional if you don't care about how neural networks actually work.
After this, we'll move back to the practical world and look at how to do multiclass classification
in TensorFlow.
You'll see that it's pretty similar to the previous section, with just a few small changes in syntax.
But the next part of this section is critical.
As you recall, I said that at some point we would stop using TFR, TAF and start using embeddings,
which are more common in the field of deep learning.
We'll look at what an embedding actually is and how we can use them in TensorFlow.
Unfortunately, we won't really get to use embeddings since they only come into play with CNN's and
ends which come later in the course.
However, we will take this opportunity to do an advanced exercise.
In particular, you'll recall that Words Avec is a technique which has been used to retrain word embeddings.
As you recall, words effect is basically a neural network, which you now understand having been through
this section of the course.
Therefore, you have everything you need in order to implement words of EC.
Now, that doesn't mean it will be easy since, as you'll see, there are some complications that arise,
but it doesn't require new knowledge.
Just more advanced application of existing knowledge.
As such, this exercise will be designated as advanced, and it will be a nice way to cap off what you've
learned about intense.