Welcome back. In this week, you'll learn how to
use the Keras layer in TensorFlow to implement
word embeddings, which is one of
the most important ideas in Natural Language Processing. If you have a vocabulary size
of say 10,000 words, rather than using the numbers from 1-10,000 to
represent these words, can you have a better way
to represent those numbers? So this week you'll learn about word embeddings and also get to train your own neural network
for text classification. It's really interesting
that I really liked how embeddings work for representing
the semantics of a word. So now instead of the word
just being a number, it's like a vector in
n-dimensional space. We're using 16 and 32 dimensions
in different examples. So for example the word dog, might be a vector pointing in a particular direction
and then the word K9, could be learned as
a vector pointing in a very similar direction, and we know they
have very similar semantic meaning off of that. All of this is actually
done for us in embeddings. So as we train like our datasets, we'll train for example with
IMDB which is movie reviews, and there's a set
of positive reviews and a set of negative reviews, and what will actually happen is the embeddings for
the words in these reviews, and we can actually
plot them on a chart. So we'll see something
that looks like a globe and one of the poles on the globe or
all the words that are clustered around for
the positive review, and all the words
on the other pole of the globe are clustered
around the negative review, and it really helps
us to start to see the semantics behind these words. I think one of the coolest things about word embedding is you can download the pretrained word embedding that maybe
someone else has trained. This gives your
learning algorithm a hint into the meaning
of these words. So when you see
another vocabulary word, maybe you see K9
for the first time, it's as if you can let
the algorithm know that K9 means
something a little bit like dog related to
dog even if you've never seen the word K9 in
your specific training. That's amazing. Today in the world
of applied NLP, word embeddings has
proved to be one of the most powerful
and useful ideas to help teams get good performance. So in this week, you'll learn how to use word
embeddings in TensorFlow. Let's go on to the next video.