1
00:00:00,140 --> 00:00:03,600
You can compile
your model as before,

2
00:00:03,600 --> 00:00:06,060
and print out the summary
with this code.

3
00:00:06,060 --> 00:00:08,640
Now training is
the simplest passing

4
00:00:08,640 --> 00:00:10,350
padded and your training labels

5
00:00:10,350 --> 00:00:12,110
final as your training set,

6
00:00:12,110 --> 00:00:13,930
specifying the number of epochs,

7
00:00:13,930 --> 00:00:15,300
and passing the testing

8
00:00:15,300 --> 00:00:19,720
padded and testing labels
final as your test set.

9
00:00:19,720 --> 00:00:21,860
Here's the results of training,

10
00:00:21,860 --> 00:00:24,710
with the training
set giving us 1.00

11
00:00:24,710 --> 00:00:28,865
accuracy and the validation
set at 0.8259.

12
00:00:28,865 --> 00:00:31,295
So there's a good chance
that we're overfitting.

13
00:00:31,295 --> 00:00:33,980
We'll look at some strategies
to avoid this later,

14
00:00:33,980 --> 00:00:37,160
but you should expect results
a little bit like this.

15
00:00:37,160 --> 00:00:39,410
Okay. Now we need to talk

16
00:00:39,410 --> 00:00:41,660
about and demonstrate
the embeddings,

17
00:00:41,660 --> 00:00:43,610
so you can visualize
them like you

18
00:00:43,610 --> 00:00:46,205
did right back at
the beginning of this lesson.

19
00:00:46,205 --> 00:00:48,290
We'll start by
getting the results

20
00:00:48,290 --> 00:00:49,475
of the embeddings layer,

21
00:00:49,475 --> 00:00:50,975
which is layer zero.

22
00:00:50,975 --> 00:00:52,490
We can get the weights,

23
00:00:52,490 --> 00:00:54,740
and print out
their shape like this.

24
00:00:54,740 --> 00:00:58,375
We can see that this is
a 10,000 by 16 array,

25
00:00:58,375 --> 00:01:00,750
we have 10,000 words
in our corpus,

26
00:01:00,750 --> 00:01:03,075
and we're working in
a 16 dimensional array,

27
00:01:03,075 --> 00:01:05,265
so our embedding will
have that shape.

28
00:01:05,265 --> 00:01:07,015
To be able to plot it,

29
00:01:07,015 --> 00:01:10,415
we need a helper function
to reverse our word index.

30
00:01:10,415 --> 00:01:11,885
As it currently stands,

31
00:01:11,885 --> 00:01:14,315
our word index has
the key being the word,

32
00:01:14,315 --> 00:01:16,490
and the value being the
token for the word.

33
00:01:16,490 --> 00:01:18,440
We'll need to flip this around,

34
00:01:18,440 --> 00:01:20,330
to look through
the padded list to

35
00:01:20,330 --> 00:01:22,580
decode the tokens
back into the words,

36
00:01:22,580 --> 00:01:25,160
so we've written
this helper function.

37
00:01:25,160 --> 00:01:27,890
Now it's time to
write the vectors and

38
00:01:27,890 --> 00:01:29,885
their metadata auto files.

39
00:01:29,885 --> 00:01:32,930
The TensorFlow Projector
reads this file type and

40
00:01:32,930 --> 00:01:34,490
uses it to plot the vectors in

41
00:01:34,490 --> 00:01:37,150
3D space so we can
visualize them.

42
00:01:37,150 --> 00:01:38,880
To the vectors file,

43
00:01:38,880 --> 00:01:40,760
we simply write out
the value of each

44
00:01:40,760 --> 00:01:42,950
of the items in
the array of embeddings,

45
00:01:42,950 --> 00:01:44,765
i.e, the co-efficient of

46
00:01:44,765 --> 00:01:47,690
each dimension on the
vector for this word.

47
00:01:47,690 --> 00:01:49,475
To the metadata array,

48
00:01:49,475 --> 00:01:51,245
we just write out the words.

49
00:01:51,245 --> 00:01:53,089
If you're working in Colab,

50
00:01:53,089 --> 00:01:55,655
this code will download
the two files.

51
00:01:55,655 --> 00:01:57,590
To now render the results,

52
00:01:57,590 --> 00:01:59,885
go to the TensorFlow
Embedding Projector

53
00:01:59,885 --> 00:02:02,540
on projector.tensorflow.org,

54
00:02:02,540 --> 00:02:05,615
press the ''Load data''
button on the left.

55
00:02:05,615 --> 00:02:07,730
You'll see a dialog asking

56
00:02:07,730 --> 00:02:09,635
you to load data
from your computer.

57
00:02:09,635 --> 00:02:12,035
Use vector.TSV for the first one,

58
00:02:12,035 --> 00:02:14,615
and meta.TSV for the second.

59
00:02:14,615 --> 00:02:18,140
Once they're loaded, you should
see something like this.

60
00:02:18,140 --> 00:02:21,825
Click this ''sphereize data''
checkbox on the top left,

61
00:02:21,825 --> 00:02:24,635
and you'll see the binary
clustering of the data.

62
00:02:24,635 --> 00:02:26,885
Experiment by
searching for words,

63
00:02:26,885 --> 00:02:28,220
or clicking on the blue dots in

64
00:02:28,220 --> 00:02:29,945
the chart that represent words.

65
00:02:29,945 --> 00:02:32,215
Above all, have some fun with it.

66
00:02:32,215 --> 00:02:34,220
Next up, we'll step through

67
00:02:34,220 --> 00:02:36,200
a screencast of what
you've just seen,

68
00:02:36,200 --> 00:02:38,195
so you can explore it in action.

69
00:02:38,195 --> 00:02:41,300
After that, you'll look
at how TFTS has built in

70
00:02:41,300 --> 00:02:43,490
tokenizers that prevent you from

71
00:02:43,490 --> 00:02:45,080
writing a lot of the tokenizing

72
00:02:45,080 --> 00:02:47,310
code that we've just used.