1
00:00:10,900 --> 00:00:14,260
Everyone and welcome to natural language processing in Python.

2
00:00:14,440 --> 00:00:20,440
This is the best and most in-depth online resource for learning about how to use Python to analyze,

3
00:00:20,440 --> 00:00:22,480
understand and generate language.

4
00:00:23,380 --> 00:00:28,510
So in this video, I'm going to describe what this course is about, why you should take it and what

5
00:00:28,510 --> 00:00:30,070
skills you will learn if you do.

6
00:00:31,270 --> 00:00:33,490
So let's start by asking a basic question.

7
00:00:33,550 --> 00:00:35,350
What is natural language processing?

8
00:00:36,010 --> 00:00:41,050
Natural language processing is a sub field of data science and machine learning that specifically deals

9
00:00:41,050 --> 00:00:42,340
with text and speech.

10
00:00:42,820 --> 00:00:47,410
It's one of the two areas of machine learning that deal with naturally occurring data sources.

11
00:00:47,680 --> 00:00:48,940
The other being images.

12
00:00:49,450 --> 00:00:54,070
But some feel that deals with images is called computer vision and by naturally occurring.

13
00:00:54,070 --> 00:00:57,310
What I mean is that people just put a lot of this stuff on the internet.

14
00:00:57,640 --> 00:01:01,870
It's not stuff you need to acquire or measure by doing a survey or an experiment.

15
00:01:02,530 --> 00:01:07,690
So these two areas are very important because they allow us to analyze and process large quantities

16
00:01:07,690 --> 00:01:10,180
of data that just happened to exist in abundance.

17
00:01:11,260 --> 00:01:15,850
The best way to understand why this field is important is to check out some applications.

18
00:01:17,690 --> 00:01:20,510
The first application now he's mentioned is spam detection.

19
00:01:20,960 --> 00:01:24,890
You probably use this every day without knowing it's happening behind the scenes.

20
00:01:26,450 --> 00:01:32,630
Another everyday use of NLP is with predictive text when you send an email or type out a text message.

21
00:01:32,900 --> 00:01:38,270
Machine learning is used to make suggestions while you type, making it easier for you to type your

22
00:01:38,270 --> 00:01:38,930
message.

23
00:01:40,600 --> 00:01:47,530
Speech recognition is another great example of how NLP has impacted our daily lives with Siri, Alexa

24
00:01:47,530 --> 00:01:48,520
and Google Assistant.

25
00:01:48,760 --> 00:01:54,070
We can simply speak commands or ask questions to our devices, and our devices will listen and take

26
00:01:54,070 --> 00:01:56,050
action based on what we've said.

27
00:01:57,720 --> 00:02:01,620
One application, which is a tiny bit abstract, is sentiment analysis.

28
00:02:02,010 --> 00:02:07,230
This is where we can use machine learning to decide whether a passage of text is positive or negative.

29
00:02:07,710 --> 00:02:12,690
This can be used by companies to learn about their reputation and buy stock traders to predict stock

30
00:02:12,690 --> 00:02:13,620
price movements.

31
00:02:15,270 --> 00:02:18,660
Now, you might wonder, why don't people just read the text themselves?

32
00:02:19,020 --> 00:02:22,590
And this is a good opportunity to consider the concept of automation.

33
00:02:23,010 --> 00:02:27,090
There is so much text in the world that you simply do not have time to read everything.

34
00:02:27,840 --> 00:02:32,490
Furthermore, if you're a stock trader, keep in mind that you are competing with other stock traders,

35
00:02:32,760 --> 00:02:35,370
and those who act more quickly will benefit the most.

36
00:02:35,790 --> 00:02:40,800
And so if you are not automating what your competition is, then you risk being not fast enough.

37
00:02:42,520 --> 00:02:46,040
Now, those are some pretty well known classic applications of NLP.

38
00:02:46,480 --> 00:02:50,890
But there have been some amazing advancements in the recent past that are worth making note of.

39
00:02:51,610 --> 00:02:57,280
One example is language translation, which is becoming an increasingly important tool thanks to technology

40
00:02:57,280 --> 00:03:02,620
which connects us with people around the world thanks to automatic transcription and translation.

41
00:03:02,890 --> 00:03:08,020
People all around the world can understand my courses, even if we don't speak the same language.

42
00:03:09,100 --> 00:03:15,100
Language translation also happens to be a very useful tool for us as it motivates the concept of attention,

43
00:03:15,370 --> 00:03:18,340
which leads us to a state of the art model called the Transformer.

44
00:03:20,340 --> 00:03:23,550
Another very futuristic technique is text generation.

45
00:03:24,120 --> 00:03:29,550
This is related to predictive text, as we previously discussed, but the capabilities of today's models

46
00:03:29,550 --> 00:03:32,430
are much greater and allow us to take things much further.

47
00:03:33,580 --> 00:03:39,010
The current state of the art for this task is GPT three, which can generate very long documents, which

48
00:03:39,010 --> 00:03:42,070
are almost totally coherent and grammatically correct.

49
00:03:42,550 --> 00:03:47,860
All you have to do is give it a prompt telling it what you want to write about, and GPT three can write

50
00:03:47,860 --> 00:03:52,330
pages of text that are very difficult to distinguish from what a human would write.

51
00:03:53,110 --> 00:03:58,780
In fact, it's been used to write online comments on social media sites where I'm sure many have already

52
00:03:58,780 --> 00:04:02,230
been fooled into thinking they were reading something written by a human.

53
00:04:02,920 --> 00:04:08,440
Now, that might seem like just a mere curiosity, but in fact, actual money making businesses have

54
00:04:08,440 --> 00:04:11,200
already been built on top of this technology.

55
00:04:12,160 --> 00:04:17,740
One of the most popular examples is GitHub Co-Pilot, which is currently in preview mode but has already

56
00:04:17,740 --> 00:04:19,630
shown great promise in writing code.

57
00:04:20,200 --> 00:04:25,780
So it's not just everyday language that NLP models can understand, but programming languages as well.

58
00:04:26,350 --> 00:04:31,900
In fact, these technologies have been applied in other areas such as genomics, where Alpha Fold2 has

59
00:04:31,900 --> 00:04:35,380
been said to solve the greatest problem in molecular biology.

60
00:04:37,500 --> 00:04:43,350
So now that you're convinced that NLP is an amazingly powerful technology with many real world use cases.

61
00:04:43,740 --> 00:04:45,300
Let's talk about this cause.

62
00:04:46,110 --> 00:04:49,710
This course will take you from beginner to advanced in the field of NLP.

63
00:04:50,160 --> 00:04:52,740
In fact, it's really more like four courses in one.

64
00:04:53,160 --> 00:04:59,580
Let me explain the first major part of this course, which is like a little mini course, focuses on

65
00:04:59,580 --> 00:05:01,710
how to process text on a computer.

66
00:05:02,280 --> 00:05:06,810
Our goal is to convert text into vectors so that we can apply vector based methods.

67
00:05:07,590 --> 00:05:13,070
As you recall, machine learning works on numbers and numerical representations, whether they be text,

68
00:05:13,120 --> 00:05:15,810
images or other custom inputs you've defined.

69
00:05:16,410 --> 00:05:21,300
But the question is how do you convert text into numbers, specifically vectors?

70
00:05:21,840 --> 00:05:25,980
In fact, this is not a trivial process, and there are many possible solutions.

71
00:05:26,460 --> 00:05:31,320
Thus, this section will look at multiple approaches, and you'll also learn some applications for the

72
00:05:31,320 --> 00:05:34,800
real world, such as how to build a document retrieval system.

73
00:05:36,640 --> 00:05:42,310
The second major part of this course, which is like another little mini course, focuses on the probabilistic

74
00:05:42,310 --> 00:05:43,270
view of NLP.

75
00:05:43,990 --> 00:05:49,570
In particular, we look at language models, Markov models and other models inspired by these approaches.

76
00:05:50,200 --> 00:05:55,330
Now that might seem quite simplistic, but in fact, these basic ideas are very powerful.

77
00:05:55,690 --> 00:06:01,090
In fact, the approaches we take for building language models in this part of the course are a prerequisite

78
00:06:01,090 --> 00:06:02,860
for understanding transformers.

79
00:06:03,310 --> 00:06:07,810
So if you want to understand Transformers, which are the current state of the art, then you'll need

80
00:06:07,810 --> 00:06:08,950
to understand this.

81
00:06:09,940 --> 00:06:14,950
This part of the course, we'll look at more very interesting applications, such as how to generate

82
00:06:14,950 --> 00:06:19,540
text articles spinning and code breaking, also known as cipher decryption.

83
00:06:20,890 --> 00:06:25,750
The third major part of this course, which is again like another mini course, is all about machine

84
00:06:25,750 --> 00:06:25,990
learning.

85
00:06:25,990 --> 00:06:31,630
Methods for NLP in this course will differentiate between machine learning and deep learning in the

86
00:06:31,630 --> 00:06:36,940
sense that this part will focus on at nine deep learning approaches, even though deep learning is simply

87
00:06:36,940 --> 00:06:38,120
a part of machine learning.

88
00:06:38,980 --> 00:06:44,020
If you took version one of this course, then you should recognize many of the applications will discuss.

89
00:06:44,530 --> 00:06:49,030
Essentially, this part of the course is a revamped version of NLP version one.

90
00:06:50,110 --> 00:06:54,220
This part of the course will be application focused rather than technique focused.

91
00:06:54,640 --> 00:07:00,160
That is to say, each section will focus on a particular application rather than the underlying machine

92
00:07:00,160 --> 00:07:01,120
learning technique.

93
00:07:01,960 --> 00:07:04,750
All of these applications should be easy to understand.

94
00:07:04,960 --> 00:07:10,120
So you should find this section to be very approachable, even if you are just beginning your NLP journey.

95
00:07:10,990 --> 00:07:14,650
For example, you'll learn about spam detection and sentiment analysis.

96
00:07:15,040 --> 00:07:18,730
You'll also learn about latent semantic indexing, as well as topic modeling.

97
00:07:19,870 --> 00:07:22,750
All of these are extremely applicable in the real world.

98
00:07:23,260 --> 00:07:27,730
For instance, as mentioned, spam detection is something we make use of constantly.

99
00:07:28,120 --> 00:07:31,810
Your email service filters every email you receive to check for spam.

100
00:07:32,500 --> 00:07:37,840
Sentiment analysis is used by companies to manage their reputation and buy stock traders to predict

101
00:07:37,840 --> 00:07:39,010
stock price movements.

102
00:07:39,940 --> 00:07:44,890
Latent semantic indexing is used by SEO experts to improve their search performance.

103
00:07:46,070 --> 00:07:48,300
To understand why topic modeling is useful.

104
00:07:48,320 --> 00:07:51,610
Consider whether or not you have any documents you need to organize.

105
00:07:52,520 --> 00:07:57,500
Perhaps if you're like me, you have way too many documents that it would simply be impossible for you

106
00:07:57,500 --> 00:07:58,970
to read all of them yourself.

107
00:07:59,690 --> 00:08:02,270
But how can you organize them if you don't read them?

108
00:08:02,750 --> 00:08:05,960
The answer is to have a computer organize and read them for you.

109
00:08:06,590 --> 00:08:12,620
This approach has even been applied in areas such as bioinformatics and by law firms for document discovery.

110
00:08:13,610 --> 00:08:17,090
And remember, this part of the course is about machine learning approaches.

111
00:08:17,360 --> 00:08:23,030
So you learn about popular techniques for NLP, such as naive Bayes logistic regression, principal

112
00:08:23,030 --> 00:08:26,210
components analysis and latency richly allocation.

113
00:08:28,710 --> 00:08:32,970
The fourth major part of this course will look at deep learning and neural network methods.

114
00:08:33,390 --> 00:08:37,919
This part is distinct from the machine learning section because it requires more in-depth knowledge

115
00:08:37,919 --> 00:08:41,370
about how deep learning works in specialized libraries.

116
00:08:41,850 --> 00:08:46,980
But like the machine learning part, this part of the course will really combine everything you learned

117
00:08:47,250 --> 00:08:48,630
from the earlier parts.

118
00:08:49,200 --> 00:08:53,580
In particular, you'll see how deep learning methods apply, what you learned about vector based models,

119
00:08:53,820 --> 00:08:55,530
as well as probability models.

120
00:08:56,670 --> 00:09:02,270
The deep learning sections will focus on three main architectures, namely fully connected neural networks,

121
00:09:02,280 --> 00:09:08,100
also known as Inés convolutional neural networks, also known as CNN's and recurrent neural networks,

122
00:09:08,100 --> 00:09:09,480
also known as Arnett's.

123
00:09:09,960 --> 00:09:13,980
The aunt and section will look at popular units such as the Elysium and the.

124
00:09:14,160 --> 00:09:19,950
Or you will see how deep learning methods can really do any of the tasks we previously discussed, like

125
00:09:19,950 --> 00:09:24,840
spam detection, sentiment analysis, language modeling, article spinning and so forth.

126
00:09:25,740 --> 00:09:29,880
And obviously, deep learning includes modern architectures such as the Transformer.

127
00:09:30,480 --> 00:09:35,430
So what you learn in this section will essentially be a prerequisite if you ever want to understand

128
00:09:35,430 --> 00:09:36,750
how Transformers worked.

129
00:09:37,200 --> 00:09:42,210
So if you're curious about Bert and GPT three, these sections are simply an SSD.

130
00:09:44,440 --> 00:09:49,990
OK, so just to recap, this course will contain four major parts, each of which may contain multiple

131
00:09:49,990 --> 00:09:52,270
sections and is really a course on its own.

132
00:09:53,050 --> 00:09:58,540
The first part covers vector based approaches and focuses on how to convert text into numbers.

133
00:09:59,470 --> 00:10:04,960
The second part covers probability based approaches and will cover important topics like language modeling.

134
00:10:06,220 --> 00:10:09,070
The third part covers machine learning based approaches.

135
00:10:09,550 --> 00:10:14,230
This will be application focused, and you'll get to learn about many of the staples of NLP.

136
00:10:15,070 --> 00:10:17,800
The fourth part covers a deep learning based approaches.

137
00:10:18,150 --> 00:10:24,010
This will teach you how neural networks are used for NLP, so I hope you're as excited as I was when

138
00:10:24,010 --> 00:10:28,510
I learned these topics, and I'm confident that they will increase your knowledge, enhance your career,

139
00:10:28,750 --> 00:10:32,290
and even give you some business ideas that you can implement for yourself.

140
00:10:32,770 --> 00:10:34,780
Thanks for listening, and I'll see you in the next lecture.