1
00:00:02,200 --> 00:00:07,000
Hey, everyone, and welcome back to this class natural language processing in Python.

2
00:00:11,620 --> 00:00:16,360
In this section of the course, we are going to look at a very unique project, one that you don't normally

3
00:00:16,360 --> 00:00:18,490
see in natural language processing.

4
00:00:19,210 --> 00:00:25,750
This project will teach you about multiple important NLP topics such as probabilistic modeling and genetic

5
00:00:25,750 --> 00:00:26,530
algorithms.

6
00:00:27,190 --> 00:00:31,810
The problem we're going to solve is that of decrypting coded messages or ciphers.

7
00:00:32,380 --> 00:00:36,940
Ciphers are very interesting because they are immediately applicable in the real world.

8
00:00:37,270 --> 00:00:40,030
For example, in warfare and espionage.

9
00:00:45,240 --> 00:00:49,410
In this lecture, we're going to outline what we will cover in the rest of this section.

10
00:00:50,280 --> 00:00:53,910
First, we're going to look at what is a cipher in the first place.

11
00:00:54,450 --> 00:00:57,750
This is the code that will allow us to encrypt our message.

12
00:00:58,350 --> 00:01:02,190
We'll do an example to see how we can encode and decode a message.

13
00:01:02,850 --> 00:01:04,650
Next, we'll look at language modeling.

14
00:01:05,220 --> 00:01:08,550
Here will build a probabilistic model of the English language.

15
00:01:09,090 --> 00:01:15,600
This allows us to ask questions like What is the probability of this sentence in more advanced sections

16
00:01:15,600 --> 00:01:16,320
and courses?

17
00:01:16,620 --> 00:01:20,670
You'll see how such language models can be used to generate new sentences.

18
00:01:21,120 --> 00:01:25,230
For example, you can generate poetry or do something like article spinning.

19
00:01:26,340 --> 00:01:31,380
Next, we'll look at the algorithm we're going to use to find the encryption code, which is called

20
00:01:31,380 --> 00:01:34,200
the genetic algorithm or an evolutionary algorithm.

21
00:01:34,770 --> 00:01:38,310
It's an algorithm that's based on the principles of genetic evolution.

22
00:01:39,030 --> 00:01:44,340
While that may sound complicated, it's actually nice because it's pretty intuitive and it doesn't involve

23
00:01:44,340 --> 00:01:47,970
any complicated math like some other machine learning algorithms do.

24
00:01:48,870 --> 00:01:54,270
So these are the three main concepts we need that all come together to form our solution to this problem.

25
00:01:54,960 --> 00:01:59,400
Number one, we have to understand a basic cipher and how to implement that in code.

26
00:02:00,060 --> 00:02:03,540
Number two, we have to understand probabilistic language models.

27
00:02:03,930 --> 00:02:09,509
And number three, we have to understand that genetic algorithms, which is what will allow our prediction

28
00:02:09,509 --> 00:02:14,160
of the encryption code to evolve and match our model of the English language.

29
00:02:14,790 --> 00:02:20,670
In other words, our translated message should have the highest likelihood since it is assumed to be

30
00:02:20,670 --> 00:02:21,660
in real English.

31
00:02:22,200 --> 00:02:27,360
Any other translation, meaning one that does not match the English language, should not have as high

32
00:02:27,360 --> 00:02:28,080
a likelihood.

33
00:02:28,620 --> 00:02:31,050
So this is a maximum likelihood problem.

34
00:02:36,170 --> 00:02:40,520
Once we've discussed all the theory we need for this section, we'll go ahead and implement everything

35
00:02:40,520 --> 00:02:41,090
in code.

36
00:02:41,780 --> 00:02:47,540
The code will be split up into multiple parts and I would recommend that you code as much of it as possible

37
00:02:47,780 --> 00:02:50,490
by yourself at every step of the way.

38
00:02:50,510 --> 00:02:55,280
I want you to ask yourself, can I finish this code without watching the rest of the videos?

39
00:02:55,790 --> 00:02:58,700
I hope that the answer is yes as soon as possible.

40
00:02:59,900 --> 00:03:05,540
Ideally, you would be able to code the entire thing yourself, or at least attempt to without looking

41
00:03:05,540 --> 00:03:07,190
at any of my code first.

42
00:03:07,580 --> 00:03:09,980
That's always the ideal approach in these courses.

43
00:03:10,610 --> 00:03:16,010
You'll also notice that this section follows the basic pattern I talked about earlier, going from theory

44
00:03:16,010 --> 00:03:17,810
to code in that order.

45
00:03:18,380 --> 00:03:22,220
So first, we discuss the theory and concepts behind our solution.

46
00:03:22,670 --> 00:03:25,850
Then we implement that theory in code only.

47
00:03:25,850 --> 00:03:28,190
The theory part contains new information.

48
00:03:28,610 --> 00:03:35,150
The coding part does not contain any new information and only serves to be an example of how to translate

49
00:03:35,150 --> 00:03:37,910
that theory into an actual computer program.

50
00:03:43,030 --> 00:03:45,880
I want to make a note that this section is entirely optional.

51
00:03:46,330 --> 00:03:47,380
Let me explain why.

52
00:03:48,190 --> 00:03:53,440
First, I think that this example will be more challenging than the rest of the examples in this course.

53
00:03:53,950 --> 00:03:59,140
A few students have told me they wanted a few more challenging examples, even though I originally designed

54
00:03:59,140 --> 00:04:00,370
this course for beginners.

55
00:04:00,940 --> 00:04:03,940
So this is a nice, challenging example for you to go through.

56
00:04:05,440 --> 00:04:10,570
I also think this is a nice example because it doesn't require specialized knowledge of machine learning,

57
00:04:10,870 --> 00:04:13,030
whereas the rest of this course kind of does.

58
00:04:13,720 --> 00:04:19,510
In this example, it requires knowledge of programming and probability and statistics, but it doesn't

59
00:04:19,510 --> 00:04:23,170
require you to understand an algorithm like naive Bayes or decision trees or a.

60
00:04:24,550 --> 00:04:29,950
Therefore, while this example will require you to work hard, it won't require specialized knowledge

61
00:04:29,950 --> 00:04:30,790
of machine learning.

62
00:04:31,450 --> 00:04:34,870
So it's not challenging because it requires all new theory.

63
00:04:35,290 --> 00:04:39,430
It's just challenging due to the fact that it's a lot of work to put all the pieces together.

64
00:04:40,870 --> 00:04:45,670
Since it's optional, you can feel free to skip this section for now and come back to a later.