1
00:00:11,110 --> 00:00:16,059
OK, so in this lecture, we will be introducing the next section of this course, which is all about

2
00:00:16,059 --> 00:00:17,110
Markov models.

3
00:00:17,620 --> 00:00:21,010
So what are Markov models and why should you learn about these things?

4
00:00:21,580 --> 00:00:23,750
Well, Markov models are essentially everywhere.

5
00:00:24,190 --> 00:00:29,470
Not only are they a staple in NLP, they can also be found in finance as they form the basis for the

6
00:00:29,470 --> 00:00:30,850
famous black holes formula.

7
00:00:31,570 --> 00:00:36,700
Markov models also act as the backbone of the Markov decision process, which is the framework that

8
00:00:36,700 --> 00:00:38,500
we use to do reinforcement learning.

9
00:00:39,670 --> 00:00:44,680
Furthermore, Markov models also act as the backbone of the Hidden Markov model, which has been very

10
00:00:44,680 --> 00:00:49,240
successful in applications like speech recognition and computational biology.

11
00:00:50,170 --> 00:00:56,740
Another major application of the Markov model is Markov Chain Monte Carlo, also known as EMC, which

12
00:00:56,740 --> 00:01:01,180
is a very powerful technique used in Bayesian machine learning, physics and more.

13
00:01:05,980 --> 00:01:09,340
OK, so now that you know why the Markov model is worth learning about.

14
00:01:09,670 --> 00:01:12,310
Let's discuss what we will look at in this section.

15
00:01:13,210 --> 00:01:15,250
We'll start by defining the mark of property.

16
00:01:16,270 --> 00:01:18,560
Note that this involves knowledge of probability.

17
00:01:18,580 --> 00:01:22,750
So if you are not comfortable with this subject, you may want to review it on your own.

18
00:01:23,770 --> 00:01:27,640
The next step will be to look at what a markup model is and how we define one.

19
00:01:28,540 --> 00:01:33,760
We'll look at the relevant mathematical objects that make up a markup model, such as the state transaction

20
00:01:33,760 --> 00:01:34,480
matrix.

21
00:01:35,110 --> 00:01:37,330
We'll also look at how to train a Markov model.

22
00:01:38,050 --> 00:01:43,990
As you recall, we are doing machine learning and thus training or learning is a key part of this process.

23
00:01:48,510 --> 00:01:53,490
Once we understand the theory behind Markov models, we will then look at how to apply Markov models

24
00:01:53,490 --> 00:01:54,990
to do an LP tasks.

25
00:01:56,450 --> 00:02:03,170
These include how to build a tax classifier using Bayes rule and how to generate your own tax to be

26
00:02:03,170 --> 00:02:04,010
more specific.

27
00:02:04,280 --> 00:02:09,770
You'll learn how to generate poetry by creating a Markov model based on an author's existing poems.

28
00:02:11,270 --> 00:02:16,040
The reason that these two applications are interesting is because they are two fundamentally different

29
00:02:16,040 --> 00:02:17,870
ways to apply machine learning.

30
00:02:18,770 --> 00:02:23,960
In one case, we are doing supervised learning, which involves trying to predict a target given a data

31
00:02:23,960 --> 00:02:25,640
set of inputs and labels.

32
00:02:26,120 --> 00:02:31,430
In the other case, we're using a machine learning model to create new things which did not exist before.

33
00:02:32,060 --> 00:02:37,190
This is an unsupervised task because it does not require any labels, only input text.