1
00:00:04,520 --> 00:00:11,960
In this lesson, we are going to talk about misaligned behavior in the context of evaluation.

2
00:00:11,960 --> 00:00:15,230
One of the stages of LM ops.

3
00:00:20,110 --> 00:00:25,900
Let's talk about what it is, the misaligned behavior, and when can it happen?

4
00:00:27,350 --> 00:00:35,750
So the misaligned behavior is the name we use when we have a lack of alignment between the behavior

5
00:00:35,750 --> 00:00:44,660
of the LM application and the values of the company the organization, the the developer, the engineer.

6
00:00:45,530 --> 00:00:52,280
Foundational LM models have been created from content produced by humans.

7
00:00:52,280 --> 00:00:55,580
If these original contents were misaligned.

8
00:00:55,610 --> 00:00:59,060
The LM model will generate misaligned content.

9
00:00:59,090 --> 00:01:02,180
You remember that we talked about that before.

10
00:01:02,180 --> 00:01:13,310
So in many cases, the behavior of our LM application is going to, uh, respond to the training of

11
00:01:13,310 --> 00:01:15,470
the LM model it is using.

12
00:01:15,470 --> 00:01:23,960
But we need to prevent, uh, problems that can find with this, uh, with this situation.

13
00:01:23,960 --> 00:01:33,230
And it is a bit because of that, uh, possibility that we will work with some of the, uh, measures

14
00:01:33,230 --> 00:01:40,670
that we we will be we, we were talking about in the previous lessons, like guardrails, etc., where,

15
00:01:40,670 --> 00:01:45,230
where we can find, uh, situations of misaligned behavior.

16
00:01:45,230 --> 00:01:50,540
We can find them at any stage of the application life cycle.

17
00:01:51,020 --> 00:01:58,250
We can find them in the data acquisition phase, in the data preparation phase, in the data modeling

18
00:01:58,250 --> 00:02:01,280
phase, and in the data interpretation phase.

19
00:02:01,280 --> 00:02:11,630
So let's say, for example, that a in the data acquisition phase, uh, we uh, we probably will want

20
00:02:11,630 --> 00:02:20,180
to avoid loading personal data, loading insecure data, uh, have issues with permission data, etc..

21
00:02:20,180 --> 00:02:20,720
Right.

22
00:02:20,720 --> 00:02:28,550
In the data preparation phase, for example, we will need to answer questions like should some data

23
00:02:28,550 --> 00:02:35,450
be omitted, should some data be anonymized, is data encryption necessary, etc.?

24
00:02:35,450 --> 00:02:42,320
In the data modeling phase, we will be asking ourselves if is the model misaligned?

25
00:02:42,500 --> 00:02:50,480
Uh, if if we are making a proper use of randomization with our data, if, uh, does the model represent

26
00:02:50,480 --> 00:02:51,950
the data, etc..

27
00:02:51,950 --> 00:02:59,180
And in the data interpretation phase we are going to be asking ourselves, is the data misaligned?

28
00:02:59,180 --> 00:03:01,190
Is the data consistent?

29
00:03:01,220 --> 00:03:02,060
Uh, consistent?

30
00:03:02,060 --> 00:03:03,530
What implications?

31
00:03:03,530 --> 00:03:06,710
Uh, have, uh, you know, the misalignment etc., etc..

32
00:03:06,710 --> 00:03:17,600
So as you, uh, as you can see, uh, there are many more issues that we are going to be facing apart

33
00:03:17,600 --> 00:03:20,360
from the technical issues themselves.

34
00:03:20,360 --> 00:03:31,190
So you will have to evaluate the, uh, responses of your LM application before launching it in order

35
00:03:31,190 --> 00:03:36,860
to identify a bad behaviors like the one we are describing.

36
00:03:38,910 --> 00:03:47,070
In the next lesson, we will talk a little bit more about the lack of reproducibility and why it is

37
00:03:47,070 --> 00:03:56,250
so a relevant to a prepare evaluation of LM applications in the right manner.