1
00:00:06,330 --> 00:00:12,420
In this lesson we will talk about latency and speed in LM applications.

2
00:00:18,710 --> 00:00:19,580
So.

3
00:00:20,310 --> 00:00:27,030
This is an area where you are going to ask yourself if your.

4
00:00:27,820 --> 00:00:35,470
Current conditions are the right ones when you are preparing to launch an LM application.

5
00:00:36,860 --> 00:00:44,180
In some cases, as you know, cost and speed, uh, go in different in, in opposite ways.

6
00:00:44,180 --> 00:00:50,690
So you may have a faster application, but this is going to cost you more.

7
00:00:50,690 --> 00:00:52,460
It's going to be more expensive.

8
00:00:52,460 --> 00:00:58,850
And in some cases you may have a more convenient application, more cost efficient.

9
00:00:58,850 --> 00:01:04,459
But this is going to have an impact in the in the lower speed of the application.

10
00:01:04,819 --> 00:01:08,150
So let's talk a little bit about that.

11
00:01:08,150 --> 00:01:15,170
Uh, can you afford for the user not to have a fast experience?

12
00:01:15,710 --> 00:01:19,430
This is the key question you should ask yourself.

13
00:01:19,430 --> 00:01:30,230
Can you afford for the user not to have a fax, a fast experience so you can usually afford it if the

14
00:01:30,230 --> 00:01:32,810
users are employees of a company?

15
00:01:33,650 --> 00:01:40,220
And you usually cannot afford it if the users are customers.

16
00:01:40,220 --> 00:01:42,770
This is like the main principle.

17
00:01:43,790 --> 00:01:49,100
Second question that is uh, is is important for you to to consider.

18
00:01:49,370 --> 00:01:56,210
There are some, uh, applications that usually require high speed.

19
00:01:56,960 --> 00:02:02,510
And where we are talking about high speed, we are talking about low latency.

20
00:02:02,510 --> 00:02:05,450
So speed and latency are opposite.

21
00:02:05,480 --> 00:02:13,820
You have high speed applications when you have low latency and you have high latency when you have low

22
00:02:13,820 --> 00:02:14,300
speed.

23
00:02:14,900 --> 00:02:20,720
So there are some applications that usually need high speed.

24
00:02:21,950 --> 00:02:23,060
What are they?

25
00:02:23,640 --> 00:02:25,680
Conversational agents.

26
00:02:26,100 --> 00:02:27,630
Virtual agents.

27
00:02:27,930 --> 00:02:29,160
Chatbots.

28
00:02:29,340 --> 00:02:31,110
Content personalization.

29
00:02:31,680 --> 00:02:35,130
Apps and recommendation systems.

30
00:02:35,790 --> 00:02:43,830
All these applications, because of their characteristics and the usually the profile of the of their

31
00:02:43,830 --> 00:02:44,730
users.

32
00:02:45,370 --> 00:02:47,770
They require high speed.

33
00:02:47,770 --> 00:02:55,120
So if your LM application is in this category, usually you will have to worry about high speed.

34
00:02:56,020 --> 00:02:59,050
There are other applications that.

35
00:03:00,540 --> 00:03:07,080
Can afford low speed or lower speed than the previous ones.

36
00:03:07,290 --> 00:03:15,540
For example, when you are dealing with research applications and this is very frequent right now in

37
00:03:15,540 --> 00:03:17,730
the legal space, for example.

38
00:03:18,090 --> 00:03:28,830
So lawyers are using LLM applications in order to a be assisted in their research, in their legal research,

39
00:03:28,830 --> 00:03:37,560
or a marketing teams are using LLM applications and when they are working in market research, for example.

40
00:03:37,560 --> 00:03:38,040
Right.

41
00:03:38,040 --> 00:03:45,510
So these kind of applications can afford to be a not very fast.

42
00:03:46,080 --> 00:03:50,730
Other kinds of applications that can work well with low speed.

43
00:03:52,130 --> 00:03:59,660
Creative writing and content generation apps, especially if they are using.

44
00:03:59,660 --> 00:04:08,150
If they are used internally in the company, they usually will work well with a low speed.

45
00:04:08,150 --> 00:04:19,250
So the key question you have to keep in mind is how speed is going to affect user experience, especially

46
00:04:19,250 --> 00:04:23,060
how low speed is going to affect a user experience.

47
00:04:23,060 --> 00:04:33,890
If your user is going, uh, to be tired of waiting in your application and is going to a competitor's

48
00:04:33,950 --> 00:04:37,880
application, you cannot afford low speed.

49
00:04:38,660 --> 00:04:41,480
The second factor, of course, is cost.

50
00:04:41,480 --> 00:04:47,780
So how much is going to cause you to have a fast application?

51
00:04:47,810 --> 00:04:53,330
So these are questions that you will have to work on and work with your customers.

52
00:04:53,330 --> 00:04:59,330
For example, if you are working in a in a consulting firm or if you are working in a, in a tech department,

53
00:04:59,330 --> 00:05:05,090
and you you need to discuss, you know, this kind of budget questions, etc., with your manager or

54
00:05:05,090 --> 00:05:06,770
different people in your company.

55
00:05:06,770 --> 00:05:14,420
These are important questions you should solve before the launch of your LM application.

56
00:05:18,290 --> 00:05:26,270
We will see in the next lesson a more about cost in LM applications and cost control.

57
00:05:26,270 --> 00:05:27,980
Very important topic.

58
00:05:28,310 --> 00:05:28,850
A.

59
00:05:30,190 --> 00:05:32,980
When you are dealing with LM applications.