1 00:00:04,830 --> 00:00:12,720 In this lesson, we are going to talk about a very important topic in the LLM applications field, which 2 00:00:12,720 --> 00:00:14,340 is cost control. 3 00:00:14,700 --> 00:00:20,370 You can have a big problem with cost if you are not careful. 4 00:00:20,370 --> 00:00:22,800 So this is a very important lesson. 5 00:00:28,120 --> 00:00:36,220 When you work with LM applications right now, let's see what happens in the in the in the following 6 00:00:36,250 --> 00:00:36,610 years. 7 00:00:36,610 --> 00:00:40,690 But right now cost is a very important issue. 8 00:00:41,260 --> 00:00:48,730 And regarding cost you are frequently going to face two questions. 9 00:00:49,240 --> 00:00:54,850 Do you use a private LM model or an open source model. 10 00:00:55,360 --> 00:00:58,990 And second, do you use the rack technique? 11 00:00:59,930 --> 00:01:06,920 Or do you use fine tuning or do you train an LM model from scratch? 12 00:01:07,430 --> 00:01:17,480 The second question I, I hope is very clear for you now, but it is not for some of the people you 13 00:01:17,480 --> 00:01:20,570 are going to deal with in your professional life. 14 00:01:20,570 --> 00:01:29,720 So there are many articles talking about fine tuning and training and LM, uh, models from scratch. 15 00:01:29,720 --> 00:01:32,810 And you have videos and courses and all that. 16 00:01:32,810 --> 00:01:41,060 And some people, which is not very familiar with LM applications, may tell you that this may be the 17 00:01:41,060 --> 00:01:42,710 way to go, the good way to go. 18 00:01:42,710 --> 00:01:43,370 Right. 19 00:01:43,580 --> 00:01:52,100 So my understanding is that right now, you know very well the answer to the second question regarding 20 00:01:52,100 --> 00:02:01,070 the first question, a let's talk a little bit about the cost of a private LM and remember, we told 21 00:02:01,070 --> 00:02:08,509 you that in the first year of LM applications, a private LMS have been the choice. 22 00:02:08,509 --> 00:02:15,290 So 99% of professional applications are using private LMS, especially ChatGPT. 23 00:02:17,120 --> 00:02:20,390 But this A is a. 24 00:02:21,520 --> 00:02:24,370 Difficult a matter. 25 00:02:24,370 --> 00:02:25,600 Why is that? 26 00:02:25,600 --> 00:02:35,380 Because a if you are not careful, you can spend a lot of money with your LM applications and private 27 00:02:35,410 --> 00:02:37,330 LM models like ChatGPT. 28 00:02:38,050 --> 00:02:43,630 So see for example, how much you can spend. 29 00:02:44,470 --> 00:02:54,940 So the main problem of a private LMS and LM applications is that the cost can escalate very quickly. 30 00:02:55,480 --> 00:02:59,350 Remember you pay for the number of tokens processed. 31 00:03:00,000 --> 00:03:03,750 And let's see a few numbers. 32 00:03:03,750 --> 00:03:05,040 So let's see. 33 00:03:05,040 --> 00:03:09,750 You want to summarize one page of text in ChatGPT. 34 00:03:11,370 --> 00:03:17,760 So this is going to cost you around $0.015. 35 00:03:18,180 --> 00:03:26,400 Nothing important, but who wants to summarize one page in a enterprise level application or in a, 36 00:03:26,430 --> 00:03:32,910 you know, open application, an application that is open to the world, right? 37 00:03:32,910 --> 00:03:33,270 Right. 38 00:03:33,270 --> 00:03:36,750 One page is something you do in one second. 39 00:03:36,750 --> 00:03:44,010 How many pages are you going to summarize in an LM application being used in a, I don't know, 50,000 40 00:03:44,010 --> 00:03:49,830 people company application or an application that is open to the whole world. 41 00:03:50,650 --> 00:03:54,820 So probably you are going to summarize millions of pages. 42 00:03:54,820 --> 00:03:58,570 And if you summarize, let's say. 43 00:03:59,840 --> 00:04:03,170 1500 pages per. 44 00:04:04,150 --> 00:04:07,990 Minute or per hour in this kind of applications. 45 00:04:08,080 --> 00:04:16,420 The cost of using ChatGPT as your private DM is going to be $20 per minute or per hour. 46 00:04:16,870 --> 00:04:21,040 So you have you can see the big. 47 00:04:21,700 --> 00:04:27,130 Jump, you know, between one page and 1500 pages. 48 00:04:27,130 --> 00:04:34,240 The cost of using private LMS can be very, very painful. 49 00:04:34,450 --> 00:04:39,610 It is one of the main considerations for a startups. 50 00:04:39,610 --> 00:04:49,270 So there are many startups that, uh, do not launch their projects or are extremely careful, uh, 51 00:04:49,270 --> 00:04:56,230 into when and how launch their project, uh, because of the cost, uh, problem. 52 00:04:56,230 --> 00:05:00,760 So this is one of the main things you will have to consider. 53 00:05:00,850 --> 00:05:05,860 You will have to calculate in advance how much is it going to cost? 54 00:05:05,860 --> 00:05:10,780 How are you going to control the cost and who is going to pay this cost. 55 00:05:10,900 --> 00:05:19,420 You will see that a private lens like, like, uh ChatGPT are going to offer you some tools in order 56 00:05:19,420 --> 00:05:26,470 to control, limit, you know, uh, get warnings when you have, uh, you know, an escalation of the 57 00:05:26,470 --> 00:05:27,610 cost, etcetera, etcetera. 58 00:05:27,610 --> 00:05:29,590 So be very careful with that. 59 00:05:29,590 --> 00:05:37,180 Pay attention with that before launching an LM application, because this can be a very painful, uh, 60 00:05:37,180 --> 00:05:39,670 very painful, uh, matter for you. 61 00:05:40,030 --> 00:05:48,460 Regarding the second question, we already know that fine tuning and training an LM model from scratch 62 00:05:48,460 --> 00:05:51,760 are things that, uh, we cannot do. 63 00:05:51,790 --> 00:05:59,680 These are things that only the very big companies or very big institutions with very big budgets and 64 00:05:59,680 --> 00:06:07,810 very big, uh, computational resources can afford for the rest of us, the rack technique is the way. 65 00:06:08,260 --> 00:06:08,530 Okay. 66 00:06:08,530 --> 00:06:11,800 So we we talk about that in a previous lesson. 67 00:06:11,800 --> 00:06:19,630 The the cost consideration is one of the main, uh, advantages of the rack technique. 68 00:06:19,960 --> 00:06:23,230 Uh, it's not the only one, but it's a very important one. 69 00:06:23,230 --> 00:06:27,850 And we will see how fine tuning and and training costs evolve. 70 00:06:27,850 --> 00:06:31,330 But right now is, uh, is a no brainer. 71 00:06:31,330 --> 00:06:38,680 I mean, there is nothing to, to to consider here because the difference is so huge that, uh, rack 72 00:06:38,680 --> 00:06:43,750 is being used in 99% of the professional LM applications. 73 00:06:43,750 --> 00:06:44,170 Okay. 74 00:06:44,170 --> 00:06:46,660 Where you find this kind of, of question. 75 00:06:46,660 --> 00:06:47,560 So. 76 00:06:48,610 --> 00:06:49,870 Cost control. 77 00:06:50,140 --> 00:06:58,240 When you are preparing the launch of a professional LMS application is something that you will have 78 00:06:58,240 --> 00:07:01,510 to study and prepare in great detail. 79 00:07:01,510 --> 00:07:12,430 So this is one area where you want to spend time and be very sure before a making an important mistake. 80 00:07:13,000 --> 00:07:20,680 In the next lesson, we are going to talk about other topics that you will, uh, face when you are 81 00:07:20,680 --> 00:07:23,500 preparing the launch of a professional application. 82 00:07:24,420 --> 00:07:25,710 LM ops. 83 00:07:25,950 --> 00:07:28,740 Let's see what it is in the next lesson.