1 00:00:03,100 --> 00:00:03,430 Okay. 2 00:00:03,430 --> 00:00:06,640 So this is going to be a very, very interesting lesson. 3 00:00:06,640 --> 00:00:08,560 This one and the and the following ones. 4 00:00:08,560 --> 00:00:16,030 Remember that you are going to have these uh detailed presentation and also this detail notebook. 5 00:00:16,030 --> 00:00:17,980 You will be able to download it. 6 00:00:17,980 --> 00:00:23,500 So don't worry if in some cases the the the size of the letters are too small for you. 7 00:00:23,500 --> 00:00:24,400 Don't worry. 8 00:00:24,550 --> 00:00:32,590 Uh, this presentation is just an initial, uh, uh, round, uh, that you will complete when you look 9 00:00:32,590 --> 00:00:34,060 at the document yourself. 10 00:00:34,060 --> 00:00:34,450 Okay. 11 00:00:34,450 --> 00:00:42,730 So we are going to start with this Lang Lang Smith in depth from zero to advance operations. 12 00:00:42,730 --> 00:00:45,820 We have a long way to go. 13 00:00:45,820 --> 00:00:51,070 And we will start with a the Lang Smith definition. 14 00:00:51,070 --> 00:01:01,780 And this is more important than uh, than it seems because this is what Lang Chain says about the platform. 15 00:01:02,080 --> 00:01:10,000 So this is what the Lang Chain team explains about the Lang Smith platform. 16 00:01:10,000 --> 00:01:19,300 So the creators of the platform explain Lang Smith in her own words, and they say Lang Smith is a unified 17 00:01:19,300 --> 00:01:21,490 DevOps platform. 18 00:01:22,420 --> 00:01:31,870 For developing, collaborating, testing, deploying and monitoring LM applications. 19 00:01:32,620 --> 00:01:37,300 So it is, as you can see, a very ambitious platform. 20 00:01:37,510 --> 00:01:41,170 They want to cover all the steps and they are. 21 00:01:41,560 --> 00:01:51,580 But you can say that this is a very ambitious project and you are seeing this a word here, DevOps, 22 00:01:51,580 --> 00:01:55,780 which probably many of you are familiar with. 23 00:01:56,200 --> 00:02:03,430 Uh, remember that in this program we are using LM ops because LM ops is more. 24 00:02:04,470 --> 00:02:11,670 Like a word that in the LM application space are we used to. 25 00:02:11,670 --> 00:02:20,490 So when you in this context, when you read DevOps you may change DevOps by LM ops, okay. 26 00:02:20,490 --> 00:02:22,770 It's going to be the same for our purposes. 27 00:02:22,770 --> 00:02:31,950 So this is the definition of Lang Smith from the Lang Chain team, a unified DevOps platform for developing, 28 00:02:31,950 --> 00:02:36,450 collaborating, testing, deploying and monitoring applications. 29 00:02:36,540 --> 00:02:44,250 Take a look at these bullet points because, for example, this one deploying is still not at 100%. 30 00:02:44,430 --> 00:02:49,230 Um, so we will you we will see that this right now is like in beta. 31 00:02:49,620 --> 00:02:51,480 We will see why we say that. 32 00:02:51,480 --> 00:02:59,730 But right now it is a very good help, you know, for developing also for collaborating and not just 33 00:02:59,880 --> 00:03:06,300 between developers, also between developers and non-technical people in your team is a, of course, 34 00:03:06,300 --> 00:03:08,730 a good platform for testing the deployment. 35 00:03:08,730 --> 00:03:15,480 The deploying is something that is is going to to be better in the next future, and monitoring is already 36 00:03:15,480 --> 00:03:15,960 there. 37 00:03:16,350 --> 00:03:22,890 So they are using a these two pitch messages in their website. 38 00:03:22,890 --> 00:03:31,500 So you know that a long chain used to have a website and now they have different websites for their 39 00:03:31,500 --> 00:03:32,010 projects. 40 00:03:32,010 --> 00:03:35,340 So they have one website for a long chain. 41 00:03:35,340 --> 00:03:37,680 They have another website for long Smith. 42 00:03:37,680 --> 00:03:39,990 They have another website for long serve. 43 00:03:39,990 --> 00:03:45,390 So in this website is where they are talking about the Lang Smith platform. 44 00:03:45,390 --> 00:03:48,810 And they are using a couple of pitch messages. 45 00:03:48,810 --> 00:03:53,760 And I think it is very interesting to pay attention to what they they say here. 46 00:03:53,760 --> 00:04:00,870 So the first one is get your LM application from prototype to production. 47 00:04:00,870 --> 00:04:10,590 This is super interesting for us because the main goal of this program is precisely to go from the toy 48 00:04:10,590 --> 00:04:15,030 demo LM application to the production to the professional application. 49 00:04:15,030 --> 00:04:18,180 So this immediately got our attention. 50 00:04:18,600 --> 00:04:22,800 And they use this subtitle with this pitch message. 51 00:04:22,800 --> 00:04:30,720 They say all in one platform for every step of the LM power application cycle. 52 00:04:30,720 --> 00:04:31,230 Okay. 53 00:04:31,230 --> 00:04:39,360 So again they are talking about the full development cycle or the full production cycle. 54 00:04:40,150 --> 00:04:50,830 So the second pitch message is Lang Smith turns LM magic into enterprise ready application. 55 00:04:50,830 --> 00:04:52,270 This is very important. 56 00:04:52,270 --> 00:04:53,860 And they explain it here. 57 00:04:53,860 --> 00:04:57,340 No more guessing or development by bypass. 58 00:04:57,340 --> 00:05:02,980 Use testing to confirm that your LM application performs as desired. 59 00:05:02,980 --> 00:05:06,790 So we talk about that in the previous blog. 60 00:05:07,830 --> 00:05:17,640 The long chain team has been observing LM app development teams at work during one year, and one of 61 00:05:17,640 --> 00:05:26,520 the conclusions they made is like a lot of LM app development teams, they they don't do testing. 62 00:05:26,550 --> 00:05:29,970 They go from prototype to production. 63 00:05:29,970 --> 00:05:38,670 They use just gut feeling, you know, a and we need to improve that because if you don't do testing 64 00:05:38,910 --> 00:05:43,740 you are risking the quality of your of your application. 65 00:05:43,740 --> 00:05:50,730 We all know that testing in LM applications is is different from a classical application. 66 00:05:50,730 --> 00:05:55,440 So that's why, you know, people are are not using testing here. 67 00:05:55,440 --> 00:06:00,000 But we have ways to include testing in our process. 68 00:06:00,000 --> 00:06:03,300 And locksmith is a very good help in that direction. 69 00:06:03,300 --> 00:06:11,220 So this is very interesting because what Lang Smith, what the the Lang Chain team is saying here is 70 00:06:11,220 --> 00:06:18,120 that until now a lot of LM projects are magical. 71 00:06:18,120 --> 00:06:24,210 Magical in a sense that they, they, they are not based on on science. 72 00:06:24,210 --> 00:06:26,160 They are not based on testing. 73 00:06:26,160 --> 00:06:28,770 They are just based on gut feeling. 74 00:06:28,770 --> 00:06:29,490 Right? 75 00:06:29,610 --> 00:06:35,310 It's totally understandable because LM, LM applications are still very, very young. 76 00:06:35,310 --> 00:06:37,680 So we are growing. 77 00:06:37,680 --> 00:06:37,980 Right. 78 00:06:37,980 --> 00:06:45,390 But this is I understand this is a very, very interesting, interesting, uh, pitch message from uh, 79 00:06:45,390 --> 00:06:46,020 Lang Smith. 80 00:06:46,020 --> 00:06:53,640 So let's talk a little bit about Lang Smith terminology, because in the previous vlogs, in the previous 81 00:06:53,640 --> 00:07:00,240 vlog you started a listening weird words, right? 82 00:07:00,240 --> 00:07:07,980 We we talk about rants, we talk about data sets, we talk about playground or have, uh, we talk about 83 00:07:07,980 --> 00:07:09,210 regression testing. 84 00:07:09,210 --> 00:07:09,570 So. 85 00:07:09,570 --> 00:07:15,810 Okay, what what does this mean in, in the context of LM app development. 86 00:07:16,170 --> 00:07:24,600 So for us, I think it is very valuable to have this kind of, you know, dictionary. 87 00:07:24,600 --> 00:07:30,240 Right, uh, where you can come back whenever you have a doubt. 88 00:07:30,240 --> 00:07:38,670 So what is a trace traces record is a record of interactions with the LM applications. 89 00:07:38,670 --> 00:07:43,140 They can contain more info beyond just input and output. 90 00:07:43,140 --> 00:07:47,010 So if we go to the Lang Smith platform. 91 00:07:47,010 --> 00:07:50,430 So here you have the Lang Smith platform open. 92 00:07:51,000 --> 00:07:54,120 Here we have some small projects. 93 00:07:54,120 --> 00:08:00,600 You will see one basic project at work in a, in a next uh block. 94 00:08:00,600 --> 00:08:07,260 And then you will see also a professional project at work, which is super interesting because the more 95 00:08:07,260 --> 00:08:14,580 sophisticated is the project, the most value adds uh, the Lang Smith platform. 96 00:08:14,580 --> 00:08:21,300 So you will see that it's going to be very interesting to see the Lang Smith platform with a professional 97 00:08:21,300 --> 00:08:21,840 project. 98 00:08:21,840 --> 00:08:29,130 This is going to be a very interesting for you, but let's take a look at one very simple project. 99 00:08:29,190 --> 00:08:36,150 Uh, so as you as you see here in this, uh, Lang Smith platform, we have a number of projects already 100 00:08:36,150 --> 00:08:36,870 created. 101 00:08:36,870 --> 00:08:41,549 And if we click on one of the projects, these are the traces. 102 00:08:41,549 --> 00:08:42,150 Okay. 103 00:08:42,150 --> 00:08:47,280 So if we click in one of the traces, this is a rag a simple rag application. 104 00:08:47,850 --> 00:08:50,820 We see that one trace is like one. 105 00:08:50,820 --> 00:09:01,620 We can we can call one operation or one one uh, interaction between the uh application and the LM model. 106 00:09:01,620 --> 00:09:02,040 Right. 107 00:09:02,040 --> 00:09:08,850 So here you can see the different steps from the input to the output. 108 00:09:08,850 --> 00:09:10,710 You see the input, you see the output. 109 00:09:10,710 --> 00:09:12,660 You also see the different steps. 110 00:09:12,660 --> 00:09:19,380 And if you pay attention here in this table you can see the latency. 111 00:09:19,380 --> 00:09:23,040 If this trace is included in the test data set. 112 00:09:23,040 --> 00:09:29,370 If we have included some annotations, the number of tokens we have used the cost because this is associated 113 00:09:29,370 --> 00:09:30,870 with with OpenAI. 114 00:09:30,900 --> 00:09:33,990 So the cost is something relevant here, etc.. 115 00:09:33,990 --> 00:09:34,530 Right. 116 00:09:35,490 --> 00:09:42,840 But the most interesting thing is to see that traces have several steps. 117 00:09:43,050 --> 00:09:48,900 So you are already familiar with the Rag technique, and you know that in the Rag technique we have 118 00:09:48,900 --> 00:09:49,710 several steps. 119 00:09:49,710 --> 00:09:57,510 So here you have the steps you can investigate uh, in these steps, you know, if if something is uh, 120 00:09:57,750 --> 00:10:02,490 uh is failing in order to find what is happening in each of these steps. 121 00:10:02,490 --> 00:10:05,970 So these steps are called runs. 122 00:10:06,000 --> 00:10:06,690 Okay. 123 00:10:07,010 --> 00:10:13,160 So the LM call is just the call to the LM model, just input and output. 124 00:10:13,160 --> 00:10:13,490 Okay. 125 00:10:13,490 --> 00:10:18,470 So if you come here you see that the LM call is super simple. 126 00:10:19,190 --> 00:10:21,950 So you have input. 127 00:10:22,670 --> 00:10:23,660 An output. 128 00:10:27,940 --> 00:10:32,050 If you go to the traces, you have much more information. 129 00:10:32,050 --> 00:10:36,280 And if you go to the runs, that's a total different story. 130 00:10:36,280 --> 00:10:38,440 The runs are small steps. 131 00:10:38,440 --> 00:10:38,950 Okay. 132 00:10:38,950 --> 00:10:45,760 So if you go to one trace, you can see different runs inside of a trace okay. 133 00:10:45,760 --> 00:10:53,260 So this is important because if you remember in the previous blog we talk about the possibility to include 134 00:10:53,260 --> 00:10:56,410 runs in the test data set. 135 00:10:56,410 --> 00:10:56,770 Okay. 136 00:10:56,770 --> 00:10:59,410 So here you have the difference traces. 137 00:10:59,410 --> 00:11:08,020 Record of interactions with the LM application can contain more info beyond just input and output LM 138 00:11:08,020 --> 00:11:11,530 calls calls to the LM model, input and output. 139 00:11:11,530 --> 00:11:21,520 Run a single execution of the LM application to process an input and generate an output annotation queues 140 00:11:21,520 --> 00:11:25,180 to add human labels and feedback to the traces. 141 00:11:25,600 --> 00:11:27,040 This is more on traces. 142 00:11:27,040 --> 00:11:34,030 This is more a easy to to to predict right data sets for evaluation. 143 00:11:34,030 --> 00:11:41,680 Few shot prompting or fine tuning build data sets from examples, production data or existing sources. 144 00:11:41,680 --> 00:11:45,370 You remember we talk about test data sets in the previous vlog. 145 00:11:46,360 --> 00:11:47,920 What is the hub? 146 00:11:48,550 --> 00:11:50,860 So this is the hub. 147 00:11:50,860 --> 00:11:53,470 They are they are calling. 148 00:11:53,470 --> 00:11:55,870 Well sometimes it's a little bit confusing. 149 00:11:55,870 --> 00:12:02,230 They have and the playground because actually you come here to the hub. 150 00:12:02,950 --> 00:12:12,220 You will see that here you have a list of prompt examples that you can reuse or you can modify. 151 00:12:12,250 --> 00:12:21,970 For example, you want to use a prompts to extract for extraction and using, for example, OpenAI GPT 152 00:12:22,000 --> 00:12:23,710 three five. 153 00:12:23,740 --> 00:12:24,280 Okay. 154 00:12:24,400 --> 00:12:26,200 So you have a number of them. 155 00:12:26,200 --> 00:12:31,540 This is a brown YouTube transcript to article. 156 00:12:31,540 --> 00:12:34,660 Convert any YouTube video transcript into an article. 157 00:12:34,660 --> 00:12:35,050 Okay. 158 00:12:35,050 --> 00:12:40,390 You go here and you see that this is a prompt that someone has created. 159 00:12:40,390 --> 00:12:44,650 And we have it here if we want to reuse it or modify or whatever. 160 00:12:44,650 --> 00:12:50,230 And if you want to edit, this is where you go to the when you go to the playground. 161 00:12:50,230 --> 00:12:54,670 So sometimes it's a little bit confusing the how and the and the and the playground. 162 00:12:54,670 --> 00:12:56,620 But the interesting thing. 163 00:12:56,890 --> 00:13:05,770 So we define the Lang Smith prompt half uh, as uh, you know, an environment for prompt engineering 164 00:13:05,770 --> 00:13:07,030 experimentation. 165 00:13:07,030 --> 00:13:07,480 Okay. 166 00:13:07,480 --> 00:13:13,570 We will see more about that later, because it is more interesting that in Sims, it's not just for 167 00:13:13,570 --> 00:13:19,600 playing with prompt engineering, because you can come here to any trace. 168 00:13:19,600 --> 00:13:27,580 And in this trace you can go to the playground and start playing with different LM models, uh, late 169 00:13:27,580 --> 00:13:30,580 model feature, you know, different prompts, etc.. 170 00:13:30,580 --> 00:13:31,960 So we will go there. 171 00:13:31,960 --> 00:13:34,990 It's more interesting than than than than it seems. 172 00:13:35,230 --> 00:13:37,750 Then what do we understand. 173 00:13:37,840 --> 00:13:42,430 Uh, for by collaboration in in in in this platform. 174 00:13:42,700 --> 00:13:48,640 So we, we understand collaboration between developers and subject matter experts mostly. 175 00:13:48,640 --> 00:13:56,290 So remember that we talk about product managers, a subject matter experts and software developers and 176 00:13:56,290 --> 00:13:57,610 LM app developers. 177 00:13:57,610 --> 00:13:57,910 Right. 178 00:13:57,910 --> 00:14:05,020 So collaboration these are platform that can include the annotations, uh, and the feedback of this 179 00:14:05,020 --> 00:14:11,350 kind of, uh, members of your team and of course, the feedback from the, the, the beta users and 180 00:14:11,350 --> 00:14:13,870 the final users of the, of the application. 181 00:14:14,590 --> 00:14:17,080 What is auto evaluation? 182 00:14:17,830 --> 00:14:27,580 You can use an LM and prompt to score your application output, or write your own functional evaluation 183 00:14:27,580 --> 00:14:32,110 test to record different measures of effectiveness. 184 00:14:32,320 --> 00:14:34,180 You will see this okay. 185 00:14:34,180 --> 00:14:39,190 So we will talk more about the evaluation about and about the different possibilities. 186 00:14:39,190 --> 00:14:46,030 When you hear when you listen about when you hear about regression testing, and sure that the new features 187 00:14:46,030 --> 00:14:52,540 or updates do not adversely affect the existing functionalities of the LM application. 188 00:14:52,540 --> 00:14:55,870 So but this is saying is when when you. 189 00:14:56,470 --> 00:15:03,850 When you listen or when you read about regression testing in in the following lessons. 190 00:15:03,850 --> 00:15:04,780 Think about that. 191 00:15:04,780 --> 00:15:13,180 So a are the changes we are making in the application a affecting negatively to what we already have 192 00:15:13,180 --> 00:15:13,540 there? 193 00:15:13,540 --> 00:15:15,850 We will talk more about this later. 194 00:15:16,450 --> 00:15:21,610 Then we have another a terme which is online evaluation. 195 00:15:21,610 --> 00:15:25,180 This is a next feature of long chain. 196 00:15:25,180 --> 00:15:26,620 This is coming soon. 197 00:15:26,920 --> 00:15:36,700 They say that they plan to have this new feature that will continuously track qualitative characteristics 198 00:15:36,700 --> 00:15:42,910 of any life application and spot issues in real time with monitoring. 199 00:15:43,630 --> 00:15:46,960 When we talk about observability, what do we mean? 200 00:15:46,960 --> 00:15:51,310 Monitoring health and performance of the LM applications. 201 00:15:51,310 --> 00:15:53,650 Insights into its behavior. 202 00:15:53,650 --> 00:15:59,920 Performance of the LM model, and the interactions between the LM application and the LM model. 203 00:15:59,920 --> 00:16:06,790 Okay, observability is a very frequent terms in LM ops, as you will see. 204 00:16:07,180 --> 00:16:13,480 Then we may hear deployment and one click deploy. 205 00:16:13,480 --> 00:16:22,780 So deployment to a in this context in the Lang Smith context when they say when they talk about one 206 00:16:22,780 --> 00:16:27,700 click deploy or deployment, they are talking about deployment to lang serve. 207 00:16:27,700 --> 00:16:34,000 As you know lang serve is the the another product from Lang chain. 208 00:16:34,450 --> 00:16:39,520 But this feature right now in in Lang Smith is still in beta. 209 00:16:39,520 --> 00:16:44,200 And you will see that right now Lang Smith has three versions. 210 00:16:44,200 --> 00:16:46,090 You have like the free version. 211 00:16:46,090 --> 00:16:49,390 Then you have the plus version and the enterprise version. 212 00:16:49,390 --> 00:16:58,600 So this possibility, the deployment to Lang serve is in beta and it is only available for the two higher 213 00:16:58,600 --> 00:17:01,780 versions, the plus version and the enterprise version. 214 00:17:02,080 --> 00:17:11,050 So in the notebook we have included a link where you can have a more when you can find more Lang Smith 215 00:17:11,050 --> 00:17:11,770 terminology. 216 00:17:11,770 --> 00:17:18,130 In fact, you are going to see some other links to the Lang Smith documentation. 217 00:17:18,130 --> 00:17:20,980 They they have a good documentation prepared. 218 00:17:20,980 --> 00:17:22,060 So this is interesting. 219 00:17:22,060 --> 00:17:26,500 And we have included some links here uh, for you as well. 220 00:17:26,500 --> 00:17:27,069 Okay. 221 00:17:27,069 --> 00:17:34,360 So let's talk about the most frequent ask questions regarding Lang Smith. 222 00:17:34,420 --> 00:17:34,780 Uh. 223 00:17:35,800 --> 00:17:44,650 So these are the six questions that, in the opinion of the long chain team, are the more relevant 224 00:17:44,650 --> 00:17:48,790 or the most frequently asked regarding the platform. 225 00:17:48,790 --> 00:17:54,250 So people ask, can you use language myth if you are not using long chain? 226 00:17:54,640 --> 00:17:56,560 The answer is yes. 227 00:17:56,560 --> 00:18:00,640 You can use language myth even if you don't use long chain. 228 00:18:01,780 --> 00:18:05,500 Second, can you self-host long Smith. 229 00:18:06,560 --> 00:18:12,890 Yes, you can sell horse meat, but with enterprise pricing, with the enterprise version. 230 00:18:12,890 --> 00:18:13,190 Okay. 231 00:18:13,190 --> 00:18:21,230 This is something that some big companies can be interested in, you know, because it involves security, 232 00:18:21,230 --> 00:18:22,850 privacy, etc., etc.. 233 00:18:22,850 --> 00:18:30,650 So this is a feature that is available but is only for the higher version of the platform. 234 00:18:31,400 --> 00:18:32,900 Is Lang Smith Secure? 235 00:18:32,900 --> 00:18:34,400 Very important question. 236 00:18:34,790 --> 00:18:41,180 Answer Lang Smith traces are encrypted and stored securely. 237 00:18:41,480 --> 00:18:43,700 This is the Lang chain team talking. 238 00:18:43,700 --> 00:18:47,480 So they are a showing. 239 00:18:47,480 --> 00:18:54,140 Or they are stating that Lang Smith traces are encrypted and stored securely. 240 00:18:54,260 --> 00:18:58,220 Can you limit the number of traces sent to Lang Smith? 241 00:18:58,220 --> 00:18:59,120 This is very important. 242 00:18:59,120 --> 00:18:59,690 Why? 243 00:18:59,690 --> 00:19:08,600 This is very important because you are going to pay once you reach the free limit in the number of traces, 244 00:19:08,600 --> 00:19:11,660 which I think is 8000 per month. 245 00:19:12,380 --> 00:19:20,660 If you want to go over that, you will have to pay, and you pay by the number of traces you you use. 246 00:19:20,660 --> 00:19:20,930 Right. 247 00:19:20,930 --> 00:19:29,030 So it is very important to to understand, to know that you can limit the number of traces that you 248 00:19:29,030 --> 00:19:30,170 send to Lang Smith. 249 00:19:30,170 --> 00:19:35,600 So you can specify the percentage of traces you send to Lang Smith. 250 00:19:35,600 --> 00:19:37,160 And we highlight here. 251 00:19:37,160 --> 00:19:39,200 This can reduce your cost to Lang. 252 00:19:39,500 --> 00:19:40,760 So this is very important. 253 00:19:42,050 --> 00:19:46,700 Next question does Lang Smith slow your application. 254 00:19:46,700 --> 00:19:47,540 Answer. 255 00:19:47,540 --> 00:19:51,950 Lang Smith does not make your application slower. 256 00:19:52,520 --> 00:19:57,260 And finally does Lang Smith use your data. 257 00:19:57,260 --> 00:19:58,190 Answer. 258 00:19:58,190 --> 00:20:01,160 Lang Smith does not use your data. 259 00:20:01,160 --> 00:20:05,450 So they do not use your data for training purposes or anything like that. 260 00:20:05,450 --> 00:20:09,170 So very, very interesting lesson. 261 00:20:09,170 --> 00:20:15,950 In the next lesson we are going to talk about the initial operations, how to create an account, how 262 00:20:15,950 --> 00:20:19,820 to create a project, etc. we will see this in the next lesson.