WEBVTT 1 00:00:00.540 --> 00:00:02.730 Hey there, Eden here, and in this video, 2 00:00:02.730 --> 00:00:05.910 we're going to implement the actor agent, 3 00:00:05.910 --> 00:00:07.470 so we're going to build what's called 4 00:00:07.470 --> 00:00:08.940 the first_responder_chain, 5 00:00:08.940 --> 00:00:12.090 which is going to be the first step of our graph, 6 00:00:12.090 --> 00:00:15.180 which is going to take as input the user query 7 00:00:15.180 --> 00:00:18.060 and generate the initial article. 8 00:00:18.060 --> 00:00:21.120 Now, we're going to cover some cool prompting techniques. 9 00:00:21.120 --> 00:00:23.370 We're going to use output parsers, 10 00:00:23.370 --> 00:00:25.710 specifically leveraging function calling 11 00:00:25.710 --> 00:00:27.870 in order to get structured output. 12 00:00:27.870 --> 00:00:31.323 As always, the code is available in the course's resources. 13 00:00:34.650 --> 00:00:35.850 Start with the imports, 14 00:00:35.850 --> 00:00:39.060 and let's import datetime because we're going to pass 15 00:00:39.060 --> 00:00:42.270 to our agent the current date and time. 16 00:00:42.270 --> 00:00:45.317 As always, we also want to import from dotenv 17 00:00:45.317 --> 00:00:47.190 the load_dotenv() function, 18 00:00:47.190 --> 00:00:49.260 which is going to take the environment variables 19 00:00:49.260 --> 00:00:50.583 from our dotenv file. 20 00:00:51.690 --> 00:00:54.750 And we want to import from LangChain output parsers 21 00:00:54.750 --> 00:00:57.780 some output parsers that would handle the output 22 00:00:57.780 --> 00:01:00.480 from the function calling from OpenAI, 23 00:01:00.480 --> 00:01:02.610 so I'm going to import JsonOutputToolsParser 24 00:01:04.020 --> 00:01:06.090 and the PydanticToolsParser, 25 00:01:06.090 --> 00:01:09.360 and both output parsers are going to take back the response 26 00:01:09.360 --> 00:01:12.810 we get the LLM with the function-calling invocation, 27 00:01:12.810 --> 00:01:15.690 and it's going to take the function-calling invocation 28 00:01:15.690 --> 00:01:18.210 and it's going to either transform it 29 00:01:18.210 --> 00:01:20.190 into a JSON to a dictionary 30 00:01:20.190 --> 00:01:24.210 or it's going to transform it into Pydantic object. 31 00:01:24.210 --> 00:01:25.080 Later in this video, 32 00:01:25.080 --> 00:01:27.453 we'll see what Pydantic object it will return. 33 00:01:28.710 --> 00:01:31.530 And we'll also import HumanMessage from LangChain 34 00:01:31.530 --> 00:01:34.293 because we're going to pass that into our LLM. 35 00:01:35.220 --> 00:01:37.620 We want to import the ChatPromptTemplate, 36 00:01:37.620 --> 00:01:40.260 and this is going to hold all of our history 37 00:01:40.260 --> 00:01:42.000 of our agent iterations, 38 00:01:42.000 --> 00:01:45.150 so we're going to append messages through that. 39 00:01:45.150 --> 00:01:47.760 And also, we want a message placeholder, 40 00:01:47.760 --> 00:01:52.380 so that is also useful when we'll plug in the history 41 00:01:52.380 --> 00:01:54.330 and the message placeholder is useful 42 00:01:54.330 --> 00:01:56.820 to being a placeholder for new messages, 43 00:01:56.820 --> 00:01:58.560 and we saw already how it is used 44 00:01:58.560 --> 00:02:01.203 in the reflection agent in the previous section. 45 00:02:02.280 --> 00:02:05.520 And let's also import ChatOpenAI 46 00:02:05.520 --> 00:02:08.553 because we're going to be using GPT-4 Turbo for that. 47 00:02:09.540 --> 00:02:12.480 All right, let's start writing our main prompt 48 00:02:12.480 --> 00:02:15.003 that will be used by our agent. 49 00:02:15.900 --> 00:02:18.480 And I remind you, the input for this agent 50 00:02:18.480 --> 00:02:21.180 is going to be the topic that we want to write about, 51 00:02:21.180 --> 00:02:24.360 and the agent is going to write us the initial response. 52 00:02:24.360 --> 00:02:26.370 Now, in the answer of the agent, 53 00:02:26.370 --> 00:02:30.090 we need the content, the first draft of the article, 54 00:02:30.090 --> 00:02:32.070 we want to also have a critique, 55 00:02:32.070 --> 00:02:35.460 so some criticism on the newly created article, 56 00:02:35.460 --> 00:02:39.210 and some search terms that will help enhance the article. 57 00:02:39.210 --> 00:02:40.320 So, before we begin, 58 00:02:40.320 --> 00:02:42.990 I have to give a quick hint on the implementation. 59 00:02:42.990 --> 00:02:45.690 We're going to be using the MessageGraph(), 60 00:02:45.690 --> 00:02:48.180 so this means the state of the graph nodes 61 00:02:48.180 --> 00:02:50.760 that will be changing upon every node 62 00:02:50.760 --> 00:02:54.210 is going to be a simple list of messages. 63 00:02:54.210 --> 00:02:56.850 All right, let's go and review the main prompt 64 00:02:56.850 --> 00:02:58.590 of our actor agent. 65 00:02:58.590 --> 00:03:00.300 So, I'm going to define a variable, 66 00:03:00.300 --> 00:03:02.250 I'll call it actor_prompt_template, 67 00:03:02.250 --> 00:03:06.270 and it's going to be a ChatPromptTemplate from the messages, 68 00:03:06.270 --> 00:03:10.050 and the messages are going to be here, the main prompt, 69 00:03:10.050 --> 00:03:13.683 and all of our history that we used to get so far. 70 00:03:15.210 --> 00:03:17.040 Let's review the system prompt. 71 00:03:17.040 --> 00:03:19.410 So, you are an expert researcher. 72 00:03:19.410 --> 00:03:20.940 The Current time is {time}, 73 00:03:20.940 --> 00:03:23.730 and this is we're going to plug in dynamically. 74 00:03:23.730 --> 00:03:27.450 And the prompt's output indicator contains three parts. 75 00:03:27.450 --> 00:03:30.897 The first part is a placeholder of {first_instructions}, 76 00:03:30.897 --> 00:03:32.910 and here, we're going to plug in 77 00:03:32.910 --> 00:03:36.900 to simply write a 250-word essay. 78 00:03:36.900 --> 00:03:38.107 The second is going to be, 79 00:03:38.107 --> 00:03:40.110 "Reflect and critique your answer. 80 00:03:40.110 --> 00:03:42.630 Be severe to maximize improvement." 81 00:03:42.630 --> 00:03:45.750 And this critique is going to be used later 82 00:03:45.750 --> 00:03:48.150 by the reviser agent. 83 00:03:48.150 --> 00:03:49.837 And the third part here is, 84 00:03:49.837 --> 00:03:52.500 "Recommend search queries to research information 85 00:03:52.500 --> 00:03:54.420 and improve your answer." 86 00:03:54.420 --> 00:03:56.460 So this is only the search query, 87 00:03:56.460 --> 00:03:58.890 so this is not yet the search result, 88 00:03:58.890 --> 00:04:01.320 and we're going to use the search queries 89 00:04:01.320 --> 00:04:03.210 in the tool execution node 90 00:04:03.210 --> 00:04:06.930 where we'll be leveraging Tavily search engine. 91 00:04:06.930 --> 00:04:09.093 So, this was the system prompt, 92 00:04:10.290 --> 00:04:11.700 and now it's time to introduce 93 00:04:11.700 --> 00:04:13.380 a prompt engineering technique 94 00:04:13.380 --> 00:04:16.170 to reuse this prompt template. 95 00:04:16.170 --> 00:04:18.870 This prompt template is also going to be used 96 00:04:18.870 --> 00:04:21.750 by our reviser node, the node which is going 97 00:04:21.750 --> 00:04:23.700 to take now all of the information 98 00:04:23.700 --> 00:04:26.670 and it's going to rewrite the article. 99 00:04:26.670 --> 00:04:28.920 So, because we're going to be using that 100 00:04:28.920 --> 00:04:30.900 in our reviser agent, which is going 101 00:04:30.900 --> 00:04:35.550 to keep revising and critiquing, revising and critiquing, 102 00:04:35.550 --> 00:04:39.270 then we also want to pass in the message placeholder 103 00:04:39.270 --> 00:04:41.310 of all of the history before that. 104 00:04:41.310 --> 00:04:43.950 So, all the information of what to search 105 00:04:43.950 --> 00:04:46.263 and what was critiqued is going to be here. 106 00:04:49.140 --> 00:04:51.300 If you didn't fully understand what are we going 107 00:04:51.300 --> 00:04:54.510 to do in the reviser agent, don't worry, that is okay. 108 00:04:54.510 --> 00:04:57.870 We're going to cover this in depth in the next video. 109 00:04:57.870 --> 00:05:00.780 However, what's now is important for you to understand 110 00:05:00.780 --> 00:05:03.600 is the first prompt we're going to send to the LLM, 111 00:05:03.600 --> 00:05:05.640 that instead of our {first_instruction}, 112 00:05:05.640 --> 00:05:07.650 we're going to plug in an instruction 113 00:05:07.650 --> 00:05:10.380 to write a 250-words essay. 114 00:05:10.380 --> 00:05:14.010 And then the agent will generate us the first draft 115 00:05:14.010 --> 00:05:16.890 with the critique and with the search queries 116 00:05:16.890 --> 00:05:18.873 that we need to search later. 117 00:05:22.680 --> 00:05:25.470 We're almost done with our prompt template, 118 00:05:25.470 --> 00:05:29.130 and now we want to use the partial() method 119 00:05:29.130 --> 00:05:33.780 in order to populate some already known placeholders. 120 00:05:33.780 --> 00:05:36.210 When we invoke this prompt template, 121 00:05:36.210 --> 00:05:39.420 then we want to plug in here the current date. 122 00:05:39.420 --> 00:05:42.240 So, we're simply going to use a lambda function 123 00:05:42.240 --> 00:05:44.970 that will output us today's date, 124 00:05:44.970 --> 00:05:47.730 and we're going to give it in the ISO format. 125 00:05:47.730 --> 00:05:49.500 And this will be computed only 126 00:05:49.500 --> 00:05:51.570 when we invoke this prompt template, 127 00:05:51.570 --> 00:05:54.093 and this will be when we invoke our agent. 128 00:05:57.150 --> 00:05:58.980 Righty, we're done with the prompt 129 00:05:58.980 --> 00:06:00.900 that our agent is going to use, 130 00:06:00.900 --> 00:06:02.460 but now we want to ensure 131 00:06:02.460 --> 00:06:05.310 that the output we get from the LLM then 132 00:06:05.310 --> 00:06:07.890 it is in a structured format, 133 00:06:07.890 --> 00:06:11.370 so we want the format to be with a response field 134 00:06:11.370 --> 00:06:13.500 that is having the original essay, 135 00:06:13.500 --> 00:06:14.820 we want the critique field, 136 00:06:14.820 --> 00:06:17.640 which is having the critique for that essay, 137 00:06:17.640 --> 00:06:19.110 and we want a search field, 138 00:06:19.110 --> 00:06:23.100 which will be a list of values that we should search for. 139 00:06:23.100 --> 00:06:25.860 And for that, we're going to leverage function calling 140 00:06:25.860 --> 00:06:29.370 and specifically function calling that would make sure 141 00:06:29.370 --> 00:06:32.130 that the output format of the LLM 142 00:06:32.130 --> 00:06:35.433 is going to be in a object we create. 143 00:06:39.390 --> 00:06:44.220 Let's create a new file, and we'll call it schemas.py. 144 00:06:44.220 --> 00:06:46.620 And this file is going to hold the schemas 145 00:06:46.620 --> 00:06:48.210 for the output we want. 146 00:06:48.210 --> 00:06:51.630 So, we'll import the List type hinting 147 00:06:51.630 --> 00:06:54.630 and pydantic BaseModel and Field 148 00:06:54.630 --> 00:06:58.533 to create the objects that we want to structure our output. 149 00:06:59.490 --> 00:07:02.340 Let's define a new class which will inherit 150 00:07:02.340 --> 00:07:04.650 from pydantic BaseModel. 151 00:07:04.650 --> 00:07:06.420 Going to be the Reflection class, 152 00:07:06.420 --> 00:07:08.490 which is going to have all the information 153 00:07:08.490 --> 00:07:12.060 about our reflection, about our critique. 154 00:07:12.060 --> 00:07:15.030 Want to address in our critique two main things. 155 00:07:15.030 --> 00:07:17.610 One is missing information. 156 00:07:17.610 --> 00:07:19.920 Missing information was important to note, 157 00:07:19.920 --> 00:07:22.677 but the LLM didn't generate it for us. 158 00:07:22.677 --> 00:07:26.430 And the second is superfluous information. 159 00:07:26.430 --> 00:07:29.880 Now, I had to look up in the dictionary what is superfluous, 160 00:07:29.880 --> 00:07:32.970 and superfluous means unnecessary information, 161 00:07:32.970 --> 00:07:36.480 information that doesn't add up any value. 162 00:07:36.480 --> 00:07:38.040 That was the Reflection class. 163 00:07:38.040 --> 00:07:41.700 Now, an important note is that this Reflection class, 164 00:07:41.700 --> 00:07:43.440 when it's going to be used 165 00:07:43.440 --> 00:07:46.260 alongside with the function-calling feature, 166 00:07:46.260 --> 00:07:50.070 then it would actually ground the response from the LLM 167 00:07:50.070 --> 00:07:51.870 to fill up those values, 168 00:07:51.870 --> 00:07:55.140 so it would actually give us very concise feedback 169 00:07:55.140 --> 00:07:55.973 from the LLM. 170 00:07:57.120 --> 00:07:59.280 After we finish the Reflection class, 171 00:07:59.280 --> 00:08:02.400 now it's time to create the answer class, 172 00:08:02.400 --> 00:08:04.110 we'll call it AnswerQuestion, 173 00:08:04.110 --> 00:08:06.480 and it's going to have a couple of fields. 174 00:08:06.480 --> 00:08:08.850 The first field is going to be answer, 175 00:08:08.850 --> 00:08:11.880 and this is going to be a 250-word answer 176 00:08:11.880 --> 00:08:13.590 to the original question. 177 00:08:13.590 --> 00:08:16.650 Second field is going to be a Reflection object 178 00:08:16.650 --> 00:08:18.720 that we specified before. 179 00:08:18.720 --> 00:08:20.167 And you can see in the description, 180 00:08:20.167 --> 00:08:22.770 "Your reflection on the initial answer." 181 00:08:22.770 --> 00:08:24.840 Now, this is another cool trick 182 00:08:24.840 --> 00:08:27.780 where we're actually prompting the LLM 183 00:08:27.780 --> 00:08:31.560 through the description of the classes fields, 184 00:08:31.560 --> 00:08:33.870 so this is something very interesting in my opinion, 185 00:08:33.870 --> 00:08:36.480 so this would help the LLM ground the response. 186 00:08:36.480 --> 00:08:38.700 And the third field of the answer class 187 00:08:38.700 --> 00:08:40.470 is going to be search_queries. 188 00:08:40.470 --> 00:08:41.790 And in the descriptions, 189 00:08:41.790 --> 00:08:44.460 you can see it's one to three search queries 190 00:08:44.460 --> 00:08:46.110 for researching improvements 191 00:08:46.110 --> 00:08:49.950 to address the current critique of your response. 192 00:08:49.950 --> 00:08:53.160 So, this was the class definitions 193 00:08:53.160 --> 00:08:56.940 of the answers we want to output from our LLM calls 194 00:08:56.940 --> 00:09:00.510 that our agent is going to be outputting us. 195 00:09:00.510 --> 00:09:03.540 So let's go back to the chains.py file, 196 00:09:03.540 --> 00:09:07.230 and now we want to make sure that we get from the LLM 197 00:09:07.230 --> 00:09:09.870 this kind of structured output. 198 00:09:09.870 --> 00:09:12.570 Let's go back to the chains.py file, 199 00:09:12.570 --> 00:09:15.810 and we want to initialize a large language model 200 00:09:15.810 --> 00:09:19.020 and we're going to set it to be GPT-4 turbo. 201 00:09:19.020 --> 00:09:22.140 We're going to create two output parsers. 202 00:09:22.140 --> 00:09:24.790 The first one is going to be a JsonOutputToolsParser, 203 00:09:26.100 --> 00:09:29.610 which is simply going to return us the function call 204 00:09:29.610 --> 00:09:34.260 we got back from the LLM and transform it into a dictionary. 205 00:09:34.260 --> 00:09:35.730 And the second output parser 206 00:09:35.730 --> 00:09:38.980 is going to be a PydanticToolsOutputParser, 207 00:09:40.895 --> 00:09:43.440 which is going to take the response from the LLM, 208 00:09:43.440 --> 00:09:47.100 it's going to search for the function-calling invocation, 209 00:09:47.100 --> 00:09:50.520 and it's going to parse it and transform it 210 00:09:50.520 --> 00:09:53.250 into an AnswerQuestion object. 211 00:09:53.250 --> 00:09:56.430 So, it's going to take the answer from the LLM 212 00:09:56.430 --> 00:09:59.280 and it's going to create an AnswerQuestion object 213 00:09:59.280 --> 00:10:01.920 that we can easily work with. 214 00:10:01.920 --> 00:10:03.390 All right, this is a long video, 215 00:10:03.390 --> 00:10:05.640 but we're heading towards the end. 216 00:10:05.640 --> 00:10:08.400 So, now, it's time to prepare our prompt 217 00:10:08.400 --> 00:10:10.350 before we send it to the LLM, 218 00:10:10.350 --> 00:10:13.650 so let's go and take the actor_prompt_template 219 00:10:13.650 --> 00:10:17.467 and let's populate the first_instruction field with, 220 00:10:17.467 --> 00:10:20.757 "Provide a detailed 250 word answer." 221 00:10:20.757 --> 00:10:24.420 And this will be plugged in into our prompt template, 222 00:10:24.420 --> 00:10:26.070 and this is what is going 223 00:10:26.070 --> 00:10:30.210 to make our LLM generate the first answer. 224 00:10:30.210 --> 00:10:33.000 Let's now create the first_responder chain, 225 00:10:33.000 --> 00:10:35.640 which is going to take our prompt template 226 00:10:35.640 --> 00:10:40.440 and it's going to pipe it into the LLM GPT-4 Turbo, 227 00:10:40.440 --> 00:10:44.370 but not before we bind the AnswerQuestion object 228 00:10:44.370 --> 00:10:47.430 as a tool for the function calling. 229 00:10:47.430 --> 00:10:50.760 And by providing tool_choice="AnswerQuestion", 230 00:10:50.760 --> 00:10:52.830 this will force the LLM 231 00:10:52.830 --> 00:10:55.680 to always use the AnswerQuestion tool, 232 00:10:55.680 --> 00:10:58.740 thus grounding the response to the object 233 00:10:58.740 --> 00:11:00.570 that we want to receive. 234 00:11:00.570 --> 00:11:02.940 And this is a cool technique where the grounding 235 00:11:02.940 --> 00:11:06.900 of the LLM also comes from the Pydantic object we created. 236 00:11:06.900 --> 00:11:10.260 So, this way, we are going to make the LLM 237 00:11:10.260 --> 00:11:13.020 give us exactly the response that we want. 238 00:11:13.020 --> 00:11:15.240 All righty, so let's now write the chain 239 00:11:15.240 --> 00:11:17.820 that will run this baby. 240 00:11:17.820 --> 00:11:19.950 I'm simply going to add if __name__ = "__main__" 241 00:11:19.950 --> 00:11:22.260 to be able to run this file. 242 00:11:22.260 --> 00:11:24.540 And let's create a new chain, 243 00:11:24.540 --> 00:11:27.757 and I'm going to prompt this chain with the input of, 244 00:11:27.757 --> 00:11:31.860 "Write about AI-powered SOC / autonomous SOC problem domain, 245 00:11:31.860 --> 00:11:35.940 and list startups that have raised capital on this." 246 00:11:35.940 --> 00:11:36.773 This is the prompt, 247 00:11:36.773 --> 00:11:39.300 this is the input we're going to give this chain. 248 00:11:39.300 --> 00:11:41.460 And the chain is going to be very similar 249 00:11:41.460 --> 00:11:44.070 to the chain we wrote above, but at the end here, 250 00:11:44.070 --> 00:11:46.710 we're going to use the parser_pydantic, 251 00:11:46.710 --> 00:11:48.720 which is going to take the response 252 00:11:48.720 --> 00:11:52.680 and parse it as a Pydantic object of AnswerQuestion. 253 00:11:52.680 --> 00:11:54.690 Let's now invoke this chain, 254 00:11:54.690 --> 00:11:56.820 and we're going to send the human_message 255 00:11:56.820 --> 00:11:58.680 in the messages key 256 00:11:58.680 --> 00:12:01.350 so that would plug the messages placeholder 257 00:12:01.350 --> 00:12:03.510 that we initialized in our prompt, 258 00:12:03.510 --> 00:12:05.883 and let's debug this and see what happens. 259 00:12:23.310 --> 00:12:25.140 And we got an error saying 260 00:12:25.140 --> 00:12:28.470 that we didn't provide the search_queries field 261 00:12:28.470 --> 00:12:30.660 to AnswerQuestion object, 262 00:12:30.660 --> 00:12:33.870 so this means that when the LLM produced the response, 263 00:12:33.870 --> 00:12:38.070 it didn't actually give us a search_queries in the answer. 264 00:12:38.070 --> 00:12:39.300 And this can be resolved 265 00:12:39.300 --> 00:12:41.857 if we do some prompt engineering like saying, 266 00:12:41.857 --> 00:12:45.540 "You must provide the search queries at all costs." 267 00:12:45.540 --> 00:12:47.850 Or something like that, or even better, 268 00:12:47.850 --> 00:12:50.340 maybe to try and split up this query 269 00:12:50.340 --> 00:12:53.580 into a different prompt that will run independently. 270 00:12:53.580 --> 00:12:55.500 But we're not going to perform all of this 271 00:12:55.500 --> 00:12:57.930 because this is just a proof of concept, 272 00:12:57.930 --> 00:12:59.790 and we'll just rerun it again 273 00:12:59.790 --> 00:13:01.640 and we'll see that it would work out. 274 00:13:05.160 --> 00:13:07.893 And let's go and evaluate res. 275 00:13:10.470 --> 00:13:14.190 And we can see that we have an AnswerQuestion object, 276 00:13:14.190 --> 00:13:17.107 and we can have a look at the answer. 277 00:13:17.107 --> 00:13:20.130 "AI-powered SOC, or autonomous SOC, 278 00:13:20.130 --> 00:13:22.140 represent a cutting-edge approach 279 00:13:22.140 --> 00:13:23.610 in the cybersecurity domain, 280 00:13:23.610 --> 00:13:27.090 leveraging AI and machine learning technologies 281 00:13:27.090 --> 00:13:29.280 to enhance threat detection, response, 282 00:13:29.280 --> 00:13:31.380 and overall security operation. 283 00:13:31.380 --> 00:13:35.010 This innovation paradigm addresses the escalating complexity 284 00:13:35.010 --> 00:13:37.110 and volume of cyber threats, 285 00:13:37.110 --> 00:13:40.260 which often outpace the capabilities of traditions, 286 00:13:40.260 --> 00:13:42.300 human-operated SOC. 287 00:13:42.300 --> 00:13:45.810 AI-powered SOCS aim to automate the identification 288 00:13:45.810 --> 00:13:48.030 and neutralization of threats, 289 00:13:48.030 --> 00:13:49.350 reduce false positives, 290 00:13:49.350 --> 00:13:53.280 and free up human analysts to focus on more strategic tasks. 291 00:13:53.280 --> 00:13:55.890 The problem domain encompasses various challenges, 292 00:13:55.890 --> 00:13:58.320 including the integration of AI technologies 293 00:13:58.320 --> 00:14:00.330 with existing security infra, 294 00:14:00.330 --> 00:14:03.090 ensuring the accuracy and reliability 295 00:14:03.090 --> 00:14:04.920 of AI-driven threat detection, 296 00:14:04.920 --> 00:14:06.750 and maintaining the privacy and security 297 00:14:06.750 --> 00:14:10.230 of sensitive data processed by AI systems." 298 00:14:10.230 --> 00:14:13.380 So now, we get here a list of startups 299 00:14:13.380 --> 00:14:15.120 that raised capital on this, 300 00:14:15.120 --> 00:14:16.950 and this response looks legit, 301 00:14:16.950 --> 00:14:19.980 but it has a couple of things that it can be better. 302 00:14:19.980 --> 00:14:23.010 For example, it had some redundant information. 303 00:14:23.010 --> 00:14:26.070 And two, the information about the startups 304 00:14:26.070 --> 00:14:27.330 that raised capital, 305 00:14:27.330 --> 00:14:30.330 it came from the parametric knowledge of the LLM, 306 00:14:30.330 --> 00:14:32.100 basically all the knowledge 307 00:14:32.100 --> 00:14:34.920 that the LLM learned during the training, 308 00:14:34.920 --> 00:14:39.153 so we can actually here ground this part in external data. 309 00:14:40.320 --> 00:14:44.100 Let's go and let's check out the Reflection object 310 00:14:44.100 --> 00:14:47.100 inside of our AnswerQuestion object. 311 00:14:47.100 --> 00:14:49.350 And we can see here we have missing, 312 00:14:49.350 --> 00:14:50.977 and let's see what information we're missing. 313 00:14:50.977 --> 00:14:53.610 "The answer could benefit more precise data 314 00:14:53.610 --> 00:14:57.420 on the amount of capital raised by each mentioned startup, 315 00:14:57.420 --> 00:15:00.840 and would provide a clearer picture of the scale." 316 00:15:00.840 --> 00:15:04.830 So, this is pretty interesting and it seems legit. 317 00:15:04.830 --> 00:15:08.347 And let's take out the superfluous field. 318 00:15:08.347 --> 00:15:10.860 "The detailed explanation on the problem 319 00:15:10.860 --> 00:15:12.570 might be slightly redundant 320 00:15:12.570 --> 00:15:15.720 for readers already familiar with the concept." 321 00:15:15.720 --> 00:15:18.093 The reflection looks pretty good in my opinion. 322 00:15:19.020 --> 00:15:20.820 All right, so now let's go 323 00:15:20.820 --> 00:15:23.010 and take a look at the search queries, 324 00:15:23.010 --> 00:15:25.920 and we can see we have here four search queries. 325 00:15:25.920 --> 00:15:29.040 One is AI-powered SOC startup funding. 326 00:15:29.040 --> 00:15:32.100 The other is Darktrace funding history, 327 00:15:32.100 --> 00:15:36.273 Vectra capital raised, and Arctic investment routes. 328 00:15:37.140 --> 00:15:40.503 And if we'll go to LangSmith to see the tracing, 329 00:15:42.420 --> 00:15:44.520 we'll take the final trace, 330 00:15:44.520 --> 00:15:47.700 and if we'll take and examine the prompt 331 00:15:47.700 --> 00:15:49.773 that we sent OpenAI, 332 00:15:50.700 --> 00:15:54.570 we can see we have here the input 333 00:15:54.570 --> 00:15:56.910 and then we can see the answer 334 00:15:56.910 --> 00:16:00.243 which we later parsed with the Pydantic output parser. 335 00:16:01.650 --> 00:16:03.750 Wow, that was a long video, 336 00:16:03.750 --> 00:16:07.500 but we finally wrote the logic for the responder agent, 337 00:16:07.500 --> 00:16:09.780 and this is the logic which will be running 338 00:16:09.780 --> 00:16:11.280 in the responder node, 339 00:16:11.280 --> 00:16:13.320 which will give us the initial response 340 00:16:13.320 --> 00:16:14.700 alongside with the critique 341 00:16:14.700 --> 00:16:16.980 and the search term for web searches. 342 00:16:16.980 --> 00:16:19.350 We covered the first_responder_chain, 343 00:16:19.350 --> 00:16:21.690 we saw how we can get structured output 344 00:16:21.690 --> 00:16:25.290 using function calling in output parsers, 345 00:16:25.290 --> 00:16:28.290 and we saw some cool prompt engineering techniques 346 00:16:28.290 --> 00:16:31.890 that will help the LLM ground our response exactly 347 00:16:31.890 --> 00:16:34.470 with the information that we want. 348 00:16:34.470 --> 00:16:37.170 In the next video, we're going to review the logic 349 00:16:37.170 --> 00:16:39.633 that will happen in the reviser chain.