WEBVTT 00:00.020 --> 00:05.930 Earlier on in the course, we looked at how you could use the Structured Outputs API from OpenAI to 00:05.960 --> 00:11.690 get structured outputs from both the response format and also using function calling or tools. 00:11.690 --> 00:16.100 In this video, we're going to have a look at how you can do that inside of Lang chain using this concept 00:16.100 --> 00:17.300 called an output parser. 00:17.330 --> 00:21.860 Now, when you're working with large language models, you'll often find that you don't want to return 00:21.860 --> 00:23.330 data such as text. 00:23.360 --> 00:25.250 You'll want to return structured data. 00:25.250 --> 00:31.220 We can do this inside of Lang chain by using an output parser, which generally in Python will use a 00:31.220 --> 00:36.710 Pydantic model, but in TypeScript or JavaScript it will use a Zod package model as well. 00:36.710 --> 00:38.600 So let's have a look at how we can do that. 00:38.600 --> 00:43.070 Firstly, we install all of these packages and we would set up the OpenAI key here. 00:43.100 --> 00:47.690 Then what we'll do is we'll then go down here and we'll then import a couple of different things. 00:47.690 --> 00:49.400 So we'll import the chat model. 00:49.400 --> 00:54.170 We will also import the Lang chain call prompts. 00:54.200 --> 00:57.320 We're going to import a chat prompt template that we saw before. 00:57.320 --> 01:00.170 And also a system message prompt template. 01:00.410 --> 01:04.100 Now we're also going to import a Pydantic output parser. 01:04.100 --> 01:09.490 So I'm going to do from lang chain core dot output passes, we're going to import the pedantic output 01:09.490 --> 01:10.330 parser. 01:10.780 --> 01:16.330 And then what we're going to do is we're then going to also add in pedantic import base model. 01:17.050 --> 01:18.190 We're going to have the field. 01:18.190 --> 01:22.090 We're going to set up our chat model with model is equal to chat OpenAI. 01:22.600 --> 01:27.610 After we've set up our chat model, we're then going to also add in one more package which is from typing 01:27.790 --> 01:29.530 import list. 01:30.190 --> 01:34.480 And I'm going to place this at the top of my imports. 01:35.800 --> 01:39.670 Then what we're going to do is we're going to set up two different types of pedantic models. 01:39.670 --> 01:45.130 One for a joke, which is a pedantic class of joke with a base model. 01:45.130 --> 01:48.970 And notice we're inheriting from the pedantic base model class. 01:48.970 --> 01:56.530 We're then going to have a setup, which is a setup to the joke and a punchline, and we're also going 01:56.530 --> 02:01.420 to create a different one, which is a jokes base model. 02:01.630 --> 02:06.970 And the jokes base model will just take a key of jokes, and it will return a list of joke. 02:07.240 --> 02:09.970 And the field description is a list of jokes. 02:09.970 --> 02:16.730 We then have our output parser that you can see which is currently set to output. 02:16.730 --> 02:17.930 Pass a joke. 02:17.930 --> 02:19.340 Pedantic object. 02:19.580 --> 02:26.720 Then what we need to do is set up a template so I can say template is equal to answer the user query. 02:29.990 --> 02:34.040 And we're going to add in some format instructions that we'll see in a second. 02:34.040 --> 02:36.650 So we'll put format instructions. 02:38.030 --> 02:40.370 And then we're going to put a new line. 02:40.370 --> 02:41.750 And then we'll put the query. 02:43.190 --> 02:45.620 And we're going to set up a system message prompt template. 02:45.620 --> 02:49.100 And we'll do that and call it system message prompt. 02:49.100 --> 02:54.050 And this is slightly different to the one that you saw before where we're doing this kind of format 02:54.050 --> 02:55.880 where we have a from template. 02:55.880 --> 02:58.010 And I'm going to also get rid of this one. 02:58.010 --> 02:59.600 And we'll just have a chat prompt. 02:59.630 --> 03:00.380 Template. 03:00.410 --> 03:07.250 Chat prompt is equal to chat prompt template dot from messages. 03:07.400 --> 03:08.840 And we're going to put in the system prompt. 03:08.840 --> 03:11.450 They're going to get rid of this. 03:11.450 --> 03:14.330 And then what you'll see here is we've got our template set up. 03:14.330 --> 03:18.650 And this system message prompt template will basically use a template string. 03:18.650 --> 03:23.470 And then when we invoke that, we're going to fill in both the format instructions variable and also 03:23.500 --> 03:24.580 the query variable. 03:24.670 --> 03:31.540 Now just to show you how that would look, we're going to then say messages is equal to chat prompt 03:31.660 --> 03:33.040 dot invoke. 03:33.070 --> 03:35.140 And here we're going to fill in these two things. 03:35.140 --> 03:37.180 So we're going to have both the query what is. 03:37.330 --> 03:45.010 We'll say what is a really funny joke about Python programming. 03:45.610 --> 03:47.890 And then we'll also have the format instructions. 03:47.890 --> 03:50.380 Now the format instructions are coming from Lang Chain. 03:50.380 --> 03:52.900 And we'll just have a look at what the messages is. 03:52.900 --> 03:58.420 So you'll see now that we've invoked that and we've saved this to a messages which has the prompt value, 03:58.450 --> 04:06.070 you can see the actual format instructions gives a real good indication as to how you specifically want 04:06.070 --> 04:07.750 to extract structured data. 04:07.750 --> 04:14.650 This is at the moment just using the prompting version with a really good description of what to extract. 04:14.650 --> 04:16.210 And then we've got our query at the end. 04:16.240 --> 04:19.630 Here you can see what is a really funny joke about Python programming. 04:19.630 --> 04:26.010 So this is purely using prompting techniques, which is how you originally had to do it when there wasn't 04:26.010 --> 04:28.800 anything like tools and structured output APIs. 04:28.800 --> 04:36.240 So now that we've got the messages formatted, what we can then do is do model dot invoke messages, 04:36.240 --> 04:38.850 and then we can have a look at the result. 04:39.180 --> 04:44.730 And if we look at this result and we look at the content of this result, you can see that it has actually 04:44.730 --> 04:48.120 produced a JSON string that we could then pass. 04:48.120 --> 04:54.720 So if we give this to the output parser, what we can then do is we can go and try and do this with 04:54.720 --> 04:57.780 a parser.parse and actually storing that. 04:57.780 --> 05:00.690 Let's try and for example store that. 05:00.690 --> 05:04.920 So we'll have joke object and we'll store this. 05:06.480 --> 05:13.980 And now what you can see here is we're actually able to extract a structured joke object directly from 05:13.980 --> 05:15.090 the result content. 05:15.090 --> 05:21.030 So purely just from prompting and adding in this parser dot get format instructions, we can already 05:21.030 --> 05:24.150 start to get structured data out directly from the LLM. 05:24.150 --> 05:25.740 And that's this format. 05:25.740 --> 05:30.360 Instructions is originally how you had to do it when you were just using prompting techniques. 05:30.360 --> 05:36.230 If we have a look, we can then do, for example, the joke object and the joke object punchline. 05:36.230 --> 05:40.220 So this is how you can get structured data out using an output parser. 05:40.250 --> 05:45.080 Now if we're having a look at the documentation for Lang chain, they have a better way of doing this. 05:45.080 --> 05:51.020 Now with the dot with structured output that will actually use a model's ability to use something called 05:51.020 --> 05:55.970 JSON mode, and JSON mode will give much more reliable results. 05:55.970 --> 06:01.220 Rather than simply relying on prompting techniques, we can use the Pydantic models that we've been 06:01.220 --> 06:01.850 doing before. 06:01.880 --> 06:09.140 The only change we have to make is that when we make our LLM, we also then make a structured LLM using 06:09.140 --> 06:13.880 this dot with structured output passing in the model that we previously had. 06:13.880 --> 06:15.980 And then we can invoke that directly. 06:15.980 --> 06:17.870 Let's have a look at how we can do that. 06:17.870 --> 06:22.040 Firstly we're going to create a new cell and we're going to call it structured LLM. 06:22.370 --> 06:28.220 And then we're going to just basically have our model dot with structured output. 06:28.910 --> 06:31.760 The first thing that you're going to put in is the schema. 06:31.790 --> 06:35.240 This can either be an OpenAI function tool slash schema. 06:35.270 --> 06:40.110 It can be a JSON schema a are typed dict class or a pedantic class. 06:40.110 --> 06:43.080 Because we're using pedantic, we're just going to put the joke in here. 06:43.080 --> 06:46.320 And then as well as that, if you want, you can also change the method. 06:46.320 --> 06:49.830 So you can see you can either do function calling or JSON mode. 06:49.950 --> 06:51.990 Let's leave the method out for now. 06:52.020 --> 06:57.720 Now we also just have to do result is equal to structured lm dot invoke. 07:00.420 --> 07:04.050 And then we can say what is a really funny joke about Python programming. 07:04.050 --> 07:06.030 And then if we look at the result. 07:08.880 --> 07:16.470 And we look at the type of that result, you'll see we have a main joke pedantic object. 07:16.470 --> 07:18.360 And we have both a setup key. 07:18.360 --> 07:20.160 So I can do result dot setup. 07:20.340 --> 07:23.340 And I can also do results dot punchline. 07:24.480 --> 07:25.320 So that's it. 07:25.320 --> 07:27.660 That's how you do output parsing using lang chain. 07:27.690 --> 07:32.130 Going forward I would recommend using the model with structured output. 07:32.130 --> 07:38.220 The only reason why you would actually use the one before is if the large language model, or the model 07:38.220 --> 07:44.010 that you're using doesn't support JSON mode or structured outputs via function calling.