WEBVTT 1 00:00:00.000 --> 00:00:02.730 Now, there's one last setting 2 00:00:02.730 --> 00:00:05.130 which you can access as a power user, 3 00:00:05.130 --> 00:00:07.260 which I want to show you, 4 00:00:07.260 --> 00:00:10.950 and that's this structured output setting. 5 00:00:10.950 --> 00:00:15.210 And the idea here is that you can kind of force the model 6 00:00:15.210 --> 00:00:19.470 to give you this output in a structured format, 7 00:00:19.470 --> 00:00:22.350 in JSON format, specifically. 8 00:00:22.350 --> 00:00:25.470 Now, JSON, in case you don't know, is simply a format, 9 00:00:25.470 --> 00:00:28.560 a data format that looks like this. 10 00:00:28.560 --> 00:00:32.820 This here is JSON. That's an example for a JSON document. 11 00:00:32.820 --> 00:00:37.820 It, in the end, is text that's formatted in a certain way. 12 00:00:38.220 --> 00:00:40.200 And having a clearly defined format 13 00:00:40.200 --> 00:00:43.320 and having text that follows a certain format, 14 00:00:43.320 --> 00:00:46.050 having data that's formatted in a certain way, 15 00:00:46.050 --> 00:00:49.740 therefore, is of course very useful if you, for example, 16 00:00:49.740 --> 00:00:53.070 plan on using this output as an input 17 00:00:53.070 --> 00:00:57.690 in another application or another step of an application. 18 00:00:57.690 --> 00:01:01.980 That's why it's quite common that you want LLMs 19 00:01:01.980 --> 00:01:05.370 to produce JSON output, formatted output, 20 00:01:05.370 --> 00:01:08.310 so that you can use the output of an LLM, 21 00:01:08.310 --> 00:01:10.620 not just as a human, but instead, for example, 22 00:01:10.620 --> 00:01:12.720 also in another application. 23 00:01:12.720 --> 00:01:15.810 For example, here, I might wanna extract 24 00:01:15.810 --> 00:01:20.810 those key financials not as text as is the case here, 25 00:01:21.060 --> 00:01:24.900 but instead, I might want to extract the key financials 26 00:01:24.900 --> 00:01:27.180 in a structured format. 27 00:01:27.180 --> 00:01:29.310 And to get the model to do that, 28 00:01:29.310 --> 00:01:31.260 you can go to the model settings here, 29 00:01:31.260 --> 00:01:35.490 turn on Structured Output, and then you have to define 30 00:01:35.490 --> 00:01:38.133 a JSON schema here in this box. 31 00:01:39.060 --> 00:01:40.290 Now you can visit this page 32 00:01:40.290 --> 00:01:42.540 to learn how a JSON schema looks like. 33 00:01:42.540 --> 00:01:45.219 If you scroll down, you will see an example, 34 00:01:45.219 --> 00:01:48.900 and you see that it is itself a JSON document 35 00:01:48.900 --> 00:01:50.580 that describes the structure 36 00:01:50.580 --> 00:01:53.820 of the to-be-generated JSON document. 37 00:01:53.820 --> 00:01:57.120 So for example, here we can copy this example 38 00:01:57.120 --> 00:01:58.950 and paste it in here. 39 00:01:58.950 --> 00:02:01.140 Now it's a bit hard to edit here. 40 00:02:01.140 --> 00:02:04.020 Later, we will also see how we could do this in code, 41 00:02:04.020 --> 00:02:06.450 but for now, this will have to do. 42 00:02:06.450 --> 00:02:09.270 Now, the start here can all stay the way it is, 43 00:02:09.270 --> 00:02:10.710 the ID, the schema. 44 00:02:10.710 --> 00:02:14.793 Now for the title, I'll say, Financial Data. 45 00:02:15.870 --> 00:02:17.670 The type should then be object. 46 00:02:17.670 --> 00:02:19.290 You need to have a properties key, 47 00:02:19.290 --> 00:02:22.560 and then you describe the shape of your data, so to say. 48 00:02:22.560 --> 00:02:25.683 For example here, I want to have a year field, 49 00:02:26.550 --> 00:02:29.790 which is of type string, which means text, 50 00:02:29.790 --> 00:02:32.430 and I describe what will be in this field. 51 00:02:32.430 --> 00:02:33.900 And this description is important, 52 00:02:33.900 --> 00:02:35.130 because it tells the model 53 00:02:35.130 --> 00:02:38.400 which kind of data to put into this field. 54 00:02:38.400 --> 00:02:41.343 So here, I'll say the year of the report. 55 00:02:42.990 --> 00:02:46.380 Then I want to have another field, which could be revenue. 56 00:02:46.380 --> 00:02:50.790 And here, maybe the type should be number. 57 00:02:50.790 --> 00:02:53.550 And the description here is the total revenue 58 00:02:53.550 --> 00:02:57.720 of that year or of the year. 59 00:02:59.550 --> 00:03:01.650 And then here, I want to have another field, 60 00:03:01.650 --> 00:03:03.850 let's say operating_income, 61 00:03:06.570 --> 00:03:10.950 and here the type is integer, sorry, 62 00:03:10.950 --> 00:03:13.050 that should also be integer here for the revenue, 63 00:03:13.050 --> 00:03:15.990 not number, that's a specific kind of number, 64 00:03:15.990 --> 00:03:18.900 a number without decimal places. 65 00:03:18.900 --> 00:03:21.330 So that's the revenue and also the operating income. 66 00:03:21.330 --> 00:03:23.610 And it doesn't matter whether the type comes first 67 00:03:23.610 --> 00:03:24.930 or the description. 68 00:03:24.930 --> 00:03:27.300 So here, the description then is 69 00:03:27.300 --> 00:03:30.033 operating income for the year. 70 00:03:30.990 --> 00:03:32.820 And you could have other requirements 71 00:03:32.820 --> 00:03:35.700 like a minimum value of zero, but I don't care about that, 72 00:03:35.700 --> 00:03:38.433 so I'll remove it and remove that trailing comma. 73 00:03:39.840 --> 00:03:42.300 And you could add more to extract more data, 74 00:03:42.300 --> 00:03:46.170 but I will just specify these three fields, 75 00:03:46.170 --> 00:03:48.930 year, revenue, operating income, 76 00:03:48.930 --> 00:03:50.703 that's all, that's all I want. 77 00:03:51.570 --> 00:03:54.150 And now, we could set this year 78 00:03:54.150 --> 00:03:55.950 by having this check mark enabled 79 00:03:55.950 --> 00:03:58.683 and then go back here to our prompt and say, 80 00:03:59.790 --> 00:04:03.183 Return the key data as JSON. 81 00:04:04.980 --> 00:04:09.120 And that prompt combined with this structured output setting 82 00:04:09.120 --> 00:04:12.030 that's set for this chat behind the scenes, 83 00:04:12.030 --> 00:04:15.810 that should get the model to hopefully return data 84 00:04:15.810 --> 00:04:17.400 in that JSON format, 85 00:04:17.400 --> 00:04:21.150 a JSON format that follows this structure. 86 00:04:21.150 --> 00:04:22.680 Now, as you can see, 87 00:04:22.680 --> 00:04:25.230 the results I got here don't look that great. 88 00:04:25.230 --> 00:04:26.880 Not only is the year missing, 89 00:04:26.880 --> 00:04:30.870 the numbers also look a bit off, to be honest. 90 00:04:30.870 --> 00:04:33.150 And therefore, what I recommend 91 00:04:33.150 --> 00:04:37.560 when working with structured output is that in your prompt, 92 00:04:37.560 --> 00:04:42.560 you also repeat the values you want the model to extract 93 00:04:42.840 --> 00:04:45.270 so that you increase the likelihood 94 00:04:45.270 --> 00:04:48.180 of it picking the right values. 95 00:04:48.180 --> 00:04:50.370 That's the case with all models 96 00:04:50.370 --> 00:04:52.890 whenever you are working with structured outputs, 97 00:04:52.890 --> 00:04:56.130 not just when working with open models. 98 00:04:56.130 --> 00:04:58.740 So structured output is not a strong guarantee 99 00:04:58.740 --> 00:05:01.110 that you will definitely get the data 100 00:05:01.110 --> 00:05:03.750 in the format you requested, 101 00:05:03.750 --> 00:05:06.660 but it makes it way, way more likely. 102 00:05:06.660 --> 00:05:09.180 And combined with a strong prompt 103 00:05:09.180 --> 00:05:13.350 that asks for JSON and maybe specifies the key data 104 00:05:13.350 --> 00:05:14.970 you're interested in again, 105 00:05:14.970 --> 00:05:16.710 which of course has to be in line 106 00:05:16.710 --> 00:05:19.080 with your structured output schema here, 107 00:05:19.080 --> 00:05:20.880 it is pretty much guaranteed 108 00:05:20.880 --> 00:05:23.490 that you will get back a result that is text, 109 00:05:23.490 --> 00:05:26.973 that is formatted according to your JSON schema here.