WEBVTT 1 00:00:00.480 --> 00:00:01.860 So let's see 2 00:00:01.860 --> 00:00:04.320 the fact of these different settings, 3 00:00:04.320 --> 00:00:07.530 and I will start with temperature. 4 00:00:07.530 --> 00:00:10.470 As mentioned, a low temperature value 5 00:00:10.470 --> 00:00:12.240 exaggerates differences, 6 00:00:12.240 --> 00:00:15.270 hence making the more likely candidates 7 00:00:15.270 --> 00:00:17.670 and tokens even more likely. 8 00:00:17.670 --> 00:00:19.140 High temperature values, 9 00:00:19.140 --> 00:00:21.150 on the other hand, flatten differences, 10 00:00:21.150 --> 00:00:23.070 hence leaving to more diverse 11 00:00:23.070 --> 00:00:26.040 and potentially creative output. 12 00:00:26.040 --> 00:00:28.260 So if I open a new chat 13 00:00:28.260 --> 00:00:30.900 and I set my Temperature to 0 14 00:00:30.900 --> 00:00:34.170 to exaggerate all probabilities 15 00:00:34.170 --> 00:00:36.420 to the point where I'm guaranteed 16 00:00:36.420 --> 00:00:39.750 to always pick the most likely candidate 17 00:00:39.750 --> 00:00:42.030 because that is what happens with 0. 18 00:00:42.030 --> 00:00:45.690 You got no randomness going on with 0. 19 00:00:45.690 --> 00:00:47.130 As you also see the tooltip, 20 00:00:47.130 --> 00:00:49.890 0 will yield the same result every time. 21 00:00:49.890 --> 00:00:51.870 So if I set this to 0, 22 00:00:51.870 --> 00:00:54.937 and I then send a message like, 23 00:00:54.937 --> 00:00:59.610 "The sky is blank space, 24 00:00:59.610 --> 00:01:01.170 and I hit Enter, 25 00:01:01.170 --> 00:01:04.500 I get blue most of the time and some text. 26 00:01:04.500 --> 00:01:07.350 Now, I'll copy my prompt, 27 00:01:07.350 --> 00:01:09.330 open a new chat, paste it in, 28 00:01:09.330 --> 00:01:11.883 and I still have Temperature set to 0. 29 00:01:13.290 --> 00:01:16.110 And of course, since that always gives us the same output, 30 00:01:16.110 --> 00:01:19.920 as I mentioned, we get exactly the same output as before. 31 00:01:19.920 --> 00:01:22.050 So if I compare these chats, 32 00:01:22.050 --> 00:01:24.630 we see these options are the same 33 00:01:24.630 --> 00:01:27.720 and all the words are exactly the same. 34 00:01:27.720 --> 00:01:30.810 The only thing that differs is the speed, so to say, 35 00:01:30.810 --> 00:01:32.673 but the text is exactly the same. 36 00:01:33.870 --> 00:01:35.400 Now I'll open yet another chat, 37 00:01:35.400 --> 00:01:37.260 and I'll ramp this up to 1 38 00:01:37.260 --> 00:01:39.420 and send that same prompt. 39 00:01:39.420 --> 00:01:41.070 Now it is a very simple prompt, 40 00:01:41.070 --> 00:01:44.400 so the output is not expected to be totally different, 41 00:01:44.400 --> 00:01:46.953 but it should also not be exactly the same. 42 00:01:47.970 --> 00:01:50.910 And indeed, whilst it may look similar, 43 00:01:50.910 --> 00:01:53.490 you can already see it has more tokens, 44 00:01:53.490 --> 00:01:54.510 so it is longer. 45 00:01:54.510 --> 00:01:55.980 And if you compare it side by side, 46 00:01:55.980 --> 00:01:58.110 or well, one after another, I guess, 47 00:01:58.110 --> 00:01:59.790 you see it is different text. 48 00:01:59.790 --> 00:02:02.730 Like this text here, for example, is different. 49 00:02:02.730 --> 00:02:04.170 It's way shorter here, 50 00:02:04.170 --> 00:02:05.850 and the options are also different. 51 00:02:05.850 --> 00:02:08.190 Here, it's Orange/Red/Pink/Purple. 52 00:02:08.190 --> 00:02:10.563 Here, it's just Orange/Pink/Purple. 53 00:02:11.580 --> 00:02:14.460 And some other text is also different. 54 00:02:14.460 --> 00:02:16.980 So that's the Temperature value. 55 00:02:16.980 --> 00:02:18.510 You can always go back to the default 56 00:02:18.510 --> 00:02:22.140 by clicking this trash bin, the Reset button, which is .1, 57 00:02:22.140 --> 00:02:25.530 so quite predictable with not a lot of randomness. 58 00:02:25.530 --> 00:02:26.370 And you can change this 59 00:02:26.370 --> 00:02:28.720 and also save it to your preset if you want to. 60 00:02:29.880 --> 00:02:33.480 Now for the sampling here we got Top K, Top P, and Min P. 61 00:02:33.480 --> 00:02:36.420 The Repeat Penalty, as the name suggests, 62 00:02:36.420 --> 00:02:39.690 simply controls how much the model is discouraged 63 00:02:39.690 --> 00:02:41.520 from repeating the same token, 64 00:02:41.520 --> 00:02:44.340 and typically, the default setting here is fine. 65 00:02:44.340 --> 00:02:47.043 It is a value I actually never change. 66 00:02:47.880 --> 00:02:50.010 Now if I open a new chat here, 67 00:02:50.010 --> 00:02:53.160 I will try a Top K Sampling of 1. 68 00:02:53.160 --> 00:02:55.770 This extreme value should again ensure 69 00:02:55.770 --> 00:02:58.620 that we always get pretty much the same output 70 00:02:58.620 --> 00:03:01.713 because we always pick the most likely token. 71 00:03:02.640 --> 00:03:05.487 So if I paste in, "The sky is," 72 00:03:06.330 --> 00:03:08.070 I get some text here, 73 00:03:08.070 --> 00:03:12.000 and if I open a new chat also with Top K set to 1, 74 00:03:12.000 --> 00:03:14.850 but the Temperature's still at .1, 75 00:03:14.850 --> 00:03:16.383 and I sent the same prompt, 76 00:03:18.420 --> 00:03:21.360 you see if I compare them again, 77 00:03:21.360 --> 00:03:22.593 it's the same text. 78 00:03:23.640 --> 00:03:25.560 Now if I use a more realistic 79 00:03:25.560 --> 00:03:27.720 or more common value, like 40, 80 00:03:27.720 --> 00:03:31.410 so that the top 40 tokens are considered, 81 00:03:31.410 --> 00:03:34.410 you will see that we don't necessarily get the same result. 82 00:03:34.410 --> 00:03:36.510 Now since the Temperature is quite low, 83 00:03:36.510 --> 00:03:37.890 it's not unlikely 84 00:03:37.890 --> 00:03:39.993 that we get the same result actually. 85 00:03:40.890 --> 00:03:44.160 As you see here, it is actually the same, 86 00:03:44.160 --> 00:03:45.930 but there could be some differences. 87 00:03:45.930 --> 00:03:47.760 And if I were to ramp up the temperature 88 00:03:47.760 --> 00:03:50.580 as I did before with Top K at two 40, 89 00:03:50.580 --> 00:03:52.353 we will get different results. 90 00:03:53.610 --> 00:03:58.610 Now, Top P Sampling set to .95 simply means 91 00:03:58.740 --> 00:04:00.750 that we consider all candidates 92 00:04:00.750 --> 00:04:05.490 that combined have more than a 95% likelihood, 93 00:04:05.490 --> 00:04:07.320 up to 40 candidates. 94 00:04:07.320 --> 00:04:10.020 If you go above 40, we cut that, 95 00:04:10.020 --> 00:04:12.390 and we only consider the top most 40. 96 00:04:12.390 --> 00:04:15.360 But in general, we include all candidates 97 00:04:15.360 --> 00:04:19.500 that make up 59% in overall likelihood, 98 00:04:19.500 --> 00:04:22.470 and we discard all candidates 99 00:04:22.470 --> 00:04:26.400 that don't have at least 5% probability. 100 00:04:26.400 --> 00:04:29.730 That's what's going on here with that all reset, 101 00:04:29.730 --> 00:04:31.650 but the default was 40 anyways, 102 00:04:31.650 --> 00:04:32.580 and that's, therefore, 103 00:04:32.580 --> 00:04:35.400 how you can configure the output generation 104 00:04:35.400 --> 00:04:36.510 and the likelihoods. 105 00:04:36.510 --> 00:04:38.970 As mentioned, it's probably not something 106 00:04:38.970 --> 00:04:40.710 you're going to be doing all the time, 107 00:04:40.710 --> 00:04:43.143 but if you need to, the setting's there.