WEBVTT

1
00:00:00.450 --> 00:00:03.600
<v Maximilian>Now, regarding that programmatic usage</v>

2
00:00:03.600 --> 00:00:06.330
of these locally running AI models,

3
00:00:06.330 --> 00:00:07.830
it's also worth noting

4
00:00:07.830 --> 00:00:11.220
that LM Studio also offers another API

5
00:00:11.220 --> 00:00:15.570
that has different endpoints than the OpenAI compatible one,

6
00:00:15.570 --> 00:00:18.810
which I was using in these examples I showed you.

7
00:00:18.810 --> 00:00:22.770
So, you could also communicate with that LM Studio AI server

8
00:00:22.770 --> 00:00:26.010
through that API and you would talk to the same models.

9
00:00:26.010 --> 00:00:28.290
The API just has a different shape

10
00:00:28.290 --> 00:00:30.780
and accepts parameters in a different shape,

11
00:00:30.780 --> 00:00:34.170
and you can learn more about that here in the docs.

12
00:00:34.170 --> 00:00:36.420
And it's worth noting that you can, for example,

13
00:00:36.420 --> 00:00:39.600
also request structured outputs by, again,

14
00:00:39.600 --> 00:00:43.950
sending such a JSON schema to the API

15
00:00:43.950 --> 00:00:47.790
because you can configure your requests.

16
00:00:47.790 --> 00:00:50.940
For example, when using the OpenAI SDK here,

17
00:00:50.940 --> 00:00:54.630
you can set a response_format property

18
00:00:54.630 --> 00:00:57.120
and set this to some JSON schema

19
00:00:57.120 --> 00:01:01.260
that describes the shape of the data in JSON format

20
00:01:01.260 --> 00:01:03.300
that you would like to get.

21
00:01:03.300 --> 00:01:05.280
You can also set other parameters

22
00:01:05.280 --> 00:01:07.710
like the temperature here, for example.

23
00:01:07.710 --> 00:01:09.600
And therefore in general, of course,

24
00:01:09.600 --> 00:01:11.430
if you plan on interacting

25
00:01:11.430 --> 00:01:13.710
with your locally running AI model

26
00:01:13.710 --> 00:01:16.200
through LM Studio programmatically,

27
00:01:16.200 --> 00:01:18.420
you should definitely also take a closer look

28
00:01:18.420 --> 00:01:19.710
at their documentation

29
00:01:19.710 --> 00:01:22.560
to learn about all the features that are available

30
00:01:22.560 --> 00:01:24.960
and all the parameters you can send

31
00:01:24.960 --> 00:01:27.840
and set through that API.

32
00:01:27.840 --> 00:01:30.480
You also might want to explore Headless Mode,

33
00:01:30.480 --> 00:01:35.010
which allows you to run that LM Studio API server

34
00:01:35.010 --> 00:01:38.760
without running the LM Studio application itself,

35
00:01:38.760 --> 00:01:40.560
so that the server keeps on running

36
00:01:40.560 --> 00:01:42.900
even if you close this application,

37
00:01:42.900 --> 00:01:45.210
which could be useful if you try to run that

38
00:01:45.210 --> 00:01:46.743
on a remote server.

39
00:01:47.580 --> 00:01:50.490
But that's all beyond the scope of this course, of course.

40
00:01:50.490 --> 00:01:53.070
This is not an in-depth programming course.

41
00:01:53.070 --> 00:01:55.860
Instead, it is about interacting

42
00:01:55.860 --> 00:01:59.130
with locally running open large language models.

43
00:01:59.130 --> 00:02:00.300
And as you saw,

44
00:02:00.300 --> 00:02:03.450
you can also interact with them programmatically.

45
00:02:03.450 --> 00:02:05.760
For example, here with LM Studio,

46
00:02:05.760 --> 00:02:07.520
with help of the OpenAI SDK.