WEBVTT

1
00:00:00.000 --> 00:00:02.940
<v Maximilian>Of course, you can use Ollama</v>

2
00:00:02.940 --> 00:00:06.420
through that command line-based chat interface.

3
00:00:06.420 --> 00:00:08.730
Or, as briefly shown earlier,

4
00:00:08.730 --> 00:00:10.230
you could use it in conjunction

5
00:00:10.230 --> 00:00:12.780
with a solution like Open WebUI

6
00:00:12.780 --> 00:00:15.750
to add a graphical user interface to it.

7
00:00:15.750 --> 00:00:17.910
Though if you want one, as explained before,

8
00:00:17.910 --> 00:00:21.270
my recommendation would be to just go for LM Studio.

9
00:00:21.270 --> 00:00:24.900
But you can also use Ollama programmatically

10
00:00:24.900 --> 00:00:28.920
just as you could use LM Studio programmatically.

11
00:00:28.920 --> 00:00:30.510
And you can therefore take advantage

12
00:00:30.510 --> 00:00:33.180
of all the benefits Ollama gives you,

13
00:00:33.180 --> 00:00:35.790
like the ability to easily build

14
00:00:35.790 --> 00:00:39.300
your own customized model versions and use those,

15
00:00:39.300 --> 00:00:41.417
so that you could have different models

16
00:00:41.417 --> 00:00:45.240
that have specific system problems for different situations,

17
00:00:45.240 --> 00:00:47.220
and you could easily switch between them.

18
00:00:47.220 --> 00:00:51.090
You can use all that programmatically as well,

19
00:00:51.090 --> 00:00:54.120
because Ollama, like LM Studio,

20
00:00:54.120 --> 00:00:57.030
also runs a server behind the scenes,

21
00:00:57.030 --> 00:00:59.580
a server to which you can send requests

22
00:00:59.580 --> 00:01:01.560
from inside your code.

23
00:01:01.560 --> 00:01:03.660
Now, unlike with LM Studio,

24
00:01:03.660 --> 00:01:07.800
you don't need to explicitly enable that server.

25
00:01:07.800 --> 00:01:11.970
Instead, it's started automatically when you start Ollama.

26
00:01:11.970 --> 00:01:14.670
Now, in case it stopped for whatever reason,

27
00:01:14.670 --> 00:01:19.200
for example, you quit Ollama through the Windows system tray

28
00:01:19.200 --> 00:01:23.070
or the macOS status bar here, if that happened,

29
00:01:23.070 --> 00:01:26.610
you can always run ollama serve to bring it back up.

30
00:01:26.610 --> 00:01:28.563
If you run that command, by the way,

31
00:01:28.563 --> 00:01:32.610
you also get a detailed output of the configuration

32
00:01:32.610 --> 00:01:36.330
that was set for that server, like for example,

33
00:01:36.330 --> 00:01:38.579
the default context length that's being set,

34
00:01:38.579 --> 00:01:41.610
and you're also stuck in a running process

35
00:01:41.610 --> 00:01:44.670
where you will see server log messages.

36
00:01:44.670 --> 00:01:47.280
If you don't want that, you can quit this,

37
00:01:47.280 --> 00:01:51.420
and you can simply ollama run a model

38
00:01:51.420 --> 00:01:52.800
and then quit, for example,

39
00:01:52.800 --> 00:01:56.010
and this will also start that server again.

40
00:01:56.010 --> 00:01:58.530
So now, Ollama is also running again here,

41
00:01:58.530 --> 00:02:00.150
and I would have to quit it again

42
00:02:00.150 --> 00:02:03.900
if I wanted to get rid of that background process.

43
00:02:03.900 --> 00:02:05.580
And with that server up and running,

44
00:02:05.580 --> 00:02:08.250
you can communicate with it from inside your code,

45
00:02:08.250 --> 00:02:10.203
and that's what we'll explore next.