WEBVTT

1
00:00:00.570 --> 00:00:02.520
<v Instructor>This course is about</v>

2
00:00:02.520 --> 00:00:04.950
running large language models,

3
00:00:04.950 --> 00:00:08.040
openly available large language models,

4
00:00:08.040 --> 00:00:11.910
specifically locally on our machines, on your laptop,

5
00:00:11.910 --> 00:00:14.130
on my laptop, or of course,

6
00:00:14.130 --> 00:00:17.700
also on that remote server you maybe rented.

7
00:00:17.700 --> 00:00:19.590
And therefore, of course, we need

8
00:00:19.590 --> 00:00:22.020
to explore hardware requirements

9
00:00:22.020 --> 00:00:25.200
and an important concept called quantization

10
00:00:25.200 --> 00:00:27.570
because as you might guess,

11
00:00:27.570 --> 00:00:32.570
running large models could in theory be tricky

12
00:00:32.640 --> 00:00:35.640
if you are not having a supercomputer at home.

13
00:00:35.640 --> 00:00:37.650
But as you'll see in this section,

14
00:00:37.650 --> 00:00:40.020
it's thankfully not really a problem,

15
00:00:40.020 --> 00:00:42.060
and you can run very capable,

16
00:00:42.060 --> 00:00:45.150
large language models on your laptop, your server,

17
00:00:45.150 --> 00:00:47.250
or whichever machine you have.

18
00:00:47.250 --> 00:00:49.530
Now, in order to understand how that's possible

19
00:00:49.530 --> 00:00:52.800
and which hardware requirements you do need to meet,

20
00:00:52.800 --> 00:00:55.710
we'll take a look at model parameters

21
00:00:55.710 --> 00:00:58.601
and their sizes and why that's important.

22
00:00:58.601 --> 00:01:02.280
We'll then derive some hardware requirements

23
00:01:02.280 --> 00:01:05.490
from that information, and we'll then explore

24
00:01:05.490 --> 00:01:10.296
how quantization helps us lower these hardware requirements

25
00:01:10.296 --> 00:01:13.500
without getting a bad performance

26
00:01:13.500 --> 00:01:15.990
or without impacting the performance

27
00:01:15.990 --> 00:01:18.213
of these open models in a negative way.