1
00:00:03,530 --> 00:00:11,780
Okay, so let's study the llama index framework in more detail in this second lesson.

2
00:00:11,780 --> 00:00:20,870
And this second lesson is going to be a very interesting for you to learn how artificial intelligence

3
00:00:20,870 --> 00:00:29,840
engineers work when we are, uh, meeting a new tool, framework, language, whatever.

4
00:00:29,840 --> 00:00:30,440
Right.

5
00:00:30,440 --> 00:00:38,120
So we are going to do in front of you exactly what we do with whatever framework or new tool or new

6
00:00:38,120 --> 00:00:40,010
language we, we face.

7
00:00:40,010 --> 00:00:40,610
Right?

8
00:00:40,760 --> 00:00:52,010
So we first thing you do is you go to the website of the framework, the tool, uh, the language,

9
00:00:52,010 --> 00:00:52,550
etc..

10
00:00:52,550 --> 00:00:52,970
Right?

11
00:00:52,970 --> 00:00:58,430
So in this website you are going to find two kinds of information.

12
00:00:58,880 --> 00:01:06,410
First what you can call the marketing information, which is the main message that the owners of the

13
00:01:06,410 --> 00:01:08,540
framework wants to send to you.

14
00:01:08,900 --> 00:01:10,310
And this is very interesting.

15
00:01:10,310 --> 00:01:15,440
And we will analyze what is the main message and the different messages that they are sending us.

16
00:01:15,440 --> 00:01:20,690
And the second part you are going to focus your attention in is the documentation.

17
00:01:20,690 --> 00:01:23,810
We will also take a look at that in detail.

18
00:01:23,810 --> 00:01:26,510
So first let's take a look.

19
00:01:26,510 --> 00:01:29,090
So we are going to work with this second notebook.

20
00:01:29,090 --> 00:01:30,680
We we are prepared.

21
00:01:30,680 --> 00:01:32,060
We have prepared for you.

22
00:01:32,060 --> 00:01:36,590
And as you can see in the in the initial part we are loading the dot HTML file.

23
00:01:36,590 --> 00:01:44,480
And then we, we just make this note here saying that the previous name of llama index was GPT index.

24
00:01:44,480 --> 00:01:48,470
We explain the reason why in the in the previous lesson.

25
00:01:48,470 --> 00:01:48,830
Right.

26
00:01:48,830 --> 00:01:54,140
So let's start, uh, analyzing studying this home page.

27
00:01:54,140 --> 00:02:01,760
So every uh, home page, every website home page has what we call a pitch.

28
00:02:01,760 --> 00:02:06,830
The pitch is the main message is the main idea they want you to keep.

29
00:02:06,980 --> 00:02:10,460
In the case of llama index, the pitch is unleashed.

30
00:02:10,460 --> 00:02:13,640
The power of LMS over your data.

31
00:02:13,730 --> 00:02:16,190
Do you remember the previous name of llama index?

32
00:02:16,190 --> 00:02:17,540
GPT index?

33
00:02:17,540 --> 00:02:21,110
This is exactly what that name was about.

34
00:02:21,110 --> 00:02:23,810
Unleash the power of LMS over your data.

35
00:02:23,810 --> 00:02:33,080
So use LMS to search into your data and to use your data in many different forms to integrate with other

36
00:02:33,080 --> 00:02:34,550
tools, etc. etc..

37
00:02:34,550 --> 00:02:35,090
Right?

38
00:02:35,510 --> 00:02:46,520
So down you will see the secondary message is like okay, we use llama index to.

39
00:02:47,750 --> 00:02:49,010
Ingest data.

40
00:02:49,920 --> 00:02:56,160
Two for data indexing and for and as a query interface.

41
00:02:56,640 --> 00:02:58,650
Okay, so boom boom boom.

42
00:02:58,650 --> 00:02:59,970
Main message.

43
00:03:00,120 --> 00:03:02,130
Secondary message okay.

44
00:03:02,130 --> 00:03:04,140
So this gives you the idea.

45
00:03:04,140 --> 00:03:06,690
Do you remember what we said about Lambda index.

46
00:03:06,690 --> 00:03:10,620
They are much more focused than long chain.

47
00:03:11,680 --> 00:03:12,490
From the beginning.

48
00:03:12,490 --> 00:03:16,030
The mind tells you we are about rag technique.

49
00:03:16,030 --> 00:03:18,610
Okay, this is what we understand here.

50
00:03:18,910 --> 00:03:23,140
Then they tell you about their main use cases.

51
00:03:23,140 --> 00:03:31,750
And as I was telling you, even when they provide you these four use cases, than chain is about this

52
00:03:31,750 --> 00:03:32,770
and this.

53
00:03:32,770 --> 00:03:35,290
This is Rag technique okay.

54
00:03:35,290 --> 00:03:37,180
These other things are okay.

55
00:03:37,180 --> 00:03:37,810
Interesting.

56
00:03:37,810 --> 00:03:41,770
But no no no right now Lamar index is about this okay.

57
00:03:41,770 --> 00:03:45,730
So you can see a screenshot of applications using this.

58
00:03:45,730 --> 00:03:47,650
This by the way is second sites.

59
00:03:47,650 --> 00:03:48,580
We will see.

60
00:03:50,200 --> 00:03:53,980
Or the origins of of sacred sites.

61
00:03:53,980 --> 00:03:56,860
You will see this document in the second sites application.

62
00:03:56,860 --> 00:03:59,950
We we we see in our in our next lesson.

63
00:03:59,950 --> 00:04:00,520
Okay.

64
00:04:00,550 --> 00:04:01,510
So.

65
00:04:02,400 --> 00:04:11,340
Then in the home page they tell you about the different types of data you can use with lambda index.

66
00:04:11,940 --> 00:04:19,260
And this is the most frequent data, uh, you are used to what is called structured data.

67
00:04:19,260 --> 00:04:23,370
These are tables, spreadsheets and databases.

68
00:04:23,370 --> 00:04:25,500
Databases are just tables right.

69
00:04:25,500 --> 00:04:27,600
And spreadsheets are just tables.

70
00:04:27,600 --> 00:04:31,080
So Lambda index works with tables.

71
00:04:31,410 --> 00:04:35,400
They work with what is called unstructured data.

72
00:04:35,400 --> 00:04:45,270
So like text, video, etc. and they work with what they call semi-structured data, which is a combination

73
00:04:45,270 --> 00:04:46,710
of the previous two.

74
00:04:46,710 --> 00:04:53,370
And they highlight here like we can work with data from Salesforce, slack, notion, etc..

75
00:04:53,370 --> 00:04:55,920
Okay, so hmm, interesting.

76
00:04:55,950 --> 00:05:00,720
A home page with four main marketing messages.

77
00:05:00,990 --> 00:05:01,860
Pitch.

78
00:05:02,510 --> 00:05:04,190
Secondary message.

79
00:05:05,350 --> 00:05:13,960
A user cases and then a explanation about what they mean when they say data.

80
00:05:13,990 --> 00:05:14,650
Okay.

81
00:05:15,100 --> 00:05:16,000
Now.

82
00:05:17,790 --> 00:05:21,300
Also interesting to take a look at the A.

83
00:05:22,100 --> 00:05:28,220
Navigation the top navigation where you can find the main contents of every website.

84
00:05:28,220 --> 00:05:28,610
Right.

85
00:05:28,610 --> 00:05:36,320
So one of the interesting things you are going to find with Lima Index, as you found with long chain,

86
00:05:36,320 --> 00:05:43,370
is that they are preparing to work with Python developers and with JavaScript developers.

87
00:05:43,370 --> 00:05:50,270
TypeScript, as you know, is just an advanced, uh, well, a type of JavaScript.

88
00:05:50,270 --> 00:06:00,380
Okay, so why is Lima index and long chain working with Python and TypeScript developers and JavaScript

89
00:06:00,380 --> 00:06:01,220
developers?

90
00:06:01,890 --> 00:06:14,310
Because they are in the border between traditional, uh, artificial intelligence engineers, the machine

91
00:06:14,310 --> 00:06:16,710
learning engineers, and the.

92
00:06:17,380 --> 00:06:22,240
Traditional internet application engineers, the full stack engineers.

93
00:06:22,240 --> 00:06:30,250
So the classic artificial intelligence engineers, the ones working in machine learning, deep learning,

94
00:06:30,250 --> 00:06:32,200
etc. they work with Python.

95
00:06:32,780 --> 00:06:41,780
But they used to work in, you know, universities, research projects, science data science projects,

96
00:06:41,780 --> 00:06:50,180
etc. and now with the LM applications, they are starting to work in full stack applications.

97
00:06:50,690 --> 00:06:51,770
So.

98
00:06:52,270 --> 00:06:55,390
This approach here are working with Python.

99
00:06:55,390 --> 00:06:57,250
Have a lambda index.

100
00:06:57,250 --> 00:07:01,900
A Python version of lambda index is addressing this community.

101
00:07:01,900 --> 00:07:04,750
The traditional machine learning engineers.

102
00:07:04,750 --> 00:07:08,290
This one the TypeScript version of Lambda index.

103
00:07:09,540 --> 00:07:16,020
Isa an approach to target the traditional full stack engineers.

104
00:07:16,020 --> 00:07:19,920
The the typical internet application engineers.

105
00:07:19,920 --> 00:07:28,620
They work with JavaScript, and when they approach Elm applications, they want to use JavaScript.

106
00:07:28,620 --> 00:07:35,820
So we will talk more about these two approaches, because our full stack applications are going to be

107
00:07:35,820 --> 00:07:44,040
a combination of Python and JavaScript, but we are not going to, uh, focus our attention on the whole

108
00:07:44,040 --> 00:07:46,260
Python and whole JavaScript frameworks.

109
00:07:46,260 --> 00:07:51,990
We are only going to use and focus our attention in the parts of Python and JavaScript that are going

110
00:07:51,990 --> 00:07:56,310
to be useful for us to build professional Elm applications.

111
00:07:56,310 --> 00:08:00,060
Okay, but interesting to see this navigation here.

112
00:08:00,960 --> 00:08:04,440
Second thing interesting here is what they call the Lima Hub.

113
00:08:04,440 --> 00:08:11,040
So in the Lima hub you are going to find as you as you can see here, the templates of Lima Index.

114
00:08:11,250 --> 00:08:17,340
So templates are ways to help you build faster Lima index applications.

115
00:08:17,340 --> 00:08:20,880
They have a one very interesting template called Create Lima.

116
00:08:21,540 --> 00:08:29,100
Then they have data loaders and agent tools similar to Lang Lang chain very early this very new very

117
00:08:29,100 --> 00:08:30,270
better still.

118
00:08:30,840 --> 00:08:35,370
And then here in templates they have the Sick Insights project.

119
00:08:35,370 --> 00:08:39,360
So here is what you we are going to go there in detail later.

120
00:08:39,360 --> 00:08:42,659
But here is where you can find about this project.

121
00:08:42,659 --> 00:08:45,600
You know the documentation about the project etc. etc..

122
00:08:45,600 --> 00:08:45,960
Okay.

123
00:08:45,960 --> 00:08:49,200
We will go there in the next lessons.

124
00:08:49,680 --> 00:08:57,180
And uh, you also have the Create Lima which is a tool to install Lima index from terminal.

125
00:08:57,180 --> 00:09:02,160
It's very interesting approach, similar to the approach that Lang chain is also taking.

126
00:09:02,160 --> 00:09:02,850
Okay.

127
00:09:03,210 --> 00:09:10,170
So here we we talk about the last features that Lima Index is talking about.

128
00:09:10,650 --> 00:09:15,210
Interesting to see you know the new ideas and projects they are working in okay.

129
00:09:15,330 --> 00:09:16,080
But.

130
00:09:17,310 --> 00:09:18,210
The interesting thing.

131
00:09:18,210 --> 00:09:21,960
Now let's focus our attention on the documentation.

132
00:09:21,960 --> 00:09:28,980
So this is the documentation of the llama index zero 943 version.

133
00:09:29,430 --> 00:09:40,200
This is the page you are going to spend a lot of time in a once you start working uh, in a, in a project

134
00:09:40,200 --> 00:09:41,190
in llama index.

135
00:09:41,190 --> 00:09:41,760
Okay.

136
00:09:41,760 --> 00:09:48,000
So now what we are going to do is a soft landing telling you about the parts you are going to find here

137
00:09:48,000 --> 00:09:52,110
and some very interesting things about, uh, some of these parts.

138
00:09:52,110 --> 00:09:59,640
But when you start working in a llama index project, you are going to focus a lot of time here, especially

139
00:09:59,640 --> 00:10:01,320
in this section, as we will see.

140
00:10:01,320 --> 00:10:04,560
So what do you have here in the notebook?

141
00:10:05,250 --> 00:10:10,020
First, you have a note about the structure of this documentation.

142
00:10:10,050 --> 00:10:12,180
The different sections you may find.

143
00:10:12,690 --> 00:10:19,380
And remember this is a very young framework, which means that it is going to evolve very quickly,

144
00:10:19,380 --> 00:10:23,460
which means that in the next months you may find changes here.

145
00:10:23,460 --> 00:10:23,940
Okay.

146
00:10:23,940 --> 00:10:24,630
Don't worry.

147
00:10:24,630 --> 00:10:31,410
This approach we are we are making is going to be useful for you no matter what content you find whenever

148
00:10:31,410 --> 00:10:33,360
you are studying this program.

149
00:10:33,360 --> 00:10:39,960
So the first, the first thing we do when we approach to a framework is like, okay, what are the sections

150
00:10:39,960 --> 00:10:41,220
of the documentation?

151
00:10:41,220 --> 00:10:47,160
In this case we have a Getting started section, which is the section we will study.

152
00:10:47,160 --> 00:10:52,020
When we are learning about a new framework, then we have use cases, which is interesting.

153
00:10:52,020 --> 00:11:00,690
Once you are starting to work in a project which enters in one of these categories, then a they provide

154
00:11:00,690 --> 00:11:09,090
a section where you can learn, you know, more about Lamar Index after the Getting started section.

155
00:11:09,090 --> 00:11:11,490
And then they have the main part.

156
00:11:11,490 --> 00:11:16,050
The most valuable part of Lamar index is this optimizing section.

157
00:11:16,050 --> 00:11:22,950
This is where you can make your applications from demos to professional.

158
00:11:22,950 --> 00:11:26,970
So this is the section where you are going to invest more of your time.

159
00:11:26,970 --> 00:11:32,430
Most of your time when you are working in a professional application and you have decided if you have

160
00:11:32,430 --> 00:11:43,140
decided to build it in Lamar Index, which is right now a very common, uh, option, because usually

161
00:11:43,140 --> 00:11:51,150
people go to long chain to start learning, but once they want to do something, uh, more professional,

162
00:11:51,150 --> 00:11:55,050
usually launching is is a little bit short for that.

163
00:11:55,050 --> 00:11:59,790
So they go to Lamar Index or to a OpenAI API.

164
00:12:00,400 --> 00:12:03,880
Long chain is working very hard to solve that.

165
00:12:03,880 --> 00:12:11,230
And they are focusing right now in improving, you know, their their level of sophistication and the

166
00:12:11,230 --> 00:12:16,150
their performance and all this area that la mendes has already developed.

167
00:12:16,150 --> 00:12:21,910
So this is the focus of long chain trying to keep up with a llama index in all these areas.

168
00:12:21,910 --> 00:12:29,320
But right now llama index is, in my opinion, the leader for the professional applications in the rag

169
00:12:29,320 --> 00:12:29,860
space.

170
00:12:29,860 --> 00:12:35,410
Okay, so probably you are going to spend a lot of time here once you start building your professional

171
00:12:35,410 --> 00:12:36,070
applications.

172
00:12:36,070 --> 00:12:36,550
But.

173
00:12:37,420 --> 00:12:45,310
Also you have here module guides section, which is uh includes guides to the individual components

174
00:12:45,310 --> 00:12:46,420
of lambda index.

175
00:12:46,420 --> 00:12:46,960
Okay.

176
00:12:46,960 --> 00:12:55,180
So when we approach to this, uh, documentation, we spend more of our time in the getting started

177
00:12:55,180 --> 00:12:56,140
section.

178
00:12:56,290 --> 00:13:02,170
And the first thing we focus our attention in was the starter tutorial.

179
00:13:02,170 --> 00:13:07,930
This is the part, uh, we use to prepare the, the first lesson, the previous lesson.

180
00:13:07,930 --> 00:13:10,450
So this is where you load how to.

181
00:13:10,480 --> 00:13:16,210
This is where you learn how to load a private document to create the vector database, etc., etc..

182
00:13:16,210 --> 00:13:16,660
Okay.

183
00:13:16,660 --> 00:13:18,220
Basics of rack.

184
00:13:19,010 --> 00:13:24,740
Then we went to a a high level concepts okay.

185
00:13:24,740 --> 00:13:31,610
So in the high level concepts this is a very interesting section where the team in Laminex is going

186
00:13:31,610 --> 00:13:38,060
to go deeper into rack concepts and especially rack stages.

187
00:13:38,060 --> 00:13:47,240
This section is super important because as you will see, lambda index is optimizing the rack technique

188
00:13:47,600 --> 00:13:48,710
step by step.

189
00:13:48,710 --> 00:13:56,180
So first they study very, very in detail each of the stages of rack.

190
00:13:56,180 --> 00:14:01,520
And then they focus their attention on optimizing on each of these stages.

191
00:14:01,520 --> 00:14:04,190
So these are very important section.

192
00:14:05,240 --> 00:14:11,330
So they go into detail in each on for each of the stages.

193
00:14:11,330 --> 00:14:16,070
And then they, they they go around some main use cases.

194
00:14:16,070 --> 00:14:25,400
Then we went to customized tutorial which is an initial a approach to optimization.

195
00:14:25,400 --> 00:14:33,440
So they tell you okay we have a basic rack, but then we are starting to uh fine tune it.

196
00:14:33,440 --> 00:14:38,000
So we are working the size of the chunks we are working with.

197
00:14:38,000 --> 00:14:44,420
We are going to work with different vector databases, we are going to use different LMS, etc., etc.

198
00:14:44,420 --> 00:14:44,750
etc..

199
00:14:44,750 --> 00:14:55,280
So this is a this is a way for lambda index to tell you, hey, we are much more than a basic rack platform.

200
00:14:55,280 --> 00:15:00,860
We are going to tell you how to improve the basic rack approach.

201
00:15:00,860 --> 00:15:03,770
Uh, in many ways.

202
00:15:03,770 --> 00:15:04,340
Right?

203
00:15:04,520 --> 00:15:09,410
Then we took a look to the Lambda Index video series.

204
00:15:09,410 --> 00:15:10,850
This is very interesting.

205
00:15:10,850 --> 00:15:19,880
Once you are, you know, into some of these, uh, areas in detail, we took a look at the use cases

206
00:15:19,880 --> 00:15:21,260
section, the same thing.

207
00:15:21,260 --> 00:15:25,400
So this section and this section are going to be interesting for you.

208
00:15:25,400 --> 00:15:33,140
As long as you are working in a project or you want to work in a project related with any of these points.

209
00:15:33,140 --> 00:15:38,300
And finally we went to the understanding section.

210
00:15:38,660 --> 00:15:46,370
This is where uh, you can see the differences between lambda index and long chain.

211
00:15:46,370 --> 00:15:51,770
So you will see that they are using different ways of loading embeddings.

212
00:15:52,560 --> 00:15:57,420
And they are using similar ways for many other things like prompt templates, etc..

213
00:15:57,450 --> 00:16:05,100
Okay, so we have our summary here, but you will probably find some notes that we didn't.

214
00:16:05,100 --> 00:16:07,500
But this is the the analysis we did like.

215
00:16:07,500 --> 00:16:15,390
Okay how are these two different frameworks working with LMS with indexing with la la la la la.

216
00:16:15,510 --> 00:16:19,650
Every time you see vector store, by the way, it's the same that vector database.

217
00:16:19,650 --> 00:16:20,820
So it's different names.

218
00:16:20,820 --> 00:16:26,040
In some cases these two platforms use different names for the same things.

219
00:16:26,040 --> 00:16:32,580
So instead of using chunks uh lambda index use the word node.

220
00:16:32,580 --> 00:16:37,290
So that's a little bit, you know like confusing.

221
00:16:37,380 --> 00:16:41,130
But you will immediately see that they are referring to the same thing.

222
00:16:41,130 --> 00:16:41,640
Okay.

223
00:16:41,640 --> 00:16:44,850
So this is a good section to compare both platforms.

224
00:16:45,750 --> 00:16:56,010
And finally, in this one, you don't have to go, you know, in detail, but it will be enough for

225
00:16:56,010 --> 00:17:03,840
you to understand where you need to return once you start working in a professional application with

226
00:17:03,840 --> 00:17:04,770
lambda index.

227
00:17:04,800 --> 00:17:10,440
Okay, so this is the way we approach a framework like lambda index.

228
00:17:10,440 --> 00:17:14,010
And we wanted to do this with you.

229
00:17:14,010 --> 00:17:20,490
So you can reproduce this approach whenever you find a new a framework yourself.

230
00:17:20,490 --> 00:17:28,950
And also to tell you that your work around Lambda index starts now.

231
00:17:29,040 --> 00:17:37,260
So is now when you need to start practicing repeating some of the exercises we have provided for you.

232
00:17:37,260 --> 00:17:42,930
But most, most, most of all, start practicing and experimenting yourself.

233
00:17:43,750 --> 00:17:47,470
So use what you have here this tutorial.

234
00:17:47,470 --> 00:17:50,920
Use ChatGPT for experiment.

235
00:17:50,920 --> 00:17:52,120
Make mistakes.

236
00:17:52,120 --> 00:17:56,560
You need to learn all these, uh, by memory.

237
00:17:56,560 --> 00:17:57,550
That's not that.

238
00:17:57,550 --> 00:17:58,570
That's not the point.

239
00:17:58,570 --> 00:18:02,380
The point is that you are familiar with the main concepts.

240
00:18:03,290 --> 00:18:12,470
And and that you know where to come back, where to return once you start working in a professional

241
00:18:12,470 --> 00:18:13,400
application.

242
00:18:13,400 --> 00:18:14,060
Okay.

243
00:18:14,060 --> 00:18:20,090
So get familiar with Lama index the same way you got familiar with a long chain.

244
00:18:20,180 --> 00:18:23,900
We are going to work with Lama index in future lessons.

245
00:18:23,900 --> 00:18:33,140
So so we will we will be back to this but for now and go ahead and get comfortable with Lama index.

246
00:18:33,140 --> 00:18:35,840
For some people it's going to be 30 minutes.

247
00:18:35,840 --> 00:18:39,650
For some people it's going to be uh, 48 hours.

248
00:18:39,650 --> 00:18:41,150
Don't worry about that.

249
00:18:41,150 --> 00:18:43,820
Each person has its own rhythm.

250
00:18:43,820 --> 00:18:44,990
Don't worry about that.

251
00:18:44,990 --> 00:18:52,790
Just keep practicing and keep working on this framework until you feel comfortable and, uh, and,

252
00:18:52,790 --> 00:18:56,450
and then go to the next, uh, lesson.

253
00:18:56,450 --> 00:19:01,790
In the next lesson, by the way, we are going to talk about the OpenAI API.

254
00:19:02,790 --> 00:19:11,850
Remember that the typical artificial intelligence engineer they normally choose between these three

255
00:19:11,850 --> 00:19:18,870
options launching Lambda index or the OpenAI API directly.

256
00:19:18,870 --> 00:19:23,880
We have been talking about these, uh, three choices in a previous section.

257
00:19:23,880 --> 00:19:28,260
And uh, right now you are familiar with the first one, with the second one.

258
00:19:28,260 --> 00:19:33,660
In the next blog, we are going to start talking about the third option, which is a very interesting.

259
00:19:33,660 --> 00:19:37,440
It has advantages and disadvantages, as you will see.