Let's understand the data flow steps for evaluation.
We learned this in our previous section in Knowledge base where there is a local data source.
We use the PDF here.
Load the PDF into AWS account that is S3 bucket.
Then we perform chunk operations on the PDF or the data source.
And then the embedding model would embed those chunks and store it in the vector database.
When we perform a search, that search query is then embedded using embedding model and looks for the
relevant content in the vector database, and then gives us the top 5 or 10 chunks that matches the
search query.
Depending on the embedding model.
So that's the retrieval part.
Let's move on to the generation part, where we provide a prompt and the search content that were retrieved
to a large language model.
Using the search content and the prompt it generates the response.
This is the Rag pipeline.
For evaluation, we need more than just the responses.
What we need is evaluation data set.
This is also the ground truth that we have to provide.
Evaluation also needs the context that we got from this search context.
We also need the prompt.
So this evaluation data set would also have the prompt.
And then all of this content is then passed to the prompt template.
And then this prompt template here would be used and be passed to the large language model two.
This model may or may not be the same as LM one, but the content that goes into those large language
models are definitely different.
So in the step two here we pass the context ground truth data prompt and the response from the current
state into another LM.
And then that large language model would generate the evaluation, which is what we will cover with
a hands on activity in our next section.
All right.
Thank you so much.
I'll see you in the next video.