And in this video, we're going to have a look at how you can build a self-correcting rag pipeline rag
standing for retrieval augmented generation.
And after that, we're going to look at how this will be integrated into the side of Landgraaf.
So the first thing I want to bring your attention to is there was a paper done in 2023 which talked
about how if you were able to grade documents and also test for hallucinations, then it was much better
than simply selecting 3 or 5 different documents for the retrieval step to help in grounding the LLM
in factual based answers or consistency of answers.
And so you've got this kind of set up over here where you've got a question, you then do some retrieval.
You grade how well that retrieval is done.
You decide how relevant the docs are.
And then you do a generation step and you check for any hallucinations, and then you check to see does
it answer the question.
And if it doesn't answer the question, then you rewrite the question and then you go back and do more
retrieval and more work.
But if it does answer the question, then you finish at the answer stage.
So this is what we're going to implement.
We'll just be doing a simple version of it and trying to keep it as simple as possible.
And what we'll do to start with is you're going to build the specific functions and you'll see how these
are built.
And then we'll tie that into a graph.
So rather than trying to build the graph we'll actually see how these individual pieces work.
And then we'll we'll decide to package it up inside of land graph.
So we're going to install a couple of packages first.
And you'll need to also install your OpenAI API key and also optional as well.
If you want to follow along with Lang Smith and see the tracing aspect.
Now the first thing we're going to do is we're going to index some blog posts.
Lillian Wang here has produced some really high quality blogs on what AI agents, what our prompt engineering.
And so we can use Lang chains community of web based loaders to specifically load those documents.
And then as well as that, split those into 250 chunks of characters.
And then we can load that into an in-memory chroma vector store.
And we can also get the retriever by basically doing the dot as retriever on that vector store as well.
All right.
Now that we've got the documents embedded inside of chroma, we can then create a retrieval grader.
And the retrieval grader is only going to be responsible for figuring out whether the documents are
relevant to that question with either a yes or a no.
Notice here how we're setting up a Pydantic model, and we're also setting up a photo as the chat model.
And we're using this dot with structured output, which will be producing an output of this Pydantic
model, so that we have a binary score that will either represent yes or no.
And the prompt basically is something along the lines of you're grading the documents based on a question.
Does it have to be a stringent test, but it has to contain some sort of relevance to the question or
the keywords or the semantic meaning and give it a binary, you know, yes or no.
And then we set up our grade prompts, and then we are saying the grade prompt is then going into the
structured LLM grader and we can come up with a question.
So in this scenario we decided agent memory.
And you can see when we do the dot invoke we're then going to be looking inside invoke the retriever
to get the relevant documents.
We'll be actually looking inside of that retriever.
We can get the second pages content as well, and we'll just do one doc as well just to put into the
page content.
And so this is how that's going to work.
So just for for an example here you go.
So you can see that it does think that that this specifically is useful.
This doc text the second article is useful.
And what we can then do is do a generation step.
So we've got our Lang long chain hub, which allows us to import prompts from long chain hub.
And we've also got a string output parser, which when we combine that inline chain expression language,
will also return a string output.
We've then got we've then also got a format docs function, which allows us to easily format the documents.
And you can see here this is how it's working.
So we're passing in the context which is all the docs.
And we're asking the question.
And then we get a generation out that is trying to answer that question.
Right.
So this is the generation code.
And we've also got something for grading the hallucinations.
So this is a binary score for hallucinations present in the generation answer which is either going
to be yes or no.
And pretty much the same code.
You've got some sort of grader that's being set up with the dot with structured output.
And we also have a system message that talks about you're looking at a generated answer.
Does this answer is it grounded in the documents.
So you'll see the chat prompt template here actually contains both the system message and also the documents
and the generation.
And so we then pass the hallucination prompt into the structured grader.
And then we can invoke that with the generation we got in the previous step along with all of the documents.
And you'll see that it believes.
Yes, being that the answer in this scenario is grounded in slash, supported by the sets of facts,
we also can grade the answer as well.
So making sure that the answer addresses the question.
So not just looking for hallucinations but making sure the answer addresses directly the question.
And it's very similar.
The same sort of syntax that we're following along here where we're setting up a grader and we have
that answer prompt as well.
And so in this scenario it says no, which basically means that it doesn't think that it resolves the
question.
So it didn't hallucinate but it didn't resolve the answer.
And we've also got a question Rewriter where we're setting up a chat model.
We've got a system prompt that's telling us to rewrite the question, and then we're giving it the initial
question and formulate an improve improved question.
And so if you remember what our original question was, it was agent memory.
And then this is what is the role and function of agent memory in artificial intelligence systems.
Then we can start setting up now that we've got all the individual pieces our graph.
So in our graph set we'll have the question.
We'll have the generation and we'll have a list of documents.
And we'll then start setting up the nodes.
So we have a retrieve node, which soul's purpose is to get the question and invoke the retriever.
And then just to add in the documents that came to the Landgraff state with that retriever.
So then you've got the generate section where we will take both the question and the documents that
we returned from the retriever, and we will invoke the rag chain with the context being the documents
and the question being the question will then return the generation as an extra key.
You can see here on the far right.
We then grade the documents.
And this grading happens by filtering the documents.
So we get the question and the documents as well.
And we can run that retrieval grader.
We'll get a binary score.
And if it's yes then we want to keep that document.
And if not then it's not relevant.
And we've got this documents being the filter docs.
And we also have a transform query node, which allows us to take in a question, taking the documents,
and then better rewrite the question from that as well.
Then the edges.
So we need to decide whether we're going to generate.
So if there's the filter documents and you've got the question as well.
So if there's no filter documents then all the documents are not relevant and we need to transform the
query.
So we go back into the query.
Else we have all the relevant documents.
So go and generate the answer.
And we've got a final one here which is grade generation versus documents and questions.
So it takes in the questions the documents the generation.
It runs a hallucination grader on the documents and the generation.
And if the grade is yes, then the decision generation is grounded in documents, and then we also grade
it on its answer.
And if the answer is graded successfully yes, then the decision the actual answer addresses both is
inside of the documents and also addresses the question.
And if it goes into these else or not useful or not supported, then it's basically failing at that
point.
And so then we can start setting up our graph with our nodes.
So we add in the retrieve the grade documents, the generate the transform query.
And we set the entry point to the retrieve node.
And then we add an edge from retrieve to the grade documents.
And we add a conditional edge from the grade documents.
And remember this is either going to go and transform the query and regenerate more document retrieval,
or it's going to actually start generating the answer.
And you can see from Transform Query.
This is where we would then go to retrieve retrieval.
If we were to go back into transform Query.
And then we've also got a conditional edge for when we generate.
If we if it's not supported then we regenerate.
If it's useful then we end.
And if it's not useful then we transform the query.
So we go back into the transform query, which would then take us back into retrieval, which would
then start grading the documents.
And then after that, you would then be able to see whether those documents are good enough.
And so then we can run our app and have a look at this.
So we've got question how the different types of agent memory work.
And we can have a look and see.
So it starts by retrieving the the document.
And then we can see that we're checking the documents, and we've assessed it and realized that we decided
to generate.
And then after that we've we've then generated we've checked for hallucinations.
We've made a decision that the generation is grounded documents.
And also we've made a decision that the generation addresses the question.
And so we've decided that this is the output.
But the important thing is that maybe if the document wasn't relevant.
So you can see here check document relevance question.
And it decided the document was relevant.
But there's some documents that aren't relevant which means those get removed.
And again it's graded that the documents and it's made a decision to generate.
It's checked for those nations.
It's the answer is grounded in the documents.
And the generation has is actually addressing the question.
So it's designed to generate.
But if we were to have documents that weren't addressing our different types of generations, and that's
where we would start seeing this extra cycle kick in.
So if I scroll back up to the top, we've encapsulated this as a graph.
But the important point is, if we grade these documents and we don't find any that are relevant, then
we're going to have to go and start rewriting the question.
And the same is true here.
If the answer doesn't specifically address the question, then we go back in here and you might set
a maximum number of iterations to go round on.
But hopefully this gives you a sort of rough understanding about how you could build an adaptive retrieval
based system.
So obviously there's easier ways to do retrieval.
One would just be to simply just select some retrieval and and generate the answer.
But this is trying to encapsulate where we're looking to see whether the documents that are returned
are actually useful.
And also, is the answer hallucinated?
Does the answer refer to the documents?
And if it does, and all of those things are true, then it goes through to the final output.
So this is what you might end up working towards.
But for more simple ones, I would definitely start with just having the question, having the retrieval
node and then having the final answer.
But this is something that you could definitely work up to with time.
All right.
Cool.
I'll see you in the next one.