Hello everyone!
In today's video we will learn about Retrieval augmented generation, also known as Rag.
Before we take a deep dive into what is Rag, let's understand what is LM and how it is used.
LM is large language model that is trained on vast volumes of data, and use billions of parameters
to generate original output for tasks like answering questions, translating languages, and completing
sentences, the model bases its response on the textual content it has ingested during training.
Now let's understand what are the challenges with LM?
Lack of specific data.
Language models are limited to providing generic answers based on their training data.
If a user were to ask about specific domain questions, a traditional LM may not be able to provide
accurate answers.
Another challenge is hallucination.
The model bases its response on the textual content it has ingested during training, and there is no
telling exactly what went into that data or how the model recombines it to generate the novel text.
Another challenge is generic responses.
Language models often provide generic responses and aren't tailored to specific contexts.
This can be a major drawback in a customer support scenario, since individual user preferences are
usually required to facilitate a personalized customer experience.
These are some of the challenges that I have listed here, but there are a lot more.
Rag in general aims to solve some of the challenges that are listed here.
Let's move on to understand what is Rag Retrieval.
Augmented generation, also known as Rag, is the process of optimizing the output of language model
so it references an authoritative knowledge base outside of its training data source before generating
a response.
Now let's understand how Rag works.
Step one is data collection.
You must first gather all the data that is needed for your application.
This can be domain specific data that Lem is not trained on.
Step two is data chunking.
Data chunking is the process of breaking your data into smaller, more manageable pieces.
Large language models.
They have a context window or embedding models also have context window.
Um, these models cannot process thousands of line items.
They have a context that they can process.
That is where data chunking is needed.
You need to understand the model context limitation and chunk the data accordingly.
Step three is document embedding.
Now that the source data has been broken into smaller parts, it needs to be converted into vector representation.
This involves transforming text data into embeddings, which are numeric representation that captures
the semantic meaning behind the text.
Embeddings is a deeper topic that needs more understanding.
I have covered it as part of another video where you can understand how embedding works.
Next topic here is handling user queries.
So once we embed the smaller chunks, you have to handle the user queries.
When a user query enters the system, it must also be converted into embeddings or vector representation.
The same model must be used for both document and query embedding to ensure uniformity between the two
processes.
So if you use a vector embedding model for document embedding, it has to be the same model that embeds
the user query.
If the two models are different, then the vector embeddings of the two models are different and you
won't get a uniform search experience.
So very important that you use the same embedding model for document embedding and handling the user
queries.
And then the fifth and the final step is generating responses with an LM.
The retrieval text chunks along with the initial user query are fed into a language model.
This is how different steps in the Rag orchestration works.
Let's understand this using a diagram.
So here is the diagram where the entire Rag system is Was explained.
The very first step here on the bottom left you see data collection and chunk.
That is step one and two where we collect the data and chunk the data.
Once you chunk the data, you send it to an embedding model and then do a vector embedding using the
model and store it to the vector database.
So step one and two is data collection and chunk.
Then step three.
Here is the chunks that were created.
You embed them using an embedding model and then you store it in the vector database.
This is usually a batch process that keeps running over and over again.
Um, then once the data is stored as a in the vector database, the user passes the query and a prompt.
So when they pass this to the Rag orchestrator, Rag orchestrator uses just the query from the from
the prompt and the query, and it embeds it using the same embedding model and then searches for the
relevant content in the vector database.
Once it searches for the relevant content in the vector database, it then extracts the relevant content,
gives it back to the Rag orchestrator.
Rag orchestrator combines the prompt, the query, and the context info that it retrieved and sends
it to.
The LLM would process the data and generate the response.
And step five.
So these are the different steps that are involved in the Rag orchestrator.
And this is how Rag works.
Let's take a real world example and understand this in a little more detail.
Thank you and see you in the next video.