Finally now have all the building blocks ready
in order to do the search in the
RAG.
So what we're going to do here is
we're going to build a chat loop where
we can actually use the RAG.
So in order to do that we make
the normal connection, we define the embedding generator
and we need that because when we search
we also need the embedding generator not only
when we ingest data because when we send
it we need to send it as a
vector instead of as text.
The vector store will take care of that
but just so we know and in real
life things like these where you embed make
the generator and so on you'll put in
dependency injection or some helper tool and not
do it all over the place.
This is just so it's easier for you
to understand in this example.
So we define our embedding generator, we make
our vector store which will just connect to
the existing one we have and then we
are gonna get our collection.
So we have our collection to search against
because we're not searching against the vector store
we're always searching against a specific table or
index depending on what it is you have
and then we're making an agent so we're
back to agents and not just vectors and
this is an expert in the company's internal
non-space.
And then we start our normal chat loop.
So now it's waiting for me to answer
set a question and let's ask it in
a slightly different way so we can see
what's going on.
So let's say what is the password to
the internet.
It could be that the specific user don't
know that it's called a wi-fi so
they just want to get onto the internet
and it's asking for a password.
If we do this let's press the right
buttons here and we get our input and
what we're going to do is we're going
to make a similarity search against the collection
and the way we do that is we
tell our collection to do a search async
with the input in our case.
We could have made the AI enhance the
question turn it into keywords and stuff like
that.
That's all the parts of optimising and figuring
out how the best way to search but
in our case we're just doing the crude
and actually giving the vector to find something
similar to what the user have asked.
And what we do is we say among
all the records give us back the top
three most relevant.
In our case when we only have 10
that might be a little much but in
other cases where you have thousand it could
be that you need to get the top
20 or something like that.
But in our case we make the search
so behind the scenes when we do this
search it will actually take this sentence what
is the password to the internet turn it
into a vector and then send that to
the vector store for doing the similarity search
a little like we did in the first
lecture of this.
So it will come back and the first
record is luckily for us the question of
what is the wi-fi password for the
office and what is the guest and we
will see that the score is quite low
and i will explain that in a little
while.
Because before we saw that our score should
be close to one when it was when
it was high and then we get another
question this question is something about logging into
my office account and the reason why this
gets a score that is relevant is because
the answer contains the word password and we
use password in this.
This one have a score that is higher
and i will again explain that when we
see the final result.
This one is a question about who is
in charge of support and my guess is
why this is relevant is because it's showing
an email and email and internet are close
to each other so for that reason it
gets a score.
And that is our tree so we get
our report back here and we can actually
just see it on screen.
So what you can see is we get
a higher and higher score and this is
an important thing for you to know because
when i showed you our C-sharp similarity
scores we talked about one being the best
and zero being the worst.
But in terms of a SQLite they have
chosen a different metric.
They have chosen that the closer to zero
the better.
So if you begin to set some limits
on i will only i would like to
get 10 records back but only if they
are above this score you would actually get
a worse thing here because it would be
the least score.
So you need to know that your specific
way of getting scores back is low to
high in this case while others it's high
to low while sometimes it's negative as well
all depending on what similarity formula they're using.
You can also just not care because this
search will always give the most relevant back
first no matter what.
So that's the reason why we get this
one.
So we have our scores now and what
we're gonna do is we're gonna put that
into a message together with the real question
and we are going to say here's the
most relevant knowledge base information.
So we are telling among all the 10
which it's not getting it's only getting these
three and among those figure out what the
answer is.
So here we have our two messages we're
sending in.
Before we have only sent one in here
but now we can actually send multiple in
if we wish to like this and then
we get our response back.
So if we see and it is good
enough to know that among these three the
right one is the first one.
So even if some kind of similarity had
hit this one to be the most relevant
the AI would still know oh that has
nothing to do with Wi-Fi passwords so
it must be the second best.
So that's the reason why we only we
don't only send the one best back.
Sometimes there can be a race condition of
what is the most similarity.
So we sent like three or ten or
that's also a science to figure out should
we send because the more we send the
more we give us input.
All this will cost input tokens as well.
So had we sent 100 it would have
cost more to answer this question.
Do we send only one it will cost
less but with the chance that it was
not the real answer because then it suddenly
goes from being kind of smart to absolutely
dumb because it didn't get the information in
the first place.
So if we just try and run it
again without the breakpoints just so we can
see the experience.
What is the Wi-Fi password?
Again it will come as the top one.
While there was another question about Christmas.
So is Xmas a day off?
It will now come back with other search
results.
So again it found it because it's the
only one mentioning eve and so on.
But if we do something else let's say
hello.
It will still search for something and that
is actually what we will cover in the
next lecture because we don't really want to
have it search anything at this point because
we're just saying hello to the AI.
But that's for the next lecture.