In this lecture, I'm going to show you
something called tool calling middleware.
And for that, I've taken a copy of
the creating tools, the first tool video we
saw, where we had three different tools for
getting person, getting person, and change colour.
And what I have done over in this
version is I have removed these information printouts
that tools are being called, because that is
too cumbersome to do.
We have middleware to do that for us.
And middleware is a method called middleware, or
whatever you call it, you can choose.
And it takes an input of the agent
that are using the middleware, or about to
do the tool call, context about the tool
call, and next objects.
If you know middleware, you normally invoke these
if you want to proceed, and not invoke
it if you don't, and then some cancellations.
We can set a breakpoint in this.
In order to enable this middleware, we have
changed our agent slightly, and that we have
set as builder on the agent, use middleware
.build. If we do this, the agent goes
from a chat client agent to an AI
agent, which is a more generic agent.
This has some negative effects later on, but
for now, it doesn't matter for us.
And the negative thing is that structured output
becomes a bit more cumbersome to do, so
it's not a total loss, but it is
a little annoying that it changes.
But if we do this, we can now
run, and we can ask about the different
people's colours.
So if we ask about Mona's favourite colour,
we hit this breakpoint, so we get that
we have an agent, and we get some
context about what we are about to call
here.
In this case, it's the tool called getPerson,
and the argument is this case, as we
expect, Mona.
So here, I'm just building up a small
little output message, so I don't need to
put it into every single tool.
And then I'm calling next to let the
tool be called.
You can do the same for Ben.
We will hit this, and our argument will
instead be Ben down here.
And finally, we can do it for John,
and see that his colour is red.
So that is the best way of telling
your system.
Right now, we are doing consoles, but it
could be to a lock that this and
this tool was called with this and this
information, instead of putting them into the tools
themselves.
But we can do more than this in
here.
We can also manipulate and even refuse to
call the tool.
So I just put in a bit of
code here, and I will set a few
breakpoints here.
We'll see in one second what these do.
So we'll start up again, and we'll again
ask for Mona's favourite colour.
And Mona is not involved in any of
the new code I put in, so she
can still go through without any problems.
If we take Ben, we actually have set
up that if it's getPerson, and the argument
value is Ben, we want to change on
the fly what's going on.
So we will hard code here that his
favourite colour is orange with a hint of
black stripes, which is certainly not the real
one, which was green.
So we can on the fly change things
here.
We can just return what the tool should
give back.
We can on the fly change some of
the argument values and the names.
So you can inspect if things look slightly
wrong, and you can on the fly change
it.
You can do that.
So you can essentially just intercept and change
whatever you want in the tool code, because
if we do this, it now says, okay,
we went a bit too long.
So now it comes back and says Ben's
favourite colour is orange with a hint of
black stripes.
So before the internal timeout of the tool
calling inside Agent Framework happened, and it said,
oh, something must be wrong.
It didn't come back.
What happens if we ask about John's favourite
colour?
In this case, I put in that John's
data is secret, so we can either throw
back null, or we can throw back an
exception.
And what we will see happen is that
it says it can't get it.
And this was not because of the exception
or the timeout.
We can try again to see what happens.
What is John's favourite colour?
You will often see it tries it twice.
This is a retry mechanism inside the Agent
Framework that it says, oh, something could have
had a hiccup.
Let me do it twice.
But it won't be let through because I'm
throwing the exception every time.
So this is how tool calling works.
There are other tool middlewares that you add
in the same manner, which we'll cover in
later sessions.
Thank you.