In the last lecture, we saw the life
of an LLM code.
And I mentioned that there was some extras
thing about this.
So let's quickly go through how you can
turn on to see these requests and responses
live.
So what you need to do is you
need to make a custom client HTTP handler.
So an HTTP handler that looks like this.
And we'll go down and see this.
And we would need to put that into
an HTTP client so it can control all
the handling.
Normally, handling will just be send and receive.
But we can actually go in and override
what's going on in the send method.
And when we do that, we need to
put in the transport object here.
Where we get a chat client pipeline where
we can put in this special HTTP client
that we control.
So that is what happens before we make
our call.
And then we have our custom client handler.
So it's just inheriting from the HTTP client
handler.
And we are able to override the send
async.
Meaning whenever behind the scenes in agent framework,
there's an HTTP client that says send async.
Send some post call or get call and
so on.
We can intersect it here.
So if I set a breakpoint here.
And this is, again, what is the weather
like in Paris?
So if we set a breakpoint here and
let it run.
To that.
We end up hitting this.
And if I press F5, we will end
up here.
Because now agent framework have taken this and
deep down underneath run async, there's an HTTP
client calling the LLM model.
And in our case, we can see that
request string.
And the request URI.
So this means this is what we are
calling.
And here is the request string.
And it's a long string as we saw.
And because it's long out here, it looks
like it's not as long as it was
before.
But it's exactly the same data with the
messages, the model, the response format, and the
tools.
So we get that.
And I'm just making it pretty, JSON serialising
it.
And then we actually do the send.
So here we can control what's going on.
So we send the request.
And we should get a response back once
it's thought about it.
And we got a status 200.
OK.
And we can get the response, which was
also what you saw.
Again, in a little less pretty way of
looking at it.
But we can see that our city equals
Paris is here and so on.
And we can also make that pretty and
let it run.
Because now the second call comes along where
we need to actually use the tool.
And if I just let it run, we
can see in the console what happens.
So we call it the first time.
Let me press the right buttons here.
There we go.
So we saw the call that we saw
in the previous video.
We see the raw response back.
We see the second call.
And the raw response back.
So that is how everything works behind the
scenes.
So I use this quite often when I
need to do some very advanced stuff.
And I can't make it work.
So I can go in and see, OK,
have it actually sent this special extra parameter
I wanted to do or not.
So for very nerdy stuff, it's really, really
interesting that you can do this.
And you can even go in and on
the fly change the requests so you can
do some very, very advanced stuff.
But that's beyond the scope of this.