Hi, and welcome to another AI in C-Sharp video on the Microsoft Agent Framework.
Today we're going to do a deep dive into chat reducers
because we have made a previous video before, links in the description,
about the two built-in chat reducers.
But what if we want to do something different
than just reduce the number of messages or do a simple summary?
We are able to make our own custom reducers,
and that's what we're going to cover today.
So we have the previous sample here, chat history reducer,
and the link to the video about that,
where we went in and used the message counting reducer
and the summary reducer, meaning four messages,
get down to four messages again,
and summary after four messages, summarized.
But now we're going to go in and see not the built-in ones,
but the built-in ones, as you can see here,
is just an iChatReducer interface.
So we can make our own, and then the first one I'm going to make here,
or let's run it first so we can actually see
how it works without any chat reducers.
That might help a little with the understanding of this.
So if I say hello to it,
this sample will just write out the answer,
how many number of messages there were,
and who wrote them, user or assistant.
And I also put in a tool so we can see tools in this.
What is the weather like in Paris?
And because we have a session on,
we can of course see that both say hello, hello back,
what is the weather like in Paris,
then we get the tool call for get weather,
and the tool result back, and then the final result.
And if we keep doing this,
we will see that this just builds up.
And this is of course good for the AI,
because now we could go in and say,
what cities have we talked about?
And it would come back and say,
we have two cities, Paris and Copenhagen.
And it can only do that because it has the entire history.
But there could be reasons for getting rid of this history,
mostly tokens and latency,
because the more we send, the more there will be here.
So the first one I want to introduce you to
is one I made all back in Semantic Kernel,
but I have now built here as well in Agent Framework.
And that is one that always just removes all tool calls.
So the way you do an iChat reducer
is you just inherit from it or implement it,
given that it's an interface.
And the only thing you need to implement is a reduce async
that takes in the previous messages and a cancellation token,
and spits back what messages should be kept.
So any messages we take out of this and put back in
will be used and the rest will be gone.
I'll put in a little utility just to see that it's been called.
And then what are we doing here?
We are making a list of the messages we want to keep
from the previous messages.
We are going in and say if that message is of role tool,
then we get rid of it, meaning we just skip it,
don't put it in to keep.
Or if the message is from the assistant
and it has a tool call request,
meaning a function call content,
then we also get rid of it.
So let's see that as an action,
in that we set a breakpoint here,
and we go up and say that instead of no reducers,
we want to use the always remove tool calls reducer.
So let's run.
Put this side by side so we can see.
So if I just say hello,
we can see it's called a reducer,
and in the first case, because this is the first message,
our previous message is actually zero.
So technically it makes no sense to go to this
because we just get to keep zero out of zero.
And it will come back. Hello, how can I assist you today?
So if I say, how are you?
Again, we are called here.
We get now two messages,
so we don't get the third one, how are you,
but we only get the two prior to what we're just about to call.
And again, this happens before we get a message.
So the two ones we have here is hello and hello back.
So we call again the tool, and what to keep?
Well, the first message is just hello,
so it shouldn't be removed,
and the second message is also just a response back.
But it's your logic in C-sharp,
so you can do pretty much anything here.
You'll see a few other examples.
So it came back, I'm just a programmer.
So now we can, for example, say,
what is the weather like in Paris?
And for context here, I just gave the agent a tool,
get weather, which just say it's sunny and 90 degrees,
no matter what city it is, just for testing.
So if we call this, again, we hit the reducer,
we have four messages now, the hello and how are you,
and we're still not going to remove anything,
because the tool calls have not happened yet.
So it always happens prior to the call, not after the call,
because that could essentially remove some of the new stuff.
So we're going to them, we see that we keep all four,
because none of them are tool-related.
And it comes back, and we can now see we have eight messages in here.
So we saw the four before, and then we said,
what is the weather like in Paris?
We got a tool call for get weather,
and we got a tool result back for what it was,
and then the assistant came back and summarized these two things.
So now we can try and say, what about Copenhagen?
Meaning the weather in Copenhagen.
So now we get our eight messages,
but now we can see that at some points,
let me just set some breakpoints here,
we get a message that was the tool call,
meaning it was this one here.
So where it asks, I need to call the get weather tool,
and we're now just skipping that,
and we also get the tool back, meaning this one here,
and we're also skipping that.
So if we set a breakpoint up here,
we now have six messages we want to return,
and that is the hello,
so the back and forward that we have really seen,
but the tool call is now gone.
And we need to remove both the assistant request and the tool,
else the AI will actually break down and say,
there isn't a matching pair of request and result of this tool.
But if we do this, and you can just copy this
if you want to have this,
we now see that that is what is being sent prior,
and we get rid of the tool calls up here for the ask of Paris,
but we of course see the new tool call down here
for the new message.
But when we send the next one, it will remove those two as well.
So this one will always remove all tools,
but the tools that were used in the last sent message. Okay.
Let's try something else.
Let's try to just make a thing
that gets rid of the word sunny in messages,
because they don't need to be as big as all this up here.
Here, it can simply see to remove all previous messages
that contains the word sunny in the invariant case.
So if we do that, let me get rid of this one, so we can see.
Essentially, we're just taking the previous messages
and filtering them a bit.
That could be filtering for some words that shouldn't be there
or out-of-date information or whatever it is.
But if we take these messages with word reducer
and set it to be the chat reducer, and restart.
So now we can again say, what is the weather like in Paris?
And we will get our four messages,
the user message, the tool call request,
the tool back, and the assistant.
And we can see that it mentioned the word sunny.
So if we say, how about Copenhagen?
It might surprise you what you will see back.
We will see that we have seven messages.
We have the user.
We have the assistant.
We have the sunny and 19 degrees.
But we said we wanted to get rid of those.
Well, down here, I only... Let me find it.
I only set the one that had text in them.
And this is not text, this is content.
So if you want to get rid of everything that has anything to do with tools,
you first need to check each tool if they mention the word sunny.
So the only thing that is gone here is actually the first reply back.
The weather in Paris is sunny and 19 degrees. That is gone.
And now we only have the Copenhagen weather.
So it's a bit contrived, but just to show you that
whenever you do this, it is your job to get rid of whatever it is.
And in this case, text is only the one
that actually gets rid of the final assistant message.
Let's try another one.
We can also go in and use agents within the chat reducer.
So in this case, I'm making a pirate summary reducer agent.
So it's a little like the summary chat reducer up here,
but we get more control. We can use tools and so on.
We can technically do that up here,
but it's not with the agent framework,
it's with the underlying Microsoft Extensions AI.
So in this case, I'm just making a pirate summary reducer
and telling it, giving the input messages,
make a summary of them in the voice of a pirate.
And I'm saying it only needs to happen after four messages.
So let's take this one, and then we'll see the code in one second.
So we'll just go down here and set up.
There we go.
So we will try again to say,
what is the weather like in Paris?
We see that our tool is being called,
but we will just say no summary yet
because we only have zero messages
and not the minimum four before we actually do a summarization.
We can try again and say, what about Copenhagen?
We only have four messages,
and in this case we have set four or lower,
so still no summarization.
So we now have eight messages
with the message from Paris and a message from Copenhagen.
Let's say Berlin.
We now have eight messages,
so we have one to do summary.
And we put in an agent, so we can just use that agent
for any kind of thing we want.
In this case, we gave it instructions to agent
that given the input amount,
summarizing the voice of a pirate.
That given the input amount, summarizing the voice of a pirate.
This is just an agent framework call like any other.
So it will come back and, in this case,
summarize those eight messages.
Now we're only returning one message back.
So we have essentially taken the eight messages,
turned it into one message.
So we will see that we have five messages.
So we have the summarized message,
we have the Berlin message,
we have the AI requesting the tool,
and the tool response,
and back what actually happens.
So that makes all sense.
We normally get four messages every time we ask.
One input, one output,
and the two tool calling request and response.
The fifth one is now our summary.
And given that we have a summary now,
we can go in and say,
what cities have we talked about so far?
Again, we get five messages,
which is more than what we want,
so we actually do a summarization of this prior.
So it will talk about all the cities and so on.
We will send that.
And it will come and say,
the cities mentioned so far are Paris, Copenhagen, and Berlin.
And now we're down to three messages
because we only got the summary,
the question, and the answer,
because it didn't need to do any calls to the tools.
Had we asked, the city mentioned so far,
what are their weather?
We can try and do it.
Now we again do a summary.
Now we're down to three, so we don't do a summary, of course,
because we are below the four.
And now we see a bunch of extra tool calls
because it grabbed three tool calls,
one for Paris, one for Copenhagen, one for Berlin.
And the prior got the same message back for all of them
because it's just hard-coded.
But that makes sense that we can get multiple backs,
and now we get the summary again, and so on and so forth.
Of course, summary will cost extra
because we do both in cost and time if we do an agent,
but you could do pretty much anything here.
And I've tried to do one more of these
where I do a city reducer agent.
So given the input number messages,
return the numbers of the messages that contains a city name.
So this one, let's put it down here and have a look at it.
This is a bit more contrived, but just for you to see.
So if we have no messages, we don't need to do anything,
but if we do have any messages,
we can go in and first use one of the other reducers,
meaning I just get rid of all the tool calls on the fly
by doing reduce down here.
This is the best way of chaining it
because you can't chain them up in Agent Framework yourself,
but you can easily just new up one of the other reducers
and quickly call reduce before you work with the data.
And then I'm building up a message of all the messages in string
and send them into the system,
so it can give me back what numbered lines are to be containing a city
and then get rid of it.
So the best way to understand that one
is probably to set a breakpoint here and restart.
So if we say, what is the weather like in Paris,
we of course are called, that's zero,
so we don't need to do anything,
and push it back.
How about Copenhagen?
So now we see that we have a user message, of course, that is Paris.
We have a couple of tool calls,
and we have the word Paris here again.
So if we do this, try and quickly stop the video
and think what would you expect comes back.
Okay, let's try and do it.
So we have four previous messages,
so we say we will always get rid of tool calls,
meaning we call that one.
Let me quickly get out of it, so we go down here.
So now we have just taken the four messages
and reduced them to two.
And that is the original one and the real answer.
So the tool calls are gone.
And now we build a small string builder here.
So what we are saying is, zero is this line, one is this line.
Then I'm doing structured output to tell,
give me all the lines that actually have a city named.
In this case, it's both the line index zero and index one.
So given that both of these contain the word Paris,
we actually remove all messages.
And that results in, if we have a look,
the weather in Copenhagen is currently sunny
with the temperature of 90 degrees.
So the reason why it knows this is because it,
what about Copenhagen?
And it had a getWeather, but it had no context of weather.
So this is just the AI being clever enough that
what about Copenhagen is getWeather.
But had we said hello prior to this,
and then ask for this, let's try to do it.
Let me get rid of all these breakpoints so we can just see it.
So let's say first hi to it.
That comes back.
What's the weather like in Paris?
That comes back, which is okay.
And what about Copenhagen?
Now we can see that the hi is there, the how back,
and then we get to what about Copenhagen.
And now you can see it's a bit unsure.
Are you talking about the weather or is it something else?
Because the context was lost because all the messages
that had a city name involved was removed.
Again, a bit contrived, but again, the primary message here is
you have full control to do anything you want in here.
You just get, here are some messages,
which of them do I want to keep?
You can use AI, you can use regex, you can use common sense,
you can use counting, anything you can do to your imagination,
you can do here in order to get rid of that.
So that's everything. See you in the next one.