Hi, and welcome to this AI and C-Sharp video on the Microsoft Agent Framework.
Today we're going to dive into a topic that has changed a lot
during the RC1 and RC2 releases of Microsoft Agent Framework,
and that is the AI Contact Provider, or AI Contact Providers in plural now,
because we have the option to have multiple of them.
But it has completely changed since the preview,
so in this video, we're going to do a really, really deep dive
on what it can do, what is all the methods you can override,
why are they there, and so on,
in order for you to better understand everything here.
So in the samples, I have an AI Contact Providers deep dive.
I've made a folder for this, and I've taken my two previous videos,
one on how to do memory, and one on how to do on-the-fly tool calling,
and put them up in here because they're all three using the AI Contact Provider.
So I'm going to primarily work here, and then I will show you
how these have changed a bit as we go along.
But what I have is a completely normal agent here,
and right now, I have no Contact Providers.
Before we got the RC1, this was called AI Contact Provider Factory,
and we can only give one option, and we needed to inject a bunch of odd things.
But now, it's just that we can view our classes.
But let's quickly run without, just for you to see what it is I'm running here.
So I have a completely normal chat loop, and if I say hi,
I'm printing out the raw HTML, or the LLM call directly,
so we can also see what is going on.
So when I say hi, the LLM call happens, the system message is being added
because I put a system message here, saying we need to wave in the prefix,
the user message sent, and the final response is here.
So this is pretty normal, other than I put in that we can actually see
what the LLM request is.
So now let's put in our AI Contact Provider.
And an AI Contact Provider is just a class that inherits from AI Contact Provider.
And it has four methods you can overwrite.
Invoking call, provideAIContext, invoke call, and storeAIContext.
And the order they are called in is that the invoking call
actually is in charge of calling the provideAIContext,
then the LLM call happens, and then the invoked happen, and store.
So in real life, there are four of these, theoretically,
but when you are going to use these,
you will essentially never use the invoking and the invoked.
Behind the scenes, these two have some very, very advanced things
of merging messages, because there can be multiple of these.
But in reality, you will stay here.
This is the user version, while this is a more technical version.
And I will argue that if you know when to use this,
you probably don't need this video.
And in the same manner, if you were to use this,
where you can go and overwrite how error handling works
when something goes wrong in the LLM,
you're probably not going to need this video.
So we're going to focus on these two ones here.
ProvideAIContext, which happens before the LLM call,
and storeAIContext, which happens after the LLM call.
But you can see I put in messages here,
so we can see they are all being called, and we'll see that in one second.
But what are we using this for?
Well, whenever an LLM call is about to happen,
we can interject things using this method.
And what we have here, let me set a breakpoint so we can try it out.
So if I say hi,
you will actually see that the invoking callAsync,
meaning this method up here, have already been called.
But right now, I don't have any code in here other than the message.
And again, we're not going to use this.
But in here, we get a context.
What is this context?
The context contains our agent.
So if we need to get the name of the agent, or the instructions of the agent,
or something here, we can get it from the context.
We can also get the session.
In my case, I put in session on this call.
If we didn't have a session, it would just be null.
But we have, in this case, our session we could go in and work with.
And then we have AIContext.
And this is right now what we have in the instructions in the system,
and what messages that has been sent,
meaning our simple message of hi.
And then it shows which tools are available right now.
And we can see there's no tools.
I'm not using tools in this sample.
So what we can do is we can go in and inspect this data, of course.
So that's nothing special here.
So we're just gonna do that.
What we can do here is we can go in.
Let me restart the code so we can see it again here.
Let's say hi again.
So we gotta get in here,
because what we can get out is the message itself.
And now you can argue, what do we need this for?
Well, we can go in and inject additional instructions.
We can inject additional tools.
And we can technically inject additional messages.
But if we do that, they will become part of the chat history.
Because if we see here,
this only happens for the LM call this one time and nothing more.
It's not like it's being appended directly to the instructions
again and again and again.
So in my case, I'm injecting some new instructions.
That is, speak like a pirate.
And that's what I'm sending back from provide AI context.
So the AI context I'm providing is speak like a pirate.
And if I run this,
we can then see now that I was called with invoking and then provide.
And we saw we just read out the message.
Then we can see that the messages, the system,
now have the prefix or messages with our waves,
but also speak like a pirate. So on the fly,
Agent Framework added this to the context.
But it's not like if we say hi again,
it only adds it one time.
It's not like it's saying speak like a pirate, speak like a pirate,
speak like a pirate.
So it is taking the original instructions
and appending what we put in for that one call.
And then if we set some more breakpoints down in invoke,
which again we don't really want to use for anything, and our store,
here we could go in and begin to store it.
So we could extract information from the response
or deal with exceptions in a different way than the default,
which is just drawing the exception.
So if we say hi again,
we can now see that invoked happened.
And if we just go down and see that invoke will call store,
we will be able to see that in context here.
We can get the agent and the session as before.
We can check if there's any exceptions.
But else we can get the request messages,
meaning what we have set,
but only for that one call,
not the entire message or chat history.
If we want that, we need to take that from the session.
And we can also see the response message,
which is what is about to go back to the user,
but haven't been shown yet to the user,
meaning we are still in the middle of the run async right now.
So if we look at that,
we can see we got invoked call async,
the store AI context.
There was no exception.
There was a request message of hi,
and a response message of advanced here,
which is the final response down here.
So that is how one AI context works.
And let's see a few of the examples I have used it for,
for context on why we want to do this.
So over here in my memory system,
we are putting in a contact provider of a custom context provider here,
where we're sending in an agent and a user ID.
And over here, we're just storing those
and saving some user ID,
which is our memory store.
That could have been a database, but right now it's just a text file.
And then we're reading all existing memory.
Again, this is a previous video,
so you can go and check that in more details.
But the main thing is that whenever we provide AI context,
we are taking that user fax and putting it into the memory,
just like we put in Speak Like a Pi before.
Now we're putting in, for example, if it knows my name,
my favorite color, and stuff like that.
And in the store AI context,
in that case, we are actually taking the request message,
and we are using an AI to figure out
what I have told the AI, my favorite color is blue, and so on,
and extract any memories I want to add or update or delete,
and save them into the file.
So we are essentially just doing a pre and a post,
pre being put in the user fax,
post being extract new user fax after the fact.
So that is how we use it for memory.
If we want to see how we use it for tools,
it's in a similar manner.
So down here we add on-the-fly tool injection.
Again, we are putting in an agent
because we want to use AI to do something about this.
And then when we are providing AI context,
we are actually looking at the message
before it goes to the real run async,
and extracting, in this case,
do we need to use some time tools, some weather tools,
or some file system tools,
and we end up with a response.
And based on that response,
we make a new context back on which tools we want to inject.
So if we need time tools, we inject some time tools,
some file system tools, or some weather tools,
and the injected instruction is just something for the file tools
and how it needs to work.
So here we are on-the-fly putting in tools.
Again, there's a separate video on this with much more details,
but essentially, again, it's just that before the LM call,
we add more instruction and tools.
In this case, we're not caring about after the fact.
Here, it's just pre-we're-going-to-do-something.
So it's a very, very open-ended system of doing this.
In my case, over here in the basic,
we're just injecting Speak Like a Pirate.
But what if we have multiple of these?
So I have my context provider 2 here as well.
And it's almost the same thing, but it only says,
instead of Speak Like a Pirate, it says,
suffix response with three robots.
But else, it's exactly the same.
Again, we're not storing any context, we're just reading it out.
But again, we could have taken whatever the AI came back with
and done something with it.
Made a summary of what we have talked about so far, for example.
So if we run it with two...
and say hi again,
we can see, let's take this away,
that we call the first AI context provider,
then we call our second context provider.
They're getting exactly the same information.
And they provide each one some data,
Speak Like a Pirate, and put in suffix with the robots.
And that is all being put into the system message.
We have the original, we have the Speak Like a Pirate,
and we have the suffix.
Beyond that, nothing else has changed in DLLM.
And we get our messages back,
and now we have that we have instructed it
to get the robots at the end.
So we can put in as many of these as we wish to
in order to do memory, tool injection,
whatever we want to do.
And we can reuse these across multiple agents,
so if we wanted for some reason to have the wave thing every time,
we would just have a wave before you answer IA context provider.
There's also a third type of context provider,
and it's called a message context provider.
So instead of AI context provider here,
we have a message context provider.
This one is a bit strange compared to the other one
in that it has both a provide AI context and a provide messages.
This one can actually provide messages, but they are permanently messages.
This one is on the fly extra messages you want to put into the system.
So in my case, I'm using the provide messages,
which is yet another way of calling this.
So we can go in and say, for example, make all text uppercase.
So if we do that and say hi,
we will now see that we call our first provider,
which is speak like a pirate, second provider,
put the robots at the end, our third provider, uppercase everything,
and in our case, it is putting in the ones that does things with instructions,
and then it's adding one more message here, make all text uppercase.
I'm a bit surprised that it puts it after the original message,
but they must have evaluated that that is best.
If I had made this implementation, just probably put it before,
and then so the real user message would be the highlighted one,
but they have chosen to put it after, and AI can understand it.
The messages comes back to all three providers,
and we now get all our instructions put together,
plus the extra message that gives even further instructions being uppercase.
So now we have three of these,
but there's one more way of using the message providers.
You can't use these for that for some reason.
Perhaps that will change, but you can also make this as middleware.
So middleware is where we do the tool calling middleware,
the OpenTelemetry middleware,
but we also have an AI context provider's middleware now.
So this one can only take message context providers.
It cannot take my AI context provider.
It will say it's a wrong type. Okay.
So if we do this, things change a bit.
Let's get rid of... let's keep these two, but keep this one down here.
Because when they are run like this, and we say hi,
we essentially end up with the same scenario,
but we are now calling for the provider.
We can see that happens before everything else.
So it's like this one is going before the ones we get up here.
And it's also calling another invoking version instead of the context,
because we cannot provide context for the myMessage right now.
I don't know if it's a bug or if it's a feature,
but they have chosen to do that.
Them being Microsoft Agent Framework team, of course.
The thing is essentially the same. We get the exact same setup.
But why would we ever want to do it down here?
Well, this down here is kind of okay that we can do this,
because, for example, if we get an agent that we don't define,
we don't have the chance to put in AI context providers.
So, for example, if we use the Google's A2A protocol,
the agent-to-agent protocol, we're just being given an agent,
but we're not being able to give it any kind of options,
meaning we're not able to put in our providers.
But down here, we can certainly put in providers,
but only providers to put in extra messages.
And that's probably one of the reasons why we don't get the context,
because they can't on the fly inject system messages,
but they can at least send extra messages like this down here.
So, it sort of makes sense that we can't inject instructions down here,
but only messages.
But that's the main reason. If you own the agent,
you probably don't want to use the middleware parts.
You will just put them in as your own providers.
But in those cases, it's a nice thing that we can actually inject them
by just putting in messages, because that is under our control,
no matter how the agent has been created.
So, that is actually everything there is to say about context providers.
They are kind of intimidating, but end of day,
you should just see them as a place to put something in before,
adjust on the fly instructions and tools,
and after the fact, do something.
So, again, my best two examples is the memory part,
where we can on the fly put in memory,
and on the fly extract it again,
and the tool calling injection, where we can on the fly
determine what tools to put in,
so we can save tokens not giving every single tool.
There's probably many other ways of doing this.
My agent framework themselves have used it to make agent skills.
I have a different video on that,
and I will compare my agent skills to their agent skills,
but they're actually using the AI context providers
in order to inject those tools on the fly.
Kind of a cool idea, but I find it a little too limiting what they do,
but that's for another video.
So, for now, I know this is a bit advanced,
but we needed to go deep dive in order to understand this,
and now that it's logged in and not changing a lot,
it made sense to do this now.
So, thank you for listening. See you in the next one.