Hi, and welcome to an ART-Sharp video on how you convert Semantic Kernel into Agent Framework.
Lots of people out there right now have Semantic Kernel, it's in maintenance mode, you should
definitely upgrade to Agent Framework when it comes out of preview in start of the year.
So now's the time to have a look at how you can do that.
On this video, I'm going to convert a sample project I have, I have a video of a live upgrade
as well in my channel, but if you want to see it a bit more structured and easy to look
at, this is the one for you.
Because in this, we are going to see how to take normal chats, streaming, structured output,
tool calling, RAG, and everything from Semantic Kernel, throwing the NuGet package away, putting
in the others, getting a lot of errors, and getting rid of them.
So let's get started.
So I have a folder here in my sample repo, and the repo is linked in the descriptions.
So I have a before and a finished.
So this is the one that actually uses Agent Framework, while this one uses Semantic Kernel.
So I'm gonna change this live, but you might want to instead look at this in the end.
So what I'm gonna do here is show you how the thing works, and what code we have here.
So we have two NuGet packages, Semantic Kernel, Agents Core, and Azure AI, and then we have the in-memory.
So these are the three packages we use.
And this is a web API, so we can do a little dependency injection on things here, but it's
not needed, both for Semantic Kernel or Agent Framework.
I have some credentials that I grab from a secret manager, and that's part of any of
the two, and it will remain the same.
And then I have something called a MyKernelFactory, which can build different kernels.
So we have the normal Kernel CreateBuilder, we add a chat completion to it, we add some
middleware in terms of IAutoFunctionInvocationFilter, and we build our kernel so we can give it
to the various objects here.
And then we use the Azure OpenAI Embedding Generator, where we can get the embedding
generator for the Rack stuff.
Then comes six map posts, one for chat, where we try to use the kernel directly, just getting
a question back.
And this happens via a chat request object, which is just a question.
MyKernelFactory is dependency injected here, and we get a response back which is just the
answer back, nothing special there.
We do something similar with streaming here, it's not really streaming because it's just
a map post, but we are doing some streaming here where we go in and just do streaming.
Then we do a map post with chat history, where we use the chat history object, we actually
don't need that one here, and we don't need these, they are just in the way.
We do a chat history, add a question, add it to the history, add another question, add
to the history, and we'll see how that will be changed in Agent Framework.
In the same manner, we have tool calling.
So we have a tool where we import the tool from object, and this tool is down here.
In general, this is in one big file, and you can see there's 273 lines of code.
That's on purpose so we can both tell how many lines of code there are after the fact.
So 273 is our starting, and we'll see in the end how much there will be left with Agent Framework.
But down here we have two tools, they are dummy tools more or less, with no real implementation
of a getWeather, it's always sunny and 90 degrees, and in terms of a getDate and time,
it's really in the real life way.
And the weather result is just a record here.
So in the tool call example, we call this tool, we give the agent the option to do tool calling,
and we just get our answer back like normal.
Then we have a structured output example, where we give our agent its response format,
and this response format is just a movie result with movies inside, where we then ask the
agent to give us the JSON back, and Deo serialize that JSON to the movie result.
Then finally we have the RAG part, where we go through and make some RAG ingestion here.
You wouldn't normally do that in the call itself, but just to keep it simple for the
demo, we ingest all the data, so all of this will not change, because that is not part
of the Agent Framework, we'll get back to that.
We get our kernel, we ask for the RAG, we do a search in the RAG, and we give that data
next to the question for our agent.
So that is all the code there is.
We can run it, and we can see we have a little HTTP file here, so we can test it.
We can ask for the capital of France, it will come back and say it's Paris.
We can ask streaming how to make soup, and after a little while it's been cooking up
the recipe, it will give us back a long answer about that.
We can do the chat history, where we see a complete chat history back.
Not really the real way you would do it, but that was the best way I could do it with just a web API.
Then we have the tool calling, and the tool calling will show that the auto-invocation
filter will happen, and show that the thing was called, and being used to find that it's
sunny, and what the time is in Paris right now.
Again, the input and output is not that important here.
Then we have structured output, where we can see we get our movies back, and we have
a rack, where we ask for what is the Wi-Fi password, which was one of the ten things.
So doing rack search, finding that, and giving us back what the Wi-Fi password is.
So all good and well, now let's do the conversion.
And in order to do that, we of course need to go to our NuGet packages, and get rid of
some, and add some.
So first we will get rid of agents core here, we don't need semantic kernel agents.
And we don't need the Azure OpenAI of semantic kernel.
We do however need this one, because this is actually just semantic kernel by name.
In real life it's actually Microsoft extension vector data, and at some point Microsoft will
actually rename this package to vector data dot in memory instead.
But right now it's just confusing, it's called semantic kernel, it really has nothing to
do with semantic kernel other than the source code is in their repo.
So we don't get rid of that, but what we get is we take our Azure dot AI dot OpenAI.
This package was behind the scenes in semantic kernel, but now we control it ourselves as
the raw connection.
The other package is the Microsoft agent framework package, which is the OpenAI package we need.
So let's put that in as well.
So two packets removed, two packets added.
And once we do that, the code will be very sad.
So we get, there will be more errors in one second.
There's actually more, there's like 42 errors.
So let's just start from the top, let's get rid of stuff that doesn't exist anymore.
Let's get rid of this, let's get rid of this.
We don't need this one either, because that was just a warning from semantic kernel.
So what we first need is we actually need a builder dot services dot add singleton.
And we need a singleton for the Azure OpenAI client.
And this might feel strange for a semantic kernel person that we need to do this on our
own, but it will make much more sense down the line.
So we get that in, and then we don't need this kernel factory anymore.
So let's just delete it entirely. There we go.
And get rid of its registration, because what we are going to do is, every time we have
a kernel factory here in the different methods, we instead want to do an Azure OpenAI client of type client.
So let's replace those in all the six methods, so we don't have this anymore.
But because it's dependency injected here, we can do this in the various map posts.
We still need embedding generator, but we need to new it up in a slightly different way.
We need to say add embedding generator, and it will be constructed on the fly.
And we do that by checking our provider.getRequiredService.
And we need the required service of the Azure OpenAI client. There we go.
So I could have made it out here and given it in, but let's do it in the proper manner.
So on that, we need to get the embedding general client, and that client should get the text
embedding small.
And then we need to say SI embedding generator.
And that one we can just return, and we have our new embedding generator.
So slightly more difficult to set up this part, but a lot of other places, it will become much simpler.
So let's tackle the first map post here.
We have a client now instead of our factory, and we can say get chat client, we get this.
And then we can say create AI agent.
Once we do that, we get an agent.
And an agent can, of course, give the answer back as well.
We just simply say agent.runAsync with our chat, our requested question.
That's a wait, and we get a response back.
Instead of all this, we can simply just say we want to give our response.text back.
This is the text it will give back to us.
So first one done, and I'm just going to copy paste this, because we will use these for
all the other ones, and I will just put one into each one of them, and then we will modify
it slightly after that.
You could, of course, make a helper method to give such an agent, but I like it being
on an individual basis, so I can have different models for different scenarios.
But back to number two here, which was our streaming, and streaming is fairly simple as well.
We can simply say agent.runStreamingAsync with our chat, request.question.
It also needs a wait for each, and give an update back.
So we can take our answer, say answer.equal.update.
This is the streaming update.
And once we have that, we can simply just give the answer back.
So number two is done.
Next we have the chat history, and that chat history doesn't really exist anymore, but
what it does here in our agent, we also have an instruction, and those instructions we
just give to this instructions, and we can give our instruction up here.
So we can get rid of this, and instead of a chat history, we say agent.getNewThread,
which is a new name for chat history, being a thread.
So we don't need our history.
We don't need to give the history our first question.
Instead we can simply just say agent.runAsync with the chat, request.question, which was
who is Barack Obama, get our response number one back here, and we don't really use it
for anything here, but in real life you of course begin to write it out, because we just
wanted to simulate a follow-up question, and our follow-up question, in order to work,
we need to give the thread in here on the various questions we want to ask.
So we want to ask us a follow-up question, how tall is he, to get our response to.
We can get rid of all this code, and we can just say that the thread here, because it's
an open-ended system with multiple versions of threads, there's multiple of them, but
we know in our case it's a chat client agent, so it has a chat client thread, and when we
get one of those, we can say chatClientThread.messageStore.getMessages, and we know that this is not null,
so we can await this, and get our messages back that has been asked between them, because
the thread will take care of putting the input and the output, the input and output, into this,
and we can go in and just return the messages, because they have the same structure back.
On to the tool calling, so we have our agent again, that agent needed some instructions,
so we just take the instructions, the old one needed a function calling behavior, so
it would actually call the functions.
By default, Agent Framework could do it, so we don't need to, but instead we need to give
the tools, and the tools is just a collection of AI tools, as they're called.
So, we need our tools out, so we can work with them a bit here, so let's put them up here.
This is our tool class, not the tools themselves.
Mr. Right buttons, and down here we need to say we want an AI function factory to create a new tool,
which can take a delegate or method info.
In our case, we just have two tools, so it's easier to just say, let's get the two tools in,
but you could also use reflection on the class itself in order to get your data.
So, down here we want the name of the tool, and we don't need the kernel function anymore.
So, we put that in, and we make one more of these, one for the get city and date,
and we take the name down here, and get rid of this, so we have the right names here.
So, now we have our tools in place.
We don't need our kernel, we don't need to import, we don't need to make this agent.
All we need to do is answer back, so we don't need the answer here, we just say agents.runAsync,
and we ask our question,
and we can await the answer.
And once we have that, we can just say response.txt.
We do, however, need to have the function calling middleware, the invocation part.
We had it down here, and a slightly different way of doing it,
where we had the context and call and stuff.
Like normal, we can get rid of that, and I will just bring in the new way of doing it.
It's something you, in real life, will never really remember,
but just take from a sample or the documentation,
and it needs to be put in here, so we can see it.
It's just a basic function called middleware.
We can get the calling agent, we can have the context,
we can stop the next if we want to, just like the other one,
with it just a slightly different nature.
And what we need to do is we, of course, need to have the agent know about this middleware,
so the way you do that is you take your agent and say
asBuilder.use in order to get middleware,
so we can take our function called middleware, and then we can build the agent again.
And now it turns from a chat client agent into something called an AI agent,
because it's a more lower level fraction, but once we have that,
everything works again.
Then we have the structured output. We need to do something similar.
We saw that our agent had some instructions, so we need to put those up.
Instructions, and it was about asking movie questions.
We then told it the response format.
In agent framework, we don't need to tell the entire agent that they can only do that.
It will be done on a call-by-call basis,
so we don't need to do this at all,
and we don't need JSON or anything.
Instead, we just need to say agent.runAsync,
but with this version that takes a movie result.
And then we have our chatRequests.question, get our weight,
and we get our response back.
And instead of doing the call here and doing the deserialization,
we can simply say response.result, which will be a movie result
because of the generic nature of this.
Finally, we need the rack setup, so all the knowledge here will stay,
but we just have our agent. We can pull it down here
so it's a little easier to see where things are happening.
So we need our agent. The agent had some instructions again,
so let's skip that.
But beyond that, we don't need any of this code.
We still need to do our rack search up front,
and then we need to call our agent, so we can say agent.runAsync,
and we need to give it the question with the augmented part of the rack,
like we did before.
We say a weight, we get our response like before,
we get rid of all this, and just say response.text.
Once we have that, we have thrown away all the things we didn't use,
and we should be ready to go.
So if we run this, let's see.
The code is running, and again, we can try and use the same test over here.
We asked what is the capital of France for semantic kernel,
and now we get the capital of France from agent framework instead.
We did it with streaming on how to make soup.
Let's take a little longer to answer like normal,
and we get our entire soup recipe back.
We had our chat history where we got the entire back and forward,
and now we get it back as an agent thread instead with the same format.
We have the tool calling,
and the tool calling can see again that our middleware was called on the fly,
and we get back that the weather in Paris is sunny and 90 degrees,
as the tool always told,
and since it's usually sea time 12.40 right now,
it's 1.40 in the afternoon in Paris right now, which is correct.
Then we have the structured output,
which now comes back in exactly the same format, just with agent framework,
and rack where we asked for the Wi-Fi password is again exactly the same.
We get our Wi-Fi back.
This is actually everything that is to it,
and we saw that we had 273 lines of code or something like that,
and now we are down to 202 lines.
So roughly 70 lines of code can be saved using a better,
more condensed agent framework compared to semantic kernel.
So you also get more concise, better named, in my opinion,
easier to work with, especially around structured output
where you on the fly can ask for the data.
So overall, quite easy to do.
Let me check my time here. It has taken 27 minutes to do this live,
including introduction and telling what the code does.
So it should be fairly easy for you as well to go from this.
And as mentioned, what we just did is down here in the finished version.
I will undo this just after this,
but all exactly the same we did before is here as well,
so you can see it and use it for your own migrations.
So good luck with those, and see you in the next one.