Hi and welcome to this AI in C#
video on the Microsoft Agent Framework.
Today we're going to look into something called
AG-UI, which is the acronym for Agent User
Interaction Protocol.
When I first saw this name I actually
thought it was something to do with Autogen
and their drag and drop features that have
been there for a long time, but it's
actually a new protocol that is very, very
new.
Came back in May, I think, of 2025,
so it's less than six months old.
So when I saw that and found out
it's a protocol, I said, oh, not another
protocol.
We have MCP, we have AG-UI, we have
all these different things.
But the more I looked into it, it
really solves an actual problem in terms of
the UI part of AI, and it does
it in quite an elegant way and a
standard way that I really hope this protocol
will kick on.
Microsoft is embracing it, so that's a really
good sign.
But what is it?
When we build our small console apps here
in this series, we don't have a problem.
We just have some consoles and we can
do normal calls, we can do streaming, we
can do tools and so on, but everything
happens in one process.
In real life, that's not often the case.
Of course, it can still be done, but
often we have a frontend and we have
a backend.
And the frontend would be the chatbot or
whatever we do, and the server have the
backend agent.
Of course, there's also backend systems and automated
behind the scenes things without frontends, and this
protocol is not for that.
But if you have this frontend, backend system,
and you've tried it in real life, you
know that it's quite simple, of course, to
have a backend and you do a request
response in order to get some information.
So you could have the user answer a
question and get a response back.
No big deal.
It's also easy just to couple a bunch
of MCP servers, own tools and such onto
the backend server and have them be able
to be called so the user can just
ask and the server can use its own
server tools.
But where it becomes tricky is when we
need to be live chat GPT and do
a stream request and get a streaming response.
Because how do you do that?
It is possible.
I have actually done it in a sample
application and also in a real application.
But if you sit with some React frontend
and a C-sharp backend, how do you
begin to send server-side events, which it
is?
And how do you make it scalable?
How do you make it easy to work
with?
So it can be done, but it's a
pain.
Another thing is, which I have actually never
thought about, what if the frontend actually have
some capabilities that you want to expose as
tools to the backend?
So just like we have server tools, what
if the frontend could be changed the colour
of something or how something looks, and we
wanted AI to do that?
I actually have really no idea without this
protocol, how you could do that.
But that is what we can do in
a very, very elegant way.
And let's go in and see some code
because there's actually not too much to it.
So here we are in the system and
we are working with this agent user interaction
and we have a client and a server.
So for that reason, I have gone up
and used the multiple startup projects.
So I run both the server and the
client.
And I will go to the server, but
I will just start the code and set
a breakpoint on the client here.
So let me run it and then we
go quickly to the server and see what
that's about.
And here we have the client UI.
Of course, I'm still using consoles, but this
could just have easily been a Blazor app
that would have this problem of talking with
a backend server.
So what happens is we start with the
normal business as usual.
We just make a client and we make
an agent.
And in this case, I make an agent
with one tool call that is called GetWeather.
And then we make a normal web application.
So had this been a standard ASP.NET
Core web UI, we have had this out
of the box.
And the only thing we need to really
do is we need to add a AG-UI
as a service, and we need to map
it to an endpoint.
In our case, just the root of this.
And then we run the system.
And if we do that, we will see
that we are just running as something here,
and we have a local host here.
In real life, it'll of course be HTTPS
and over the internet instead of just a
local host here.
So you can see the server is nothing
other than instantiating an agent and exposing it.
The client is also fairly simple in that
we just make an HTTP client because we
need to talk with the server.
And we do that over the URL we
have chosen.
Then I make something with the colour of
the text here.
Right now you can see it's white.
And then I make an AG-UI chat client.
So instead of a normal chat client, I
make an AGUI client.
And all it needs is an HTTP client
and the server URL.
And once I have such a client, I
can do sort of the same as I
normally do using the agent part, create AI
agent.
And here I can give it tools, which
will be the local tools.
We also gave tools over here.
But suddenly we can give this proxy of
an agent tools that will be handed on
to the server, which is quite a cool
concept.
This agent, however, is not a fully blown
agent.
For example, if we give it instructions, it
will not follow those instructions.
And that's because it's just a proxy.
We can give it tools, but the rest
we cannot really give because it won't follow
that along.
So it's a kind of a shell of
an agent.
And for that reason, for example, if we
want to make an instruction, we need to
give it ourself here.
And in the same manner, it cannot work
with the threads.
So we also need to give that on
our own.
This might be because it's so new a
protocol, and we might at some point be
able to give the instructions and the threads
like normal and behind the scenes, everything would
work.
But for now, we need to do a
little extra here, but trust me, it's worth
doing compared to what we get.
Because what we get, and it can be
perhaps a little difficult to understand if you
haven't had this pain of how in the
world do I get a client to talk
to a server and stream the data back?
Again, can be done, but trust me, it's
damn annoying.
But in this case, we're just going in
and making an agent.
So if I say hi, it can come
back.
And if we noticed there is no API
keys or endpoints or anything here, there's just
the URL.
And the URL can probably also be protected
by normal HTTP.
But in our case, we don't need it.
It's just an endpoint for our client to
work with.
So what we do is a little like
we normally do.
The only thing is we keep track of
all the messages.
Instead of in a thread, we just keep
them.
But then we can suddenly use streaming here.
And streaming, yes, in one process is easy,
just back to console.
But here, it's actually going over HTTP and
sending the server events back.
So if we look behind the scenes, we
can actually see that there is some text
event streams going on back and forward when
I wrote hello.
So okay, we can say then, is that
the only thing?
No, it's also able to expose the different
things that happens on the server.
And it does that using the different content
back from the updates here.
So it can give us back the text.
That's what we're seeing here.
It can give us back any function call,
which we haven't made yet, any results of
those function calls, and any errors that might
be.
And again, we are able to give a
local tool, again, something I would never have
thought about how to do in real life
because I would just have said that's impossible.
But we can actually give this tool and
on the fly when we call here, that
tool is being passed onto the server, given
as if it was something that could happen
on the server.
When the AI wants to call it, it
communicates back to the client, executes the thing,
which is in our case, just change the
colour, and go on its merry way to
work.
So let's see some of this in some
code here.
So I could, for example, go in and
say, what is the weather like in Paris?
And if I do that, it should say
like, not link, sorry.
And now it knows it's just a tool
call and not try to find a link
to the weather in Paris.
So what it did, it made a tool
call on the server, got the response back,
and we could see on the client, it
actually gave information on what it was doing.
So we can make a GUI that indicates,
hey, we are calling this tool with this
information.
Again, something that was very, very difficult to
send using these streaming events.
But this is packed in, so we can
do it very, very simple.
And we get the result back.
And of course, that result from the tool
can be used.
So we wouldn't need to write this out,
but if we want to make a GUI
that is very interactive and telling exactly what's
going on, we can do it compared to
in the past, where this was really a
pain.
In the same manner, we can go in
and then call a local tool and pass
it onto the server.
So we could, for example, say, change the
colour to green.
And now it did a tool call.
It passed the text onto the server.
The server found out it got that extra
tool.
It passed that request back to the client
where it ran this piece of code down
here.
We can set a breakpoint and say, change
this colour to yellow.
So this is happening over streaming on the
fly from a server.
Again, I would have no clue how to
make this with the normal HTTP.
But it works.
We can suddenly give client-side tools.
I could see desktop apps doing something directly
in a system while the AI is remote.
It is kind of incredible, in my mind,
what this can do.
And the only thing that are the drawbacks
is we need to keep track of the
entire conversation, and we need to make the
system call upfront ourself.
The rest happens quite automatically.
And again, we just get these events back
so we, on the fly, can update a
GUI showing how it's going.
So, so much pain and things I have
thought about needing to do and not done
because it was hard can suddenly be expressed
in a very, very simple way.
And again, we don't have any credentials here,
which is also incredible.
So as you can hear, I'm really a
fan of this protocol, and I really hope
it gains traction because this will make the
life of us who also needs to build
a GUI on top of an AI much
simpler.
So with that, thank you for attending this
one, and see you in the next one.