Hi, and welcome to this AI and C-Sharp video on the Microsoft Agent Framework.
Today we're going to make a review of the harness features
that have just been released in Agent Framework.
So let's get into a little about what it is and then see some code.
So what is an agent harness?
A harness is just a fancy word for taking an LLM model
and putting instructions and tools on top of them.
And especially an agent harness,
you might know them from things like GitHub Copilot CLI.
This is actually an agent harness where you have some instructions,
meaning the system information on you need to write code,
and you have some tools for reading files,
writing files, finding files, and so on.
And similarly, you could choose codecs,
or you could use the GitHub Copilot inside Visual Studio Code.
Cloud Code is an agent harness.
And this is what the Agent Framework team have put into their system.
So let's talk a little about the code and the three samples they have.
Because I'm going to use their samples,
because this is a lot of code,
and I'm also a bit unsure why we need this sample in the first place.
But let's talk a little about the code first.
So inside, we can go in here,
inside the Microsoft Agent Framework's own code,
they also have a bunch of samples.
So this is the thing that I'm looking at right now.
So they have three samples, they have a research sample,
they have some research with sub-agents that goes in and do some stock analysis,
and some data processing of some sales data.
So an open agent tool, and then two other tools.
The second tool uses sub-agents.
And besides those three, there's a bunch of extra code up here.
And I mean a bunch, because inside these three is what is needed
in order to build something like GitHub Copilot.
Not as fancy, but of course it's just a sample.
But there's over 4,000 lines of code up here.
And I don't want to take and recreate that,
so that's the reason why I'm going with their sample instead.
Besides the sample data, which is the bulk of all this,
there is some new things down in the Microsoft Agents.ai package,
which is under a Harness namespace,
where there's a bunch of new AI context providers.
And a little middleware.
So there's things for setting an agent mode, plan mode, or execution mode,
like you might know in coding agents.
There are some tools for file access,
there's tools for memory and for storage of files.
There's spinning up sub-agents.
There's a to-do item, just like you see in GitHub Copilot,
where it makes a, now I need to do this, now I need to do this.
And then there's a tool for, hey, approve this tool,
and don't ask me again, system.
And then there's a new NuGet package itself,
which is Microsoft Agents.ai Harness,
where we have something called a Harness agent,
and some options for that.
And I'm finding it a bit strange that Harness is both in the raw package,
and have its own package.
So I don't really understand why this code is inside here,
and not in the Harness package,
because that's where it belongs, in my opinion.
But I'm not having the insight into what this is about.
But let's go in and see this.
Because if we look into the research sample, which is the first one,
what we can see is that we use just normal Azure OpenAI,
and we use a model like always.
And then we go in and set some token limits,
because it can actually do quite a lot of tokens,
and has long-running background work.
And then a bunch of instructions on how to do plan mode,
how to do execution mode.
I'll let you see this on your own.
And then we make an agent,
and we'll just use OpenAI client against Azure.
And we're going to use the responses client,
and then we use the special one as AI chat client without stored output.
This turns off that it's storing the conversations up in Azure OpenAI,
and instead just keep it in memory.
And then it has this new asHarnessAgent.
And that might look like a fancy extra thing,
but it's actually not too much what's happening inside this one.
The only thing it really does is that it sets up the default compaction.
So it's essentially a chat reducer that's built in.
It sets up an in-memory chat history provider.
And beyond that, it doesn't do too much.
It does a little about function call invocation and message injection.
So a bit of setup, but nothing major.
You can see there's almost no code here in the agent on this.
So a bit strange that it's even made, but it is.
And that's what's part of the new package,
together with the options,
where you can more or less set the same thing as a normal agent.
So I'm a bit puzzled why they did this, but that's okay.
And then in this case, we add a to-do provider.
So a to-do provider is just a set of tools
for adding and removing and getting remaining to-dos
and keeping track of those to-dos.
So we could have quite easily made this ourselves
with various tools and stuff,
the modes for plan or execute,
and some memory system that saves some memory down into the system.
Then it gives access to a web search tool
and a self-built search tool that just uses download UI.
Then it takes this tool approval,
so it can stop asking again and again.
And then it sends it into all this extra code.
And again, I won't go into this.
Again, there's 4,000 lines of code that essentially builds up a UI.
And that UI, if we run it here, looks like this.
So we can write slash exit, we can write slash to-dos,
and see we have no to-dos, and so on and so forth.
And what we can do is we can, for example, say...
So that's what topics should we research.
First 10 people in space.
And if we do this, it goes into a same mode
as a copilot CLI or a codex,
and begins to do various sets of work.
So it's now calling that it needs to go into the right agent mode.
It created three to-do items now,
clarify the counts of people,
verify 10 people,
prepare concise thing,
made a plan for that, so it used its fine memory tool.
And now it has made that,
and we are being asked a few things.
Should we have it in chronological order,
including suborbital flights, only orbital flights, or something?
Say only orbital flights.
So these are just instructions of asking for clarifying questions,
and all the work around getting this UI to work.
And it works in some places.
Here you can see it just goes off the screen with the response,
and not wrapping.
But now asking, are we okay with this plan,
and do we want to execute it?
And let's execute it.
So now it's into execution mode,
reading the plan,
getting the to-do items,
do a bunch of web search calls now.
So most of the code is actually building this GUI.
And I will get a little later into why I think they built this.
But it's going through the different work,
and I've seen it sometimes stall, and I need to write continue,
just like some of the coding agents as well.
And it tries to keep track of how many tokens it uses.
I don't think they are correct, those limits,
but that's another thing.
So it comes back with its answer,
and who was the first in space, second in space, and so on,
based on various sources that it found on the fly.
So a little like GitHub Copilot or Cloud Code.
The second demo is not as impressive,
but let's go in and see it. So in that one,
we're essentially just seeing that it adds a tool
that is a web search tool.
And in a parent agent,
it gets that it can have sub-agents,
which is just a way of spawning new agents on the fly, if need be.
So if we start this sample,
it needs to get some stock tickers,
some Microsoft, and let's take Atlassian.
It's easy to remember that it's called team.
So now it will spin up two sub-agents, one for each of them,
and do the work, finding the price,
apparently at the end of last year.
I've seen this fail a few times, not getting the right data, but...
Again, there's all this extra code that keeps track of all the state,
how far is everything in the...
So it's a big, big orchestration task,
those 4,000 lines of sample code.
And yes, sometimes it just comes back and it couldn't do it.
We can say try again, and sometimes it works.
So it spins up two new sub-agents instead,
and try to figure it out.
So now it found it for one of them, but not the other.
It depends on what's going on, and it's, of course,
based on how good is the instructions, how good are the tools.
The final sample is where there's some sales data
of some various things that's been sold,
and who's the salesperson.
And similar to the other ones, it's just a data analysis
that will now have access to various files,
and being able to grab those data.
Oh, that's not the right one. So let's say,
what is the most sold item in quantity?
This one I've seen throw back a lot of reasoning back,
and suddenly in the middle of...
The sentence just stopped, but let's see if it goes better this time.
It actually went OK.
So it found that the notebook pack is the one that sold the most.
It did it wrong.
Let's tell it that.
Yeah, OK. It's still wrong.
And it just goes off.
Again, this is not the important part.
The important part is that there is this harness around an HTML file,
an agent that's not very smart here.
That's another matter of opinion.
But that's what we have in three samples.
And my big question is why.
Why in the world are we getting these demos? They're cool.
It's cool to see a bigger demo,
but it's not like I can take those 4,000 lines of code
and just copy over and ship it to my customer or anything.
Or if I do, I would be stupid and not use a tool like Codex instead,
which is a dedicated tool to do it.
I think they made this sample to show that it's possible to do.
And I think it's made for some upcoming demo
that they need to do, perhaps doing Microsoft Build in June.
I'm recording this middle of May.
I think this is there.
Look how cool and how advanced you can build something
in Agent Framework,
because just showing a chat, just showing tools and so on,
is not really any news.
But showing that it's actually a very, very cool feature tool
that you can build around.
And I actually had this idea on my to-do list of videos as well.
I wanted to build a coding agent myself as well,
and this is what essentially they have done.
And also shown that it's not that much about AI
and much more about building a good GUI
and building the right tools for an algorithm
and keeping the orchestration right.
And as we saw, it failed a bunch in good instructions and so on,
so those demos will probably be iterated on
until they need to do a demo on them.
That's at least my thought process about this,
because it's not really showing us how to use Agent Framework,
not to use the various features.
It's just they have built a bunch of tools just like we could.
I had no doubt in my mind that it was possible
to build something like GitHub Copilot using Agent Framework.
Of course, it could be done,
but that it's a bunch of extra code that needs to go into it.
So again, in here,
all the things that has nothing to do with the Agent Framework
is 4,000 lines of code here,
and then just a few lines of code down here
for the various out-of-the-box tools and so on.
So I don't really feel that it fits
inside a Microsoft Agent Framework.
I could see it if they just had an example
and not had all of this and just kept it here,
and had this kept over here.
But why it's inside being maintained is a bit the thing,
because if you really want to do file memory,
you probably don't want to do file memory.
You want to do memory in a database or a vector store or whatever.
So why this is now inside?
It's experimental, so they can take it away, of course,
but it really puzzles me why we are seeing this.
So I can't really see myself using this that much
when there is dedicated tools.
Had Codex, Cloud Code, and all these things not existed,
of course, it would make sense for us to build them, but they do.
But again, they have them.
We can try them out.
If you need to build something like this,
you definitely need to go in and start exactly
how they did the small little details.
But in normal AI that you're probably going to make
with Agent Framework, this is not really something
we can use for too much, in my opinion at least.
But if you think differently, please let me know in the comments.
But that's everything for me this time.
See you in the next one.