Hi, and welcome to this AI and C-Sharp video on the Microsoft Agent Framework.
Today we're going to revisit a topic that is old to me,
but new to the Agent Framework, and that is agent skills.
So agent skills have now an official implementation,
while I, a couple of months ago, made my own implementation.
So we're going to look at Agent Framework's new implementation,
and compare it a little to what I have, if I need to remove it again,
or if the official implementation lacks some features.
So let's get into it.
And just a reminder of what agent skills are.
It's a new format that is a complementary to MCP servers,
again, invented by Anthropic and open-sourced on agentskills.io.
And it's a simple and open format that gives agents capabilities and expertise.
And it's mostly for local agents because it requires a folder structure.
So, for example, if you have a folder called agent skills,
and you add three folders, employee handbook, secret formulas,
and speak like a pirate.
Each one of these needs to have a skill.md.
That's part of the spec.
And inside such a skill, you have a front matter up here.
We have a front matter that needs to have exactly the same name here as here
in order to be a valid skill.
Then you give it a description, and then you have an instruction body.
And what happens is that LLMs and AI in general
needs to get the simple description and name,
and then on demand load the rest of the skill
so it doesn't cost too many tokens if not used.
So we can just have a skill file, but we can also have extra files
that can be reference files, other MD files, for example,
assets in terms of pictures and stuff like that,
but also scripts, in this case a Python script here.
So, the idea is that you have this folder,
you load it up into the system, and then you take it...
AI are going to use it in various different ways.
We'll see how Alien Framework does it in one second.
Because, if we have a look, it's under AI context provider
in my sample repo because it's implemented using that.
And I have this test folder as we just saw
with the employee handbook, speak like a pirate,
and a script for some secret formula.
And the way Alien Framework have implemented it
is they have a context provider called FileAgentSkillsProvider.
And the only real thing we need to give to it is
a list of different parts where you have skills.
You can give some options.
They're fairly limited and quite buggy at the moment.
You should be able to change the instruction prompt,
but if you try to do it, it just doesn't add a prompt
for some reason, and breaks down.
So there's some bug around that, but at some point,
it should be able to help you set the instructions.
Once we have that, we don't need to do anything more.
In my case, I'm just adding a little extra tool
to run Python scripts because that's not part of the spec,
to be able to run scripts at the moment.
A little more on that later.
Also, to show this, I'm showing the raw call,
and I'm doing tool calling middleware
because what happens behind the scenes
is that agent skills are on-the-fly injected skills
in terms of two skills, one to tell what skills there are,
to read a specific skill, and another to go in
and read resources for a skill.
So if I run this...
So before that, I'm just getting an agent,
and after that, I'm just using a...
simple chat loop with sessions.
So if I'm saying, the answer, hybrid mode,
what we will see is that, first of all,
the call, we can see that on-the-fly instructions are being added,
that it has these skills,
one for each of the skills, the employee skill, the pirate skill,
the secret formula skill, and so on.
So this text is what you should be able to manipulate,
but at the moment, it doesn't seem like you can do that.
Then, we can see that what happens behind the scenes
is we have a load skill function or tool,
and we have a read skill resource option.
So that's what is there, and we get our calls back, and so on,
and we see that we end up in pirate mode.
We could also then say...
isParentallyPaid,
which is an HR skill.
So here we are using one of the reference documents,
and if we have a look again...
to go down here...
So here we used a read skill here, and it comes back.
If we try to call the secret formula with the Python script,
it tends to not work.
Read the secret formula...
in that it fails because it can only give the relevant part to the script,
and not the real part.
The relevant part is here, but it doesn't know
where that is in relation to the rest of the system.
So there's some limitations. I know they are working on script support.
I don't know how they want to have script support,
specifically, but right now it's not really possible
compared to my own setup here,
where I have Agent Skills, the same folders,
and up here we are using my interpretation of this,
which is Agent Skills, where we can get the instructions,
and get them as tools.
I also expose them as tools, one for what tools are available,
and three tools, what tools are available,
read a specific tool, and read a resource.
So I'm using three tools instead of two tools for that.
So let's do a little side comparison in what we can do in this.
So the official implementation is here,
and my version of agentskills.net, which is the NuGet package,
which is also part of my Agent Framework Toolkit,
which is a set of NuGet packages on top of Agent Framework.
So the new one is built in, it's from Release Candidate 2 and higher,
and mine is a NuGet package that you need to install if you want to.
The delivery mode that they have chosen is AI contact providers,
but behind the scenes they are simply just injecting extra instructions
and adding two tools.
I do sort of the similar, I can do two generic tools,
but I also have an option to have each skill be its own tool
if you rather want that for more discovery.
And then you can get the instructions back as text that you can paste in.
So I'm not using AI contact providers for this,
because I want to have more control here.
Here we don't really have any control,
in that we can't really control the instructions,
where they are compared to the rest of the instructions,
and here we just get every tool no matter what.
Here we can actually control it.
So instructions are automatic in the official implementation,
while it's configurable in my version.
Logging-wise, they have standard iLoggers,
while I instead give a log error object back
on which of the skills perhaps couldn't load.
So that's a matter of opinion, you could put that into a logger yourself.
But else they have more out-of-the-box logging.
Then there's skill validation, if the skill actually follows the format.
And it seems that they don't care at all.
There is these rules, for example,
that this name needs to be exactly the same name as this.
You can write something different here, and it will still load as a skill.
I put in that you can choose the strict mode and get errors back if it's wrong,
or you can go into loose mode.
That is sort of what the official implementation does.
Then a huge limitation they have is that you just get these skills,
and you just get these tools.
But you can't really get them back as objects,
so you're not in any way able to do a subset of the skills.
It's all or nothing, and you can't get individual instructions
if you want to use those together with some more curated parts
of your instructions file.
That is fully configurable in my implementation
because you get each skill back as a .NET object
with all the information in it.
So that is much simpler to do.
You need to do a little more code, but you have more control over it.
Another huge problem with the official implementation right now
is that it doesn't give the subfile parts.
We saw that with the script not being able to be run
because it just gives back the name and description,
and when it reads it, it's only what's in the body of the Markdown file.
I give the body of the Markdown file back,
but then I also tell, and these files are available for this skill
with the full parts, and that makes AI able to actually read
and execute the various scripts and subassets.
In terms of reading skill metadata, not really something that's too important,
but you can have extra things like license and metadata
on an official agent skill.
The official implementation only takes out the name and description and uses it,
while I grab everything that you get back in the objects if you need to.
Honestly, you wouldn't probably need this for anything at the moment.
And finally, the ability to actually shoot scripts.
None of us actually have it,
because each script execution can be very different.
How do you want to run your Python,
what kind of environment, what kind of restrictions, and so on,
and how to run a BAT file, how to run an executable, and so on.
You need to bring your own tools, in my opinion.
Right now, in the official implementation, there's no support at all.
You could bring your own tool, but again, the problem is the subfile parts
that are not exposed to the LLM,
which causes it to actually not be able to call it correctly.
Again, they have an open issue that they want to enhance it with that,
but I don't see a way they can actually make it easy
and cheap token-wise as well.
So, right now, it is cool that they added it,
but it's a very, very limited set of features that they gave,
and I'm really, really surprised that they don't even validate the skills correctly.
So, that is my review so far.
I will definitely not use the official one yet, when I have the other one,
because I'm simply just having much more level of customization here.
But perhaps, over time, the official implementation might add more features
and come to a point where we can actually throw away
third-party skills like mine.
But for now, I would definitely not use the internal implementation.
So, that's everything. See you in the next one.