Hi, and welcome to this AI in C#
video on the Microsoft Agent Framework.
Today we're going to look at controlling the
reasoning effort on reasoning models like, for example,
ChatGPT-5.
So, in order to see this best, the
best way is to see the difference.
So, I'm going to quickly set this up
so we can run this sample, which is
in my sample repo that you can get
for free on GitHub.
So, I'm just going to get my configuration,
and what I'm going to do here as
well is, in my OpenAI client here, or
the Azure OpenAI client, if that's what you're
using, I'm also setting the network timeout.
You might not need this, but, of course,
if you have a reasoning model and you
give it a very, very hard question, it
can take multiple minutes to actually get an
answer.
So, I just included this for the sake
of how you do it.
You simply just new up, and then there's
extra options where you can set the network
timeout.
So, fairly simple to do.
But why do we need it?
Well, if we, for example, go in and
use ChatGPT-5 mini here, and we use
it without any extra configuration and ask, what
is the capital of France and how many
people live there?
If we do that, it will take a
little while to come back because ChatGPT-5
is a reasoning model, and the default, if
you don't set anything against many people's belief,
is that it's set to medium.
There's low, high, medium, and then there's something
called minimal, which is the lowest of them
all.
But right now, we have used medium, which
means when I write out this answer and
show the token count, we can see that
we used 19 tokens for input, but 623
for output, and 512 of these for reasoning.
And reasoning is good if you have a
very hard question, but if it's an easy
question, then it's not smart to just use
reasoning.
That's the reason why ChatGPT-5, if you
give it an easy question, it can answer
quickly, but if you give it a hard
question, it takes longer.
And that's because it's turning on and off
the reasoning.
For the APIs, you can't do that.
It doesn't happen automatically, so you are in
control.
But if you just don't care and just
run the models, you are actually using the
medium reasoning capability.
And that might not be what you want,
because first of all, it takes longer to
give an answer, and second, every token counts,
so you're spending more money on some easy
questions.
So let's see how we can control this.
And this is unfortunately quite cumbersome in my
mind here, because what we do is we
make our agent, but then we need to
give our options to that, where we need
to give a chat client agent options, and
inside that we need to give chat options.
And in that, we give a new option
where we set this broad representation factory, which
can finally set the reasoning effort level to,
in my case, minimum.
But we have the options of minimum, low,
medium, which is the default, or high.
And this is specific for OpenAI and Azure
OpenAI.
Other models, like Claude and Gemini, does this
in completely different ways.
So what I'm showing is something that only
works with Azure OpenAI now, and you need
to go in and check documentation of the
other models on how you set that for
that reasoning effort.
So if we do this, we can actually
get a response.
And in my case, I've set it to
minimal, which is a new effort they recently
introduced, because of this problem of everyone is
going to HTTP5, but they also need it
for simple answers back.
And for that, minimal is the best for
a reasoning model.
So if we ask exactly the same question,
first of all, we should get our answer
back faster.
Roughly the same answer, but you can see
the output tokens are now 64, and zero
was used for reasoning.
And that, of course, is much cheaper and
much faster compared to up here.
And then you need, of course, to begin
to evaluate, is this answer better because we
were thinking than this answer?
And that's not the point here.
You need to know in more real scenarios
if you need reasoning or not.
So you definitely need this if you want
to do HTTP5 and do some simple stuff.
And you definitely need this if you need
to set it up to, for example, high
and use even more tokens because it's a
very advanced thing, like code reviews or something
that is more advanced.
As I mentioned, this is Composon.
I have actually asked the Agent Framework team
about if they want to do something more
about this.
And right now, the answer is no.
This is the way you do it.
And the reason for it is that all
the different models do it in different ways,
and they don't really want to settle in
on one specific way with a string here
or because others need to, say, max allow
tokens and stuff like that.
So right now, they have said, we're not
going to fix it, change it.
Again, there's nothing wrong.
It's just Composon.
So this is what we need to live
with.
And, of course, what I have done in
my case is I have extracted this away
because I'm beginning to find more and more
things that are not strange, but Composon with
the Create AI Agent, there's multiple ways of
doing it and stuff like that.
So I have made my own Create AI
Agent for Azure OpenAI and for OpenAI.
And these are in the repo as well
if you want to copy them.
And at some point, I might release them
as a NuGet package.
But they're down here in my extensions.
And what I have made is just a
copy of them with a new name.
And here, I'm putting in the reasoning effort
so we can abstract this away.
And I also fixed that there is actually
a bug in the current one.
If you set both reasoning and tools at
the same time, this is fixing that as
well.
So annoying we need to have this, but
I understand the team needs to be generic
and work with every single AI model out
there while I can make these for Azure
AI and OpenAI which work the same.
So in my case here, I can then
– and this is what I'm going to
do in my code – is I'm going
to use these new overloads of Create AI
Agents in order to get something simple as
this.
And it will give exactly the same answer
as this one up here.
So depending on what you want to do,
you use yours.
But just be aware that if you use
it raw, it is a bit extra to
write in here in order to do it.
So do some kind of abstraction away from
it.
Make a helper method, extension method, in my
mind.
But that's everything there is to reasoning.
The most important thing is, of course, that
you remember that any reasoning model by default
is medium.
And if you don't do anything about this,
this will cost your users a lot of
time and you a lot of money based
on the tokens if you don't are aware
of it at least.
So we're done.
Thank you.
See you in the next one.