Hi and welcome to this AI and CCR video and a little mini-series about reasoning.
So LLMs can do reasoning, thinking, they call it different things. But what is it
and how can we control it? So let's do a small intro and then there will be a
separate video on how OpenAI do it, Google do it and Antropic do it. But what
is reasoning and why does it matter to us? Well reasoning was introduced into
the O1 model back in September 2024. So OpenAI was the one who did it and it
was sort of an involvement of at that moment in time people were getting a lot
of hallucinations from AI and someone began to say if you tell it things step
by step, you have probably heard that if you have been around for that, it thinks
better. And this is kind of an involvement of that by moving such a
thing step by step into the model itself. Because reasoning is a little like
having a human having an inner dialogue. They call it chain of thought in the
technical terms. But to have a question before answering instead of just saying
what comes to mind. And we know it from ourselves. If we just ask a question and
we immediately need to answer or by mistake answer, we might say the wrong
thing. While if instead we go in, let's say, what's the capital of France, we might
know that by heart. But if we want to think a little more about it, we might
come up with a better answer in the end. And that is what reasoning is all about.
How much of this inner dialogue is shown to the user when it comes to, for
example, chat GPT and Google and so on, is very much from model to model. Some of
them showing all of it. DeepSea was very famous for doing that. Some of them only
show a summary and some of them it's configurable in their APIs what they do.
So reasoning is a good thing. If we think a little about what we answer, we
often get a better answer. But there's some drawbacks. Sometimes reasoning is
simply not needed. In this case, we just ask hello and if the LLM needs to talk
about, think a lot about what hello means, what it is in cultures and so on
and so forth, just saying hello, that can lead to overthinking. And overthinking is
wasting the user's time and the developers money. Because the reasoning
an LLM can do counts toward the output tokens. It's still going back and
forth like doing a chat loop where you have multiple conversations back and
forward. Because this inner monologue happens, it's still predicting next
tokens and someone needs to pay. So it's output tokens and those are the
expensive ones. So we didn't really want an LLM to think so much about just
the question or message hello. While if we asked it a deep question or deeply
technical questions, yes we want to do it because else the answer that it would
just give back quickly might be wrong. So it's a balancing act when to reason and
when to not. Some models are slowly getting better and internally
understanding when to think hard about something and when to not. That is called
the auto reasoning. Some models don't do it, some do it better, some do it worse
and so on. But if we know upfront that we don't want to think a lot about this
because we know the kind of scenario we are in in our code, it's better to tell
upfront how much it should think. And we can control that. So in this mini-series
we're gonna first dive into OpenAI in part one to see how you control it
there. Then Google in part two and Androvic in part three. And the reason
why we need to check out and do it in different models is because this is not
an industry standard yet and they do it in various different ways. Even Google
does it in two different ways now. Slowly going to the way of how OpenAI
do it. And for that reason I'm breaking it down into three parts because it's
also what is called breaking glass because Microsoft Agent Framework
cannot really take a decision on how reasoning is done when not even those
three big ones cannot choose by themselves. So it's a more low-level
setting and it can be difficult for someone new to Microsoft Agent Framework
to actually figure out how to even set this reasoning. And for that reason we
will go into three separate videos on it after this. So see you in those.