In previous lectures, I have hinted at something
called Structured Output and it's finally time to
learn about it.
The best way to learn about Structured Output
is to see the problem that we will
face and then see how we fix it
with Structured Output.
So in this case, I have a normal
client and a normal agent and this agent
have given the instructions that it's a movie
expert.
So I want to give it the question,
list the top three best movies according to
IMDB, which is a website for movies if
you don't know them.
And if we do this normally, we can
certainly get a message back.
And what we got back here in this
case, let's give it a little more room,
is that we got back the Shawshank Redemption.
We got as of October 23 and we
had some things about it being fluctuating because
of course it could have changed since it
hasn't.
And it chose to give us back a
1-2-3 in Markdown where they put
the year in here and it gave the
IMDB rating as well.
So this might be good enough for a
chatbot because this is nice, but only if
we wanted to make a website where we
listed those three movies and put the rating
in a specific field, the year in a
specific field and so on, we would need
to sit and pass this text.
And we could do that, but then something
like this will happen if we ask again.
We get it slightly in a different manner.
This time it's still using Markdown, still using
1-2-3, but instead of the rating,
it thought it was best to give back
the genre of the movie.
So this is non-deterministic.
Every time we ask, we might get a
different scenario.
Sometimes it will give back rating, sometimes the
genre, sometimes the director, sometimes it's Markdown, sometimes
it's not, sometimes it's a list, sometimes it's
not.
So we would be probably worse off with
an AI trying to pass this because we
can't just live with text.
So for that reason, we have structured output.
So structured output is essentially just calling RunAsync
like we did before, but telling we want
it like this.
So what we have here is a movie
result.
This is just a class, and inside it,
it has a list of movies.
And these movies are the title of the
movie, director, the year of release, and the
IMDb score, which is none of these gave
it back.
And of course, we could have tried to
write in our description, not that it's only
an expert.
We could have said, we always want to
have this, this, and this information about each
language, but we will still need to pass
text.
But doing it like this, we will instead,
when we run Async here, we would get
a response back, which essentially is JSON.
And now we have full control over how
things look because what happened here was that
beside our question we sent, we also sent
a JSON schema on how we want it
back.
So agent framework is really nice at just
giving us this object and turning it into
a schema, having these fields, and saying we
can get it back.
And we get it back as JSON.
We even get it back with a dot
result here because the new response is a
chat client agent response of type movie result.
So now we get our result, so we
can simply inspect it like any other .NET
object, and we get our data back.
In this case, I'm printing it out in
format here, but of course, we could send
this result back, send it over the wire,
use it in a HTML GUI, or whatever
we need to do.
So we can also see the raw JSON,
and this is the power of structured output
because unless you're doing chatbots, this is what
you want like 95% of the time.
So this is very, very powerful, and we
can use these objects throughout how we want.
It costs a little extra tokens.
I didn't show the token count, but because
we need to send a schema on how
the object looks like, it will cost more
tokens.
It will take a little longer, but it
is definitely worth it, and this is my
favourite feature of any LLM that we can
do structured outputs.