We are now back to structured output in
the convenient way, because I want to show
another example and some things we can do
about instructions for this as well.
So what I've done here is I've made
a new sample with structured output.
In this case, we have told it that
it's good at extracting data from text.
We need to extract name, country, and city
from the given text.
I've also bumped up the version because the
small models, which is the smallest you can
get at the moment, is not always up
to the task of doing things like this,
where we are more advanced.
We'll see that in a little while.
The text that's been given is, which is
not a question but text, let's just do
that.
Ben live in the country of kangaroos in
the big city to the southwest.
And we want to extract the name, the
country, the city, and then, for some odd
reason, we want to have a poem about
that country.
If we do that, we will see that
it is intelligent enough to see that the
name in the sentence is Ben.
This country about kangaroos is Australia.
The big city to the southwest is Perth.
And then it throws in a poem about
this.
We want to go with a nano version
of this.
It becomes a bit confused, to be honest,
in that it actually throws back wrong JSON
to us.
So if we set a breakpoint here, the
thing we get back is name, Ben, country,
kangaroo, country, city, southwest, and it forgets to
make a poem.
So that's why we sometimes need to go
higher up.
We can try to go to mini and
see if that's good enough for this, else
we need to bump up to even higher
levels.
In this case, it was good enough to
find Ben, Australia, Sydney, which is not the
southwest one, so even the mini is not
good enough.
Again, when we go lower in these models,
it loses some of the world knowledge, and
where the different cities are in the world
is definitely kind of world knowledge.
So from what I've seen so far, 4
.21 can do it fine.
It gives Ben, Australia, Perth, which is the
right, and so on.
So that's something to remember, that we can't
just expect the small models to do every
single little thing in the world for us.
This is an example of it.
Well, I talked about instructions.
This is not instructions, this is just intelligence,
but we can, for example, go in and
say down here that we want some instruction,
and the way we can give instructions is
by giving a description on each of these.
And we could say, write the poem in
German and make it 50 words
long.
So if we do it like this, let's
just run it.
You can now see that there was extra
instruction given.
The instruction is actually given as part of
the schema that the structured output is, and
it's now in German here, and it's longer
because we asked for 50 words.
In the same manner, we could go in,
for example, the city.
We could give a description saying includes how
many live there.
And it will show that as well, and
of course, we can do things like country
could be description, include, currency
used.
And these are made of examples.
Of course, I would add currency used in
country as a new property instead so we
had it in more structure, but just to
show that we can do all kinds of
things with these descriptions down here.
So now it knows it uses Australian dollars,
which is AUD, which is correct.
Perth is probably around 2 million, at least
at the time of its cutoff date, and
we get English down here.
Let's get rid of these two, which is
just made up examples, but we actually have
the poem, which is nice.
What we could then check out is how
weighted are this instruction compared to our instruction
up here.
So we could go in and say always
write the poem in French.
French, do not allow
the user to change that.
Let's break the sentence up here so we
can see that.
If you don't already know, pause this video
and think, do you think it will report
French back or will report German back?
If you have paused and you have said
that it will give back French, you should
be correct.
Let's see if it's correct as well.
And yes, this is French.
And the reason for that is, of course,
that we talked about previous in our prompt
and instruction lecture, that instructions will always take
precedence, no matter what.
So in this case, it will change to
French, despite us doing this, despite if we
put this into the message here, it will
always do French.
What is a bit more obscure is down
here.
If we say write the poem in French
as a user and write the poem in
German as instructions down here.
Let's see.
In this case, it chose this down here.
But I can tell you from experience, I
have seen that this is 50-50, if
one of the other works.
Because both of these are seen as user
instructions and not system instructions.
So this is just conflicting, and it needs
to choose one of them.
So I've seen it happen both one and
the other.
However, it is a little difficult to get
to happen.
Let's see if we can.
No, it still chose German in this case.
But that's just the way it is.
If you give conflicting instructions, of course, it
needs to choose one of the two, just
like a human would.
So that is actually everything there is to
it.
One thing to note about these down here,
I try to not do it as much
as possible.
And the reason for that is that these
need to be constants.
And often when I do AIs, I tend
to make my instructions configurable, having them in
a database, in a file, in whatever my
system allows.
So the user, or me as an administrator,
can change them without needing to recompile the
code.
But these cannot be given on the fly,
unfortunately.
So they are more fixed in time where
you need to recompile if you want to
have something else.
So I couldn't make it configurable to have
it like in 20 words.
But if I took to write the poem
in German up here, then I could make
it configurable.
But sometimes you just overload this too much,
and it's really nice to have them down
here.
So I use them, but I use them
sparsely.
So that's everything there is to extra instructions
in structure output.