Let's talk cost and how much it will
cost you to follow along in this course,
but also cost in general.
The topic is really, really difficult to place
in a course like this because it could
either be at the front, but then we'll
talk, in this case, I will talk about
some concepts that haven't been explained yet, or
it can be at the end, which I
think is too late, so I'll put it
up front.
Bear with me that there is a few
things we haven't talked about yet in this,
and they will be, of course, explained along
the way, and I will give some high
levels in this video.
The good thing about cost when it comes
to AI in general is that it's a
pay-for-what-you-use approach in all
of the providers.
So there's no idle fees, meaning that if
I make an Azure account or an OpenAI
account and don't use it for a month,
it's not like it costs you anything compared
to, for example, if you had an Azure
app service.
If you use it or not, you would
pay because it's a spinning system.
So it won't cost you anything to have
multiple accounts to different providers.
I have, like, 10 different, and it doesn't
cost me anything when I don't use them.
There's, however, that most providers need to charge
up your account with, like, around $5.
Azure and Google being the exception, where you
can pay as you go, and then you
pay after the fact.
But if you want an OpenAI account, you
need to spend up front $5, and then
you can, of course, use those as you
go along.
And $5 would be plenty to follow this
course.
There are a few providers that offer free
tiers with some heavy rate limits.
Google have some free tier.
GitHub models have a free tier.
Hugging Face and Cohere have as well.
But, for example, Cohere, you can make one
call per minute.
And Google, you can only use all the
very, very cheap models, not some of the
more advanced models and so on.
What you pay is not in amount of
time you spend on the LLM or anything.
You pay in what is called tokens.
Let's learn what those are.
Tokens is a kind of language translation from
English to a language of tokens, which is
common words set together in order to not
need to know every single word in the
English language.
So you can see here I have made
a high.
This is a sample on how many tokens
there are in this sentence.
And you can see "-this", "-is", "-a", is
a token by themselves.
So I use very, very common words here.
But have I used an uncommon word like
platypus?
That would have been broken up into multiple
tokens because it won't have a specific token
by itself.
But I have one token specifically set to
it.
So these 66 characters is 15 tokens in
total.
And you roughly learn that you shouldn't really
care about this too much.
You know many tokens, less tokens and so
on.
15 tokens is not a lot.
You pay in three different or four different
ways.
You pay for input tokens.
That is your prompt.
So if we say, what is the capital
of France?
We pay for the tokens of that sentence.
There will be additional input tokens that you
also pay for.
Like whenever we introduce tools and structured output
and stuff, there will be additional metadata that
need to be sent to the LLM.
For example, when we say, what is the
weather in Paris?
And we have a weather tool.
We will also pay for the schema that
says, how is that tool defined?
So even if you say, hi, and you
have a lot of tools, it will cost
you more tokens.
We will get into that when we talk
tools.
Then there's something called cash tokens, which is
reuse of tokens.
Because whenever we begin to do conversations, we
will ask a question and then we'll do
a follow-up.
And we don't need to spend tokens on
the first part of the question again, because
that would be the same answer.
It only happens when you begin to use
more tokens, like 1024.
Before that, it will not try to cash
tokens.
But if you have some heavy use and
with very much follow-up, all your things
will probably be cash tokens, and cash tokens
are cheaper in general.
Finally, we have the output tokens, which can
be put into two categories.
The first being what we get back.
So when we ask, what is the capital
of France?
We will get that it's Paris.
And we pay for the word that the
capital is Paris.
If we use reasoning models, like Chat-GPT-5, we
are also getting some tokens that are being
used for the internal reasoning of the model.
So it might think about what is the
capital of France before it answers.
Or if it's a very complex question, it
will talk back and forward with itself and
use those.
You don't really get those tokens back, but
you still pay for them, because they are
still tokens being used by the LLM.
These are called reasoning tokens, and they will
be part of the output token.
So you will see an output token, for
example, of 1000 output tokens, and then you
will know 300 of them were used for
reasoning, while the last 700 was actually the
text you received back.
Let's see some examples, and also why this
can be very difficult to understand, and you
need to learn from experience how tokens work.
So this is one of the examples in
the sample repo, so you can go check
out the code if you want to.
But if we take a simple setup like
our Hello World, and just ask a question,
what is the capital of France, we get
the output, of course, that the capital of
France is Paris.
And if we go to the sample repo,
which we have here, we can go in
and see that I'm just using the chat
client, and then I'm taking out from the
response, which is where we have our text
back, the capital of France, we can also
get a usage object.
And if that usage object is different from
zero, meaning the LLM we are working with
support reporting usage back, and most of them
do now, we get our input tokens, we
get our cache tokens, we get our reasoning
tokens, and how many of them were output
tokens, and how many of them were reasoning.
So that is what we are seeing here.
We are not doing anything special yet.
But some interesting thing happens, because we are
using here a chat-gpt 4.1 nano, one
of the cheapest models you can get.
And it's used some input tokens.
What is the capital of France?
There's no extra tools or anything.
So this is the raw.
What is the capital of France is 14
tokens.
It used zero cache tokens, because it's a
too little sentence to begin to cache, and
there's no follow-up.
And it used eight tokens for the output.
The capital of France is Paris.
And zero of them was reasoning, because it's
not a reasoning model.
It spent roughly one second doing this.
In general, you will see that Azure is
faster at answering back than OpenAI is.
But it's all depending on what time of
day you do it.
If you do it in peak hours, it
will be longer.
If you do it in non-peak hours,
it will be shorter.
So you can't really look at the time
and say, okay, every time I will ask,
what is the capital of France?
It will be like this.
Sometimes it will be longer.
Sometimes it will be shorter.
But I did it with different models at
the exact same time, so they can roughly
be compared.
We also have the same code, but now
with another model, GPT 5 nano.
So we get, in this case, it actually
used one token less.
It must have figured out that it wanted
to do it in a different way.
It shouldn't really happen, but that's the way
it is.
We see that it gives a shorter output,
but it used more output tokens, and it's
also using slightly more time.
And the reason for that, we can see,
is it's using more reasoning tokens.
And this is because GPT 5 nano is
a very, very, I won't call it unintelligent,
but not the most intelligent of these models.
So it needs to think, oh, this can
be a serious question.
I need to think how to give that
answer.
So it actually spent some tokens on this.
So in this case, using this one over
here, despite it actually having a higher per
cost token count, will cost less.
And we can also see we get slightly
faster back.
But then something interesting happened.
If I go to the biggest and greatest
model, GPT 5.2, as of this recording,
we get exactly the same input, exactly the
same output.
And then we see that our output token
is only six tokens here, despite it being
the model that can think for a long
time.
This is because GPT 5.2, we could
also have taken GPT 5, is a more
intelligent model, so it knows that this question
is not a hard question.
So it doesn't need to think about it.
This model has less world knowledge.
This one has more world knowledge.
So this was just ready to answer question
back, while here it needed to think about
where it had that in its storage.
So for that reason, you cannot really just
say, bigger model, more cost, in terms of
output, because things like this.
But have you asked a very scientific question
about some universal models and so on?
This model would have definitely began to use
a lot of reasoning tokens and way more
reasoning tokens than this one over here.
We will talk about reasoning in general and
how much reasoning and so on.
But just to show you, it is very,
very difficult to figure out.
So here we actually use the biggest models
and spend the least amount of tokens.
Those tokens still cost more, which we will
see in one second.
But it is a balancing act on how
cheap or how intelligent you need to have
a model.
This is a simple question.
So we should definitely use a simple model.
In this case, I would use this one
over here, despite this being slightly less tokens.
While here, it is kind of overkill because
you think, oh, this is a serious question.
We need to think a lot about it.
So that wouldn't fit well.
While in the bigger models, this is way
overkill for a question like the capital of
France.
It shows that by being good at answering
it.
But still, why spend the tokens for a
higher price?
Because this is like 50 times more priced
than this one is.
But let's talk price.
And when we talk about these, what is
the capital of France and so on, we
cannot really measure how much it costs because
it's dirt cheap to do that.
So what I've done is I've calculated, I've
taken Hello World, as we saw in the
previous section.
And we asked, what is the capital of
France?
And beyond that, I've also asked how to
make soup.
How to make soup is a system or
is a message that will give quite a
long answer back.
Because there's several steps involved in doing it.
So if I took those questions, when I
press F5, I run it once.
If I press F5 and run it 1
,000 times, I would end up roughly spending
52,000 input tokens and 375,000 output
tokens.
This number being higher because how to make
soup gives a higher number back.
We can try Hello World from the last
section here to say how to make soup
and see that it's a much longer answer
back.
Put it over here.
And right now I'm running as a JGV
guide manual, so it really, really thinks about
how to make soup.
And you can see we got a longer
answer back and we can see we spent
a lot of extra output tokens even if
we took away the reasoning.
It's still 700 tokens compared to the 6
and 8 we saw before and it used
11 seconds to do it.
So longer answers will also take longer to
get back.
So 1,000 times, 52 input tokens, 375
output tokens.
If we look at the different providers, there
would be no upfront cost in doing this,
these 1,000 times for Azure OpenAI, Microsoft
Foundry.
OpenAI would cost 5 upfront US dollars.
Same with Anthropic.
Google models, GitHub models would cost 0 upfront.
Gemini would be no cost.
XAI would cost 5 upfront.
OpenRival would cost 5 upfront.
And HuggingFish would be 9.
And we look at these 50,000 input
tokens and think, oh, that might be a
lot or 375.
No, it's not really, because these models are
working in 1 million tokens per stuff.
So Azure OpenAI, even when we went to
one of the big models like ChatTV 5,
we would end up using...
For every input token, we would use 1
.25 US dollars.
And for every output token, we would spend
10 dollars.
So that would be that input-wise, it
would cost us 0.065 dollars.
And output-wise, it would cost 3.75.
And that's for running it 1,000 times
with the big model.
We go to OpenAI's pricing.
And let's have a look.
We have 5.2, which costs a little
more, 1.75. But if we look at,
for example, ChatTV 5 Nano here, we are
down to 0.2 instead of 1.25
per million tokens.
And output, not 10 dollars per million, but
0.8. The rough estimate is when you
go down a pitch, it's seven times cheaper,
and then seven times cheaper again.
So it would be like 50 times cheaper
to run this 1,000 times with what
we have set up right now.
So take this number, divide it by 50,
and you would roughly have how much time
you would do this, even if you ran
my samples 1,000 times.
If we look at Foundry, it's the same
pricing.
There could be some measurable storage cost, OpenAI
being exactly the same.
So Azure OpenAI and OpenAI cost exactly the
same.
They have a price agreement that they need
to cost the same.
OpenAI have a premium version where you get
faster response times, but that costs are doubled.
If you went to Anthropic, it would cost
you more.
They are roughly two times more expensive at
most things.
GitHub models could be free, as mentioned.
Gemini costs roughly the same as Azure and
OpenAI.
Grok costs a little more, so they are
on the level of Anthropic.
And things like OpenRouter, where you have multiple
different models, is various, but roughly the same
prices as the real providers.
So as you can see, even if you
ran it 1,000 times, you wouldn't even
spend $5, and that would be on the
big models.
So all these numbers should be down to
50 times less if you use the nano
models.
And that's actually everything, except we just have
one extra thing.
There is a few costs that don't go
into tokens.
So things like uses of hosted tools.
We haven't talked about hosted tools yet, but
you know them from ChatGPT that it can
do a web search or it can do
code interpreter, meaning it can generate graphs and
stuff like that.
They cost extra, so they are not built
in tokens.
They are built per time because, for example,
a code interpreter will spin up a container
in the cloud and execute your code.
In reality, this is actually the highest cost
I have when I use AI in general
because the tokens are so cheap, at least
if you stay with the small models.
If you went to ChatGPT 5.2 Pro,
then you would pay a lot and lot
of money.
We can see them up here.
So ChatGPT 5.0 Pro here, it costs
you 21 million tokens compared to 1.25
and $168 per million tokens.
So if you go to Pro, you definitely
pay, and you are definitely probably over-engineering
your system, but it can be done if
your scenario provides it.
In general, I would just say take the
cheapest model as possible, and until it doesn't
prove that it's a good model for you
and you need to go up in price,
do that.
But beyond that, stay on the cheap models.
But again, code interpreter, web search, cost you
per thousand calls and so on.
Images is the same and so on.
If you use the Microsoft Foundry where you
have your conversations and agents up in the
cloud, there's a small storage feed.
It's the normal Azure storage.
If you use things like vector searches and
so on, there will, of course, also be
some cost involved in having those, but we
will get into those as well.
But the main thing in all this is
just following this course will cost you pennies.
It will probably be the $5 upfront if
you don't want to use Azure OpenAI or
roughly that amount as you go through it
and run it a lot of times.
So that is it for this one.