Hi, and welcome to this AI and TCI video on an OpenAI specific topic, which is their service tiers.
So, again, this is OpenAI specific, because they have something called Flex Standard and Priority,
where you can choose your speed versus your pricing.
Azure OpenAI doesn't have the same, or they have something,
but it's an invite only, so we will leave it out of this video.
So, this is for OpenAI specific.
So, if you go to the pricing model of OpenAI, you can see there's something called Batch.
We have a separate video for that, but there's Flex, Standard, and Priority.
Standard is what you normally always see, and we see the different input and output pricing,
while Priority has been there for a while and is double the price.
For the double the price, you get less errors, less timeouts, and faster latency in general.
A new thing they have is Flex, which goes the other way, which is slower and more latency,
but if you can manage that, you get it for half the price, meaning the same as the batch prices.
So, these are the three options, and the way you control it is either one of two ways.
One way is that every API key you have is linked to what is called a project,
and in projects, you can go in and set if it should run Default or Priority.
It cannot run Flex at the moment, but you simply just switch to Priority,
and then you're paying double the price now with better performance,
and if you want to switch back, we switch back to Default.
So, this is how you control it.
If you don't want to put anything in code, you simply switch, and you get a better quality of the service.
And I've seen that, in reality, this is actually quite needed if you have very big requests
like several hundred thousand tokens in one go.
Before we switched in our project to Priority, we simply couldn't use OpenAI for that,
but that might have been a period where they had problems with delivering service.
But you can switch. If you want to switch in code, you can also do that,
and we do that down in the chat completion options or the response API options,
where we have something called Service Tier.
And Service Tier can be Auto, meaning it follows what is in the project. It can be Flex.
If we look at the pricing again, Flex doesn't work on every single model.
You can only see it's JGP 5 and 03.
So, some of the earlier models, if you use Flex, it won't work.
While Standard works for everything, and Priority also, more or less, works for everything.
So, we can set that.
We can set default, meaning even if you have a project that is Priority, you can switch it back to default,
or you can have a default project that sometimes, on a case-by-case basis, needs to be Priority.
And that's actually everything there is to it, just setting this extra thing.
In real life, you would probably just go up in the portal and switch everything to the Premium Service,
or have two projects, one that are Service Tier and one that are Default Tier.
But that's everything. See you in the next one.