Hello everyone and welcome.
In this session we will do a hands on activity on Llama guard 311 B vision model.
For that, you'll have to go to the hugging face and request access to this model.
I'm on this hugging face interface.
And if you notice here this is the model card.
You can read about all the details that they have provided.
Very first thing that you'll have to do is request access.
Now I already raised the request for this model.
I'll show you how this looks like.
So here is the request access that comes along on the model card.
Before you raise a request, it says you have to agree to the terms and share your contact information
to access this model.
And then there is a license agreement.
Expand this.
And all the way at the bottom you send the request.
It gets approved in few minutes to maybe couple hours.
Usually it comes along very quickly.
I'm here on the Google Colab interface.
If you notice here I have the Pro account.
It's very important that you understand that Llama Guard three vision is a very big model.
It needs a lot of compute and memory.
So in this case the resources that I'm using is system Ram as 83.5GB, GPU Ram is 40GB, and the disk
memory is 235 gigabyte.
Okay.
So now I'll go ahead and start with the code.
First check what are my compute allocations.
So I'll run Nvidia SMI command.
So here you get all the details about the Nvidia processes.
How much is compute and memory allocation for the notebook.
Right now no job is running so no processes are running.
Let's go ahead and upgrade the transformers.
So now I'll go ahead and execute the command upgrade transformer.
So I upgraded the transformers.
And then I'll import certain classes from Transformers that we need for running the model.
In this case, I have imported Lama for condition generation auto processor, Lama processor for us
to help download and run the model.
Then from typing import list any.
And also import torch.
I'll go ahead and provide the model ID.
So now with Lama guard multimodal model ID I'll go ahead and load the model.
Looks like there is a typo.
Now I have used the tokenizer from llama processor by passing model id to processor.
Let's get the model from llama for condition generation from pre-trained method where pass model id
torch and device type is auto.
Let me run this.
It takes a while for this to run.
Now we have two problems here.
We don't have a token in our colab secrets.
So let me go ahead and add the code for for accessing the hugging face interface.
I missed a step here.
Before accessing the models from Hugging Face Hub, go to login.
In this case I'll have to provide a token.
I will go to the hugging face interface and get my token.
Got token and login.
Now run the interface.
Now there's a different problem.
It's a client error where you're forbidden.
For this URL, there is a special permission for the model to be publicly accessible.
I'll share that with you right now.
So under your account, go to Access tokens.
When you generate this access token, you have to go edit permission and then give read access to all
public gated repository you can access.
It is very important for you to get access to this model in the Colab account that you own.
I'll come back here to the Google Colab account and run this.
It works seamlessly.
Also, this is a huge model.
It takes time for the notebook to download the model, so I'll go ahead and pause the video until the
model is downloaded.
I will resume the video when this is done.
If you notice the model is downloaded, it's a very big model for hosting.
So here I have used two methods that would help us with the execution of the Llama guard three vision
model.
One of them is display image.
Display image will be used to display the image that will be part of the conversation.
And the other method is llama guard test.
M stands for multimodal.
This method has tokenizer as an input.
Along with tokenizer.
We also pass model the actual model, which will generate the output the conversation image category
and exclude category keys.
Categories is a very important upgrade to the Llama Guard three model.
It would help users add their custom categories that they want to add on top of the 13 default categories
that they offer.
You can also exclude the categories that you don't want.
So let's say you have a model and you want to use it for different use cases.
Every use case has different categories that you want to guardrail against.
So you can exclude the categories that you don't want for your use case.
It's a very convenient feature where you can exclude these categories.
So now let's go ahead and understand the code here.
In this code block if category is not none which basically means that you're providing the categories,
then tokenizer will apply the chat template with the conversation.
You add generation prompt as true.
Set Tokenizes false.
So I set this up as false.
There are no special tokens, so this is false as well.
Then you provide the list of custom categories and exclude category keys.
If you want to exclude anything.
This can be an empty list as well.
And in case you don't have custom categories, then everything else is very similar to what we did above,
except that there are no categories.
So that's what this else block is all about.
Then we will use tokenizer to tokenize the input, and we pass the template along with the image, which
will return the tensors as dt.
Then we'll use the model instance to generate the output responses with inputs.
Do sample as false.
Top p and temperature values are none along with the max new token as 50.
When the output is generated, then we use tokenizer to decode the output, get the response back,
and return.
Here is the code block that would help us invoke the model with necessary inputs.
These inputs are an image Example tax.
So in the content directory in the Colab, I have uploaded the example text jpeg image, and then I
convert it to the RGB format.
Here is the conversation where the role is user and the content is type text.
How can I file tax.
And then we have type as image.
I have commented out this this code that we will go through it later on.
However, with this input that we have I'll pass it to the llama guard.
Multimodal test method with tokenizer model, conversation and image.
Let's go ahead and execute this method or this block of code.
So now this code ran.
And if you notice here we have begin of text start header ID as user and header ID.
And then there is an image because since it sends the image to the model and check if there is unsafe
content.
And here are the default categories that it provides.
The user is asking how can I file taxes?
And then this is the system prompt.
And the response came back as safe.
This is end of turn.
So this is an example where the image was processed with the conversation and determined to be safe.
Now let's understand and do a couple of other things.
How can I provide my own custom categories?
So for that I will have to add the custom category.
I'll go ahead and add the categories here.
So I provided a custom category dictionary with S1 as custom category one and S2 as another category
that says this will be removed, right?
So technically this won't be removed.
This is another category two.
And let's go ahead and run this.
Now I have to pass this value to the method.
Let's run this one more time.
There you go.
So now you have custom category one.
AI models should not talk about this.
This is the custom category one and this is another category.
So now we provided two custom categories.
That model will evaluate on only these categories.
But what about excluding certain categories.
So let's say out of the 13 categories that were provided you don't want to evaluate S2.
So in this case what we can do is we can exclude category S2 as a list.
And then I'll pass exclude category keys to the method.
And then let's execute this.
If you notice there is no S2.
It says S1 and S3.
You can also now do S7.
Privacy is different for different companies.
So let's exclude privacy.
So if you notice it does unsafe content categories without S7.
I hope you learned about Llama Guard three and how it works with default and custom categories.
You can also exclude certain categories.
There has been more configuration options available with this model compared to its previous version,
and it also does the image processing along with the prompt.
Thank you so much.
I'll see you in the next video.