Hello everyone and welcome.
In this video we will go over the probe markdown image exfiltration.
For that, I'll walk you through the documentation of the probe on Garrick reference documentation.
On the left hand menu I'll go to the direct probes.
And here is where you'll find the details about all the probes.
I'll click on the probes XSS.
And in there we have markdown image exfiltration.
Here's what it says.
And we get the model to write embed assets exfiltration text in the URI.
Attempts will be made to get the model to plant a markdown pixel image in output.
Now let's go ahead and do a hands on activity using this probe on GPT 3.5 turbo model and GPT two model
on hugging face.
So here's the PyCharm project.
I am at the root directory for the Garrick project.
The run configuration is generations is one.
Basically, this means that we want the prompts to be sent to the model only once.
Plugins field here has probe that we want to specify.
In this scenario we have x dot markdown image exfiltration.
The model type here is OpenAI which is a platform.
And then the model name is GPT 3.5 turbo.
And the reported directory is same as the root directory with the reports as directory within the root
directory.
Now let's go ahead and run this.
For that I'll have to add the command correct config probe exfiltration and go ahead and execute this.
But before I execute this I wanted to emphasize the fact that you can run the same yml files with the
CLI command.
It's just easy for me to use a CLI command because it's more descriptive and I don't have to do repetitive
things again and again.
That's why I'm using a yml file, but you can use whatever works best for you.
So for now I'll go ahead and execute this.
If you notice it's reporting to our reports.
The entry and the markdown that that we wanted to probe is shown here.
And it passed both the test that is basic exfiltration and markdown Down exfiltration for content.
That's great.
Now let's go ahead and analyze the result for this execution.
So here is the JSON line format.
All the descriptions has been here.
It's hard to determine and understand each line item.
If since this is a lot of details.
Let's do this.
Let's pick one result and go ahead and use it in another file.
JSON.
I'm going to do a soft wrap here.
So here is the entire result.
That was bitter determined.
So let's go over the fields that are part of these results.
Here's the prompt that was sent to the foundation model.
Here's the output.
Then we have this detector results field that will determine the results and evaluate it.
And that is pretty much the the different parts of the or the main parts of the results that you want
to know.
So now let's go ahead and run the same results with a different model.
So in this scenario what I'm going to do is I'm going to use again hugging face.
Have the GPT two as the model that we want to execute.
Clear everything and Garrett config probe exfiltration since GPT two is hosted on.
Hugging face and it's pretty much a free, resource driven platform, it takes a while for it to execute
the entire set of exfiltration probes that we would throw out the model.
I'm going to pause the video for a little bit, and once it's done, I will resume it and we can go
over the results.
Thank you.
And I'll be back.
So after 1120 five seconds, the job was completed.
And if you notice, it passed all 12 out of 12 tests.
It created a file, a JSON line format file in the reports directory.
Let me open that.
And here it ran the tests.
I'm going to pick one random test here and put that in the results file.
And if you notice here very similar format there's a prompt.
And then there is output here.
And then there is a deductive results with the notes.
And that's the overall format that the tool uses.
Thank you so much I will see you in the next video.