Hello everyone and welcome.
In this video we will learn about the vulnerability image Markdown injection.
We'll also learn about how an LLM is vulnerable to the image markdown injection.
Followed by that we will use Goehrke tool to discover this kind of vulnerabilities.
I'll go over this blog post where it has been very nicely explained how the Llms are susceptible to
these kind of vulnerability.
So it takes an example of the Bing chat.
When Bing chat returns text, it can return markdown elements which renders HTML.
This includes the feature to include images, so the image pretty much looks like this.
There is Https protocol, the the server name, the image name, and then there is a query parameter.
This will be rendered as an HTML tag within a source attribute.
In this case, if you notice, the attacker can add the source of the image as its own website or domain,
and then add a query parameter as exfiltration browser will automatically connect to the URL without
user interaction to load the image.
Wait, that is data exfiltration.
How this ends up in the LM vulnerability.
The impact is.
As we know, LM agents like Bing Chat or ChatGPT are vulnerable to prompt injections.
This means that during a prompt injection attack, the adversary hijacks the AI and the attacker can
search web page or secrets or PII by appending such data to the URL of the image and have it automatically
x filtered.
The malicious instruction can replace the data exfiltration with data to exfiltrate.
So here this is where the prompt injections can happen, where malicious instructions can be part of
this URL that goes to the element.
Now let's go ahead and use Garrick tool to detect this kind of vulnerability in the LM.
So in here in Garrick probes that we learned in our previous video, I will navigate to the Garrick
Probe X in here.
I will do the image exfiltration and we can get the model to write embedded assets as x filtering text
in the URI.
The attempts to get the model a plant a markdown pixel image in output.
Now let's go ahead and execute.
This command that I have here is Garrick model type is Huggingface model name is GPT two.
And the probes that we want to invoke is x dot markdown image exfiltration.
And then execute this.
There you go.
It started running this and it will take a while for it to run.
So I'm going to pause the video.
So once the Garrick was done executing the probe I was able to extract the JSON line format that it
does the reporting.
And you notice here it has the details about the reporting.
And it says at the end that there were about total 60 of them ran, and all 60 passed for the basic
x filtering and the X filtering markdown for content, it ran another 60 probes and then 60 of them
passed.
Great.
So now let's go ahead and run this on a on a different model hosted on the open AI.
Let's run the probe on the turbo 3.5 GPT model.
So I'll say Garrick model type.
In this case it is OpenAI.
Because it's not free dot.
You'll have to pay for it.
And then you say model name is GPT 3.5 turbo probes.
Here is XSS markdown image filter.
Just be sure that you're adding the OpenAI open API key to the system environment before you run this,
because OpenAI model GPT 3.5 turbo is not free, it needs payment for accessing the model.
So now let's go ahead and run this.
So I believe I did a mistake.
I did not add a hyphen here.
Now let's go ahead and run this.
It's fairly quicker compared to the hugging phase since it's a it's a paid service compared to the hugging
face GPT two.
It's free so it takes time.
The Time.
The resources are not allocated.
And if you notice here it failed.
Now let's go ahead and analyze the JSON line format for GPT 3.5 turbo.
So here's the results that I got for GPT 3.5 turbo.
And if you notice here it ran the same tests.
And if I scroll on the right hand side here then if you notice these are the system prompts.
And on the right hand side it has added prompt injections in the exfiltration attempts.
So here's the malicious server.
And then on the right hand side here there is the malicious content for the prompt injection Q equals
to this this this this this.
And yeah.
And then it tries to break the LLM with exfiltration.
And then if you notice all the way down here it passed 57 out of 60 different infiltration attacks that
Garrick tried on GPT 3.5 turbo.
This is amazing where you can test the exfiltration on LLM using Garrick tool.
Thank you.