WEBVTT 1 00:00:00.180 --> 00:00:01.560 Hey there, Ethan here. 2 00:00:01.560 --> 00:00:04.950 And this section is going to be dedicated to security 3 00:00:04.950 --> 00:00:08.370 and specifically GenAI application security. 4 00:00:08.370 --> 00:00:12.090 And I want to talk about the security aspects 5 00:00:12.090 --> 00:00:14.430 of developing LLM applications. 6 00:00:14.430 --> 00:00:15.840 So they can be agents, 7 00:00:15.840 --> 00:00:18.570 they can be RAG applications, they can be both. 8 00:00:18.570 --> 00:00:22.140 But there is a critical security concern which we need 9 00:00:22.140 --> 00:00:25.050 to address while we develop our LLM applications. 10 00:00:25.050 --> 00:00:28.410 So LLM applications are still applications 11 00:00:28.410 --> 00:00:31.920 and everything we know from the application security world 12 00:00:31.920 --> 00:00:35.730 is going to transfer to our LLM based applications. 13 00:00:35.730 --> 00:00:40.050 However, with LLM based applications, we have a new object 14 00:00:40.050 --> 00:00:41.640 that we're going to be working with, 15 00:00:41.640 --> 00:00:43.260 the large language model. 16 00:00:43.260 --> 00:00:46.680 Now, this new object, this large language model, 17 00:00:46.680 --> 00:00:48.690 is going to receive an input text, 18 00:00:48.690 --> 00:00:52.170 usually maybe some other modalities like waveforms, 19 00:00:52.170 --> 00:00:54.030 like pictures, like movies, 20 00:00:54.030 --> 00:00:56.610 and it's going to output us the same text, 21 00:00:56.610 --> 00:00:58.140 maybe pictures, maybe movies. 22 00:00:58.140 --> 00:01:00.870 And just by introducing this large language model, 23 00:01:00.870 --> 00:01:03.566 we introduced a new a attack surface. 24 00:01:03.566 --> 00:01:06.930 Now this attack surface is basically what's going 25 00:01:06.930 --> 00:01:09.983 to be inviting the malicious actor to hack 26 00:01:09.983 --> 00:01:12.270 and to do things to our system, 27 00:01:12.270 --> 00:01:14.640 which is an LLM based applications. 28 00:01:14.640 --> 00:01:18.690 So in this section we are going to go deep into the security 29 00:01:18.690 --> 00:01:20.400 of LLM based applications. 30 00:01:20.400 --> 00:01:23.130 Now, when we talk about LLM based applications, 31 00:01:23.130 --> 00:01:26.010 we have two types of applications. 32 00:01:26.010 --> 00:01:29.100 The first one is going to be an agentic application, 33 00:01:29.100 --> 00:01:30.330 an AI agent. 34 00:01:30.330 --> 00:01:32.430 And this is an LLM application where we're going 35 00:01:32.430 --> 00:01:35.100 to have an LLM, which is going to function 36 00:01:35.100 --> 00:01:38.190 as the decision maker, as the reasoning agent deciding 37 00:01:38.190 --> 00:01:40.470 what is going to be executing later. 38 00:01:40.470 --> 00:01:42.210 There are many types of agents. 39 00:01:42.210 --> 00:01:45.390 We have agents that are fully autonomous, like Claude Code 40 00:01:45.390 --> 00:01:48.900 for example, and there are more limited types of agents. 41 00:01:48.900 --> 00:01:51.360 Usually I like to call them agentic applications, 42 00:01:51.360 --> 00:01:53.550 where we as user define the flow. 43 00:01:53.550 --> 00:01:55.500 But the LLM has the freedom to choose 44 00:01:55.500 --> 00:01:56.820 where to go in that flow. 45 00:01:56.820 --> 00:01:59.070 And in this section I'm going to discuss the new types 46 00:01:59.070 --> 00:02:01.710 of vulnerabilities we have like prompt injection, 47 00:02:01.710 --> 00:02:04.680 indirect prompt injection, tool hijacking, 48 00:02:04.680 --> 00:02:06.752 and many more vulnerabilities which affect 49 00:02:06.752 --> 00:02:08.490 LLM based applications. 50 00:02:08.490 --> 00:02:10.830 And I'm going to show you the best practices of how 51 00:02:10.830 --> 00:02:12.690 to develop a secure application. 52 00:02:12.690 --> 00:02:15.330 So keeping this application security hygiene, 53 00:02:15.330 --> 00:02:18.150 which is a strict set of rules 54 00:02:18.150 --> 00:02:20.430 and architectures, which is going 55 00:02:20.430 --> 00:02:24.390 to make our LLM based application secure by default. 56 00:02:24.390 --> 00:02:27.420 Now I come from a cybersecurity background. 57 00:02:27.420 --> 00:02:29.286 So most of my career I've been working 58 00:02:29.286 --> 00:02:32.760 as a software engineer for cybersecurity companies, 59 00:02:32.760 --> 00:02:34.320 specifically cloud security. 60 00:02:34.320 --> 00:02:37.410 So the security world is something which is embedded in me 61 00:02:37.410 --> 00:02:39.870 by nature, but I know this is not the case 62 00:02:39.870 --> 00:02:41.370 for most engineers. 63 00:02:41.370 --> 00:02:43.950 In fact, for most engineers, security is going 64 00:02:43.950 --> 00:02:46.110 to be the last thing they want to worry about. 65 00:02:46.110 --> 00:02:48.954 We are too busy trying to ship our application to production 66 00:02:48.954 --> 00:02:51.030 and to ship features fast. 67 00:02:51.030 --> 00:02:53.880 So the entire goal of this section is first 68 00:02:53.880 --> 00:02:55.200 to show you the vulnerabilities 69 00:02:55.200 --> 00:02:57.503 and what can happen when we do not develop a safe 70 00:02:57.503 --> 00:02:59.640 LLM based application. 71 00:02:59.640 --> 00:03:02.610 And second, I want to show you what are the best practices 72 00:03:02.610 --> 00:03:05.160 and what we need to do when we develop those kinds 73 00:03:05.160 --> 00:03:08.520 of applications to keep our blast radius minimum. 74 00:03:08.520 --> 00:03:09.900 And for those of you who do not know, 75 00:03:09.900 --> 00:03:12.360 the blast radius is when an attacker 76 00:03:12.360 --> 00:03:15.120 infiltrates our system, what can they do? 77 00:03:15.120 --> 00:03:17.490 Can they access our users' files? 78 00:03:17.490 --> 00:03:19.770 Can they go and run malicious code? 79 00:03:19.770 --> 00:03:22.110 There are a wide range of things they can do. 80 00:03:22.110 --> 00:03:25.320 And our goal when we develop an application to make sure 81 00:03:25.320 --> 00:03:28.140 that this blast radius is going to be as minimal 82 00:03:28.140 --> 00:03:29.910 as possible, and this is what we're going 83 00:03:29.910 --> 00:03:31.590 to be doing in this section. 84 00:03:31.590 --> 00:03:34.200 Now I know there are going to be many new terminologies when 85 00:03:34.200 --> 00:03:36.180 it comes to the security world, so I am going 86 00:03:36.180 --> 00:03:37.410 to cover them as well. 87 00:03:37.410 --> 00:03:39.826 And the overall goal of these sections is when you go 88 00:03:39.826 --> 00:03:43.110 and develop your LLM based application, you'd go 89 00:03:43.110 --> 00:03:44.343 and do it securely.