WEBVTT 00:00.080 --> 00:00.720 Okay. 00:00.720 --> 00:01.400 Let's see. 00:01.440 --> 00:02.000 Let's see. 00:02.040 --> 00:02.560 Drum roll. 00:02.560 --> 00:04.200 Please bring back up the chat. 00:04.280 --> 00:08.240 Copy this up and do a new chat. 00:08.360 --> 00:10.280 This shouldn't be required because you don't have memory anyway. 00:10.280 --> 00:10.960 But I feel. 00:10.960 --> 00:12.240 I feel a superstitious. 00:12.280 --> 00:13.880 We should have it be clean and fresh. 00:14.080 --> 00:14.720 Send. 00:14.760 --> 00:15.720 Watch the lines. 00:15.720 --> 00:16.480 Watch what happens. 00:16.480 --> 00:17.200 The agent's thinking. 00:17.240 --> 00:18.120 Tick tick tick. 00:18.160 --> 00:19.360 Wow, it was so quick. 00:19.640 --> 00:21.160 Uh, do you think that worked? 00:21.320 --> 00:21.960 I don't know. 00:22.000 --> 00:23.400 Let's, uh, let's go find out. 00:23.440 --> 00:25.360 Let's hop over to Pipedrive. 00:25.360 --> 00:26.120 Let's have a look. 00:26.120 --> 00:29.600 We're going to start by going to contacts over here. 00:29.840 --> 00:32.040 So you'll see I'll just make sure we've got everything. 00:32.040 --> 00:32.680 Looking latest. 00:32.680 --> 00:33.280 Here it is. 00:33.440 --> 00:37.880 We've got here, uh, under uh, people contacts as people. 00:37.880 --> 00:40.800 People has John Smith, who works at IBM. 00:40.960 --> 00:43.440 That's in there, John Smith at Ibm.com. 00:43.440 --> 00:44.560 That looks familiar. 00:44.680 --> 00:46.520 It's in Pipedrive. 00:46.560 --> 00:48.920 There's also organizations click here. 00:48.920 --> 00:51.600 And this has IBM as an organization. 00:51.800 --> 00:54.200 So uh, and with me as the owner. 00:54.200 --> 00:57.120 So we've got that under people and organizations. 00:57.120 --> 00:57.960 That's cool. 00:58.200 --> 01:02.020 And now we just have to go and find out where is Leeds Press? 01:02.020 --> 01:02.780 Leeds. 01:02.780 --> 01:08.540 And here is John Smith one lead and John Smith pops up. 01:08.580 --> 01:10.420 Here are John Smith's details. 01:10.620 --> 01:14.620 Uh, source was the API organization associated with is IBM. 01:14.660 --> 01:16.380 The person is John Smith. 01:16.500 --> 01:22.220 And so the details have all been loaded correctly in Pipedrive. 01:22.460 --> 01:27.460 That all happened in a flash by this pipeline. 01:27.460 --> 01:31.900 And hopefully you've got a good sense of how reliable this is. 01:31.900 --> 01:36.340 Structured outputs basically guaranteed to produce it in this format. 01:36.460 --> 01:40.460 This is just a strict, uh, workflow on rails. 01:40.460 --> 01:42.540 It's just going to do step by step. 01:42.660 --> 01:48.300 And so we can be pretty confident that this is going to work reliably, create the appropriate records 01:48.300 --> 01:50.900 with the right IDs in Pipedrive. 01:51.140 --> 01:52.220 That's a success. 01:52.340 --> 01:53.020 Okay. 01:53.060 --> 01:55.220 Let's save that and go back. 01:55.220 --> 02:02.000 To amplify back into the business development folder, we're going to create another Subagent and this 02:02.000 --> 02:03.600 one is going to be called. 02:03.600 --> 02:06.280 Click here and let's call it the SDR. 02:06.720 --> 02:10.040 Subagent stands for Sales Development Rep. 02:10.240 --> 02:17.560 SDR is like your outbound sales person that goes and tries to nurture and win people over. 02:17.960 --> 02:19.880 That's what we're going to do now. 02:20.080 --> 02:20.480 Okay. 02:20.520 --> 02:23.920 So again we're going to start with a chat message. 02:23.960 --> 02:25.200 On chat message. 02:25.560 --> 02:26.520 Fine. 02:26.840 --> 02:33.840 We're going to have that going into a uh AI agent of course. 02:33.880 --> 02:35.840 So I paused there like it could be something different. 02:35.880 --> 02:39.960 No AI agent, obviously, that's what the course is all about. 02:40.160 --> 02:42.280 Uh, we're not going to have structured outputs. 02:42.280 --> 02:47.600 We're going to use tools this time because we want to give this thing flexibility and autonomy. 02:47.760 --> 02:49.440 That's what it's going to be about. 02:49.600 --> 02:53.960 Uh, and you'll be able to compare and contrast the different approaches for the chat model. 02:53.960 --> 02:57.640 I'm going to stick with open AI, but you may pick your model of choice. 02:57.640 --> 03:02.710 And it's always good to experiment with different models and find out what they are good at, where 03:02.710 --> 03:04.750 they fall behind, where they do better. 03:05.150 --> 03:11.150 I'm going to I'm going to use the the best one right now, and I'm going to stick the reasoning effort 03:11.150 --> 03:14.670 down on low because we don't want it to take a long time. 03:15.110 --> 03:19.670 And so this is what we're going with a nice let's hide this for a second. 03:19.670 --> 03:21.110 And nice setup. 03:21.110 --> 03:22.350 We also don't need memory. 03:22.350 --> 03:24.950 This is just going to take a task and do it. 03:25.150 --> 03:29.070 Um and we're now going to give it two tools okay. 03:29.110 --> 03:33.350 So the first tool that we are going to give it, we're going to actually use the the Pipedrive tool, 03:33.350 --> 03:36.870 the one that we didn't use last time because we use it as a node, not a tool. 03:36.870 --> 03:39.270 It's connected by default to my, my account. 03:39.270 --> 03:46.550 We're going to look at the person records and we're going to have the get many person. 03:46.550 --> 03:51.950 And we're going to have it just return all just have this return all the people. 03:51.950 --> 03:57.070 So look, what I'm about to do now is not like a particularly scalable end solution, because you'd 03:57.110 --> 04:01.010 want to filter, you'd want to bring back a portion of the people, you'd want to do this differently. 04:01.010 --> 04:06.610 So that's part of the assignment for you, is to build this in a way that could be used more productively 04:06.610 --> 04:07.650 at greater scale. 04:07.730 --> 04:11.730 This is going to work fine for now, but you would want to do this in a more organized way and allow 04:11.730 --> 04:15.930 your SDR to be smarter about how it selects the people it wants to work on. 04:15.930 --> 04:22.290 But I'm going to have it draft emails to send to these people to communicate what we have to offer. 04:22.530 --> 04:24.970 Okay, so we're going to have it to get all the people. 04:24.970 --> 04:29.090 We don't need to specify any parameters because it's just going to get the whole lot, get many people, 04:29.090 --> 04:30.650 get all person. 04:30.810 --> 04:31.530 All right. 04:31.650 --> 04:32.970 That's the first tool. 04:32.970 --> 04:37.290 So one thing that might come to mind to you is you may hopefully you've had at the back of your mind 04:37.330 --> 04:39.290 the lethal trifecta. 04:39.530 --> 04:45.810 You've been thinking that we've given this thing access to private data in Pipedrive. 04:45.970 --> 04:51.010 It's potentially getting in things that have been taken from the internet in the form of people's names 04:51.010 --> 04:52.010 and email addresses. 04:52.010 --> 04:53.330 So we don't know where they're coming from. 04:53.330 --> 04:57.290 And they could contain all sorts of bad embedded things in them. 04:57.610 --> 05:03.830 And we're about to do something that's going to communicate externally by sending emails. 05:03.830 --> 05:05.390 And there's a bunch of concerns with that. 05:05.390 --> 05:08.510 One of them is that that would then be the lethal trifecta, for sure. 05:08.790 --> 05:14.310 And secondly, that generally building automated systems that send emails is something which needs to 05:14.310 --> 05:15.510 be taken very carefully. 05:15.510 --> 05:16.150 Indeed. 05:16.150 --> 05:16.870 It is good. 05:16.870 --> 05:22.830 It is completely fine for, uh, a human to send an email, even if that has been drafted. 05:22.830 --> 05:28.430 In fact, it's fine to have it be drafted by machines, but to send out mass emails is something that 05:28.430 --> 05:31.590 is potentially problematic, particularly if people haven't opted in. 05:31.590 --> 05:34.710 And there is, of course, plenty of rules around this, like GDPR in Europe. 05:34.710 --> 05:36.310 But that's not what we're going to do. 05:36.350 --> 05:43.430 We are going to have this agent draft emails and save them in my draft folder for me to review, and 05:43.430 --> 05:48.510 for me to choose to send it out from me, potentially after changing it when I want. 05:48.550 --> 05:51.470 And that is definitely a safer process. 05:51.590 --> 05:56.990 And also, of course, then means that we are not exposing ourselves to the lethal trifecta. 05:57.310 --> 06:02.610 So I am now going to add in the Gmail tool, the tool that we used a long time ago. 06:02.610 --> 06:08.610 Now it feels like it remembers my account details, and I'm going to have the resource be a draft, 06:08.610 --> 06:11.890 and I'm going to want it to create a draft. 06:11.890 --> 06:15.010 I'm going to let the model set the subject. 06:15.010 --> 06:20.850 I'm going to have the email type be an HTML email, and I'm going to let the model define the message. 06:20.850 --> 06:24.330 It's going to be able to create drafts for me. 06:24.330 --> 06:30.850 And that is now defining my second, uh, of my, uh, of of my tools. 06:30.850 --> 06:32.450 And that's the end of the tools. 06:32.450 --> 06:37.610 And you can see that by giving the agent access to this tool, I'm very happy for it to do something 06:37.610 --> 06:41.570 like reading the people decide when and how it wants to call this. 06:41.570 --> 06:44.410 The autonomy is going to be used for good. 06:44.450 --> 06:46.490 It's a good time to use autonomy. 06:46.610 --> 06:49.530 We don't need to put this into a rigorous workflow. 06:49.530 --> 06:50.650 So this is great. 06:50.650 --> 06:53.290 It's time for us to write a system prompt and then give it a whirl. 06:53.330 --> 06:53.890 Okay. 06:53.930 --> 06:56.250 So I'm going to open this up. 06:56.250 --> 07:02.400 I'm going to, uh, change this, actually, rather than a system prompt, this can just be the, the, 07:02.680 --> 07:04.280 um, the user prompt. 07:04.280 --> 07:06.480 We can just say define below. 07:06.680 --> 07:07.600 We can just paste. 07:07.600 --> 07:09.800 I'm going to paste in here the prompt I've written. 07:09.800 --> 07:11.280 It's going to be you are an STR. 07:11.320 --> 07:16.400 Use your tool to read all the person records in the Pipedrive CRM system. 07:16.480 --> 07:19.800 I'm just saying the Pipedrive CRM less is more for each person. 07:19.800 --> 07:25.880 Use your tool to create a draft email from Donna, the CTO of accounting firm account AI. 07:26.160 --> 07:31.280 That's my new job to pitch account AI's innovative accounting software products. 07:31.480 --> 07:33.760 The email should be professionally formatted in HTML. 07:34.000 --> 07:38.320 Explain in the email that if people are interested, they should call blah blah blah blah blah to speak 07:38.320 --> 07:41.840 with our AI agent and schedule a demo. 07:42.360 --> 07:46.200 Okay, so that's going to be our user message. 07:46.360 --> 07:47.240 Save that. 07:47.280 --> 07:48.480 Let's get out of this. 07:48.880 --> 07:50.960 Uh, and of course with this, we don't need this anymore. 07:50.960 --> 07:52.720 We're not using a chat. 07:52.720 --> 07:55.200 So I'm going to take this out as the trigger. 07:55.360 --> 07:58.660 And instead we can have as our as our trigger node. 07:58.900 --> 08:02.060 We can just have trigger manually. 08:02.300 --> 08:03.180 That's fine. 08:03.420 --> 08:09.140 Just, uh, just hook that up like that when clicking Execute Workflow, it will kick this off. 08:09.340 --> 08:10.780 It's time for us to give it a try. 08:10.940 --> 08:12.060 Okay, here we go. 08:12.100 --> 08:13.700 I'm pressing Execute Workflow. 08:14.220 --> 08:15.140 Off it goes. 08:15.460 --> 08:16.540 The agent is running. 08:16.540 --> 08:18.020 It got the people right away. 08:18.220 --> 08:18.820 It's very quick. 08:18.820 --> 08:19.460 Pipedrive. 08:19.740 --> 08:20.820 And now it's thinking. 08:21.220 --> 08:22.260 Now it's thinking. 08:23.740 --> 08:24.700 Still thinking. 08:25.220 --> 08:28.380 It's creating a draft that was done and it's done. 08:28.380 --> 08:29.980 Workflow executed successfully. 08:30.980 --> 08:33.620 All it remains is to go and look in my draft emails. 08:33.620 --> 08:37.540 Okay, well, I've checked my drafts and sure enough, I do have a draft. 08:37.660 --> 08:38.900 Uh, and here it is. 08:39.140 --> 08:45.020 And, uh, it's a nicely formatted, professional looking email to John, and it's all set up here, 08:45.020 --> 08:49.980 and it has the phone number and it mentions it has a nice signature for me with a little if you've reached 08:49.980 --> 08:50.820 the wrong person. 08:51.180 --> 08:56.560 So I think that this shows you the good and actually also the bad of of LMS. 08:56.560 --> 09:00.040 It's it's great that it did it so efficiently and effectively. 09:00.040 --> 09:03.400 But you will notice that it's also invented tons of stuff. 09:03.400 --> 09:07.920 We didn't tell it like detect anomalies and policy exceptions earlier. 09:08.200 --> 09:11.760 And you know that some people might say, look at it hallucinating. 09:11.800 --> 09:16.640 And I would argue, again, AI users can use words like that and can say it's hallucinating. 09:16.680 --> 09:19.920 As far as you or I are concerned, it's not hallucinating. 09:19.920 --> 09:21.160 It's doing its job. 09:21.160 --> 09:22.080 It's an LLM. 09:22.080 --> 09:27.480 It's designed to generate plausible content, and that's precisely what it's doing. 09:27.480 --> 09:31.960 We didn't tell it not to invent something that it didn't, that we didn't tell it. 09:32.000 --> 09:33.880 We just told it to write a sales email. 09:33.880 --> 09:36.320 And this is a realistic sales email. 09:36.320 --> 09:41.680 And so if we don't want it to do that, we have to tell it that and it's up to us to measure it and 09:41.680 --> 09:42.600 correct for that. 09:42.600 --> 09:45.360 So when models don't behave well, don't blame the model. 09:45.360 --> 09:46.760 Blame the AI engineer. 09:46.800 --> 09:48.240 It's your job and my job. 09:48.400 --> 09:53.080 So let's go in and fix that job, and maybe we can try fixing something else while we're at it. 09:53.120 --> 09:54.680 Okay, in here we go. 09:54.840 --> 09:57.140 So let's update this user message. 09:57.140 --> 09:58.900 Let me just expand this down a bit. 09:59.220 --> 10:00.460 Let me update this. 10:01.060 --> 10:13.620 Explain that people should uh, let's say uh, do not include any additional details about about account 10:14.540 --> 10:19.580 AI or the software that you do not know. 10:20.380 --> 10:38.420 Simply state that it is, uh, innovative accounting software that will streamline costs, drive efficiency, 10:39.020 --> 10:40.260 very bland stuff. 10:40.700 --> 10:41.500 Efficiency. 10:41.980 --> 10:43.700 All right, let's give that a shot. 10:43.700 --> 10:50.380 So we've now improved the prompting, uh, and, uh, made sure that we correct for, for for that, 10:50.660 --> 10:52.380 uh, I'm also going to come into this draft. 10:52.420 --> 10:57.330 Did you notice it didn't actually have who the email was addressed to because that's not set by default 10:57.330 --> 10:58.010 in a draft. 10:58.010 --> 11:03.650 I think we can probably force that by coming in here and setting the two email and saying that we want 11:03.650 --> 11:06.650 the model to define who the email goes to as well. 11:06.850 --> 11:08.290 So let's try that. 11:08.330 --> 11:13.770 Let's now go back and give this thing a second run so we will clean everything up. 11:13.770 --> 11:15.450 We will press execute workflow. 11:15.730 --> 11:16.770 Off it goes. 11:17.130 --> 11:18.050 It's running. 11:18.850 --> 11:19.850 It's thinking. 11:21.570 --> 11:23.130 And it's got we got another tick. 11:23.370 --> 11:27.090 It all seems to have run workflow executed successfully will be the judge of that. 11:27.090 --> 11:28.250 Let's go take a look. 11:28.250 --> 11:29.210 Well here we go. 11:29.250 --> 11:31.330 There's the old email again as a reminder. 11:31.370 --> 11:33.170 You remember that that's what it used to be. 11:33.210 --> 11:34.690 Here comes the new email. 11:35.170 --> 11:36.050 There we go. 11:36.210 --> 11:38.570 It's addressed to John Smith at ibm.com. 11:38.570 --> 11:40.930 Exactly the information that we passed in originally. 11:41.130 --> 11:45.090 And it's nice and short and sharp and it only mentions the things we told it. 11:45.130 --> 11:46.650 It didn't make anything up. 11:46.650 --> 11:49.610 It's very clear it still looks professional and crisp. 11:49.810 --> 11:52.610 Now we can properly declare success. 11:52.730 --> 11:55.550 That is our second subagent our SDR. 11:55.750 --> 11:58.870 Built and working onwards. 11:59.110 --> 11:59.790 Okay. 11:59.990 --> 12:00.510 Okay. 12:00.550 --> 12:03.350 So that wraps up week four, day four. 12:03.390 --> 12:05.230 Part of the capstone project. 12:05.230 --> 12:09.950 We built two subagents the Revops Subagent the SDR Subagent. 12:09.990 --> 12:11.910 They both work nicely indeed. 12:11.950 --> 12:16.630 One of them using structured outputs and workflow, the other of them using tools. 12:17.030 --> 12:23.150 And both of them somewhat independently testable and and confirmable, if that's a word. 12:23.190 --> 12:27.510 And now we can say we're ready to move on to complete that purple box. 12:27.510 --> 12:32.030 The business development manager and then the other agent that I haven't told you about yet. 12:32.190 --> 12:40.350 And so with that, we can tick off day four of week three, meaning that there's only one day left. 12:40.350 --> 12:41.670 The capstone day. 12:41.910 --> 12:43.310 Amplify your business. 12:43.310 --> 12:44.350 That's what's coming. 12:44.470 --> 12:45.710 It's coming tomorrow. 12:45.910 --> 12:52.790 You are now 93% of the way through this program, 93% of your way to being a pro. 12:53.070 --> 12:54.390 I will see you tomorrow.