Hello everyone!
In our previous video, we asked the question how to play ping pong and we got the LLM generated bot
response.
We went also went into the detail of how the Nemo guardrail runs time.
Execute this on the back end.
So now let me run this one more time.
So I got the response back from the bot utterance saying important to sport that that involves hitting
a ball back and forth with the racket and so forth.
While doing so, if you noticed here, it took almost 5.23 seconds and it took three random calls to
execute this stack in a real world application.
5.23 seconds is a lot of time for guardrails to get executed.
There are ways we can improvise this, and that is what today's topic will cover.
So how we can improvise our LLM execution to run quickly.
For that what I'll do is in our config file I have specified this rails configuration.
What this rails configuration do is it provides a dialogue interaction and it says execute the LLM call
in a single execution.
So do not invoke LLM multiple times.
And it's enabled as true.
That is what the deduction I'm given to the Nemo runtime framework.
And let's execute this.
I got the response back.
But now this time if you noticed it took only 1.93 seconds and just one call.
This is great improvisation in terms of executing guardrails within this one call.
Let's now go through the info logs and understand how this happens on the actual runtime execution.