-: So let's go now to LangSmith
and let's go and review the trace here.
So here we got a long trace.
Now this entire trace, let me show you
how it's going to be matching the architecture.
So let me first go and minimize everything here.
And yeah, so I'm going to minimize it to all the nodes
that are running here
so we can see the entire thing ran for almost 50 seconds
and this is the cost.
It consumed 35K tokens here.
So let me go and compare it
to the reflection architecture here.
So we first start with the responder,
which is going to make the first draft.
So here it's going to have the response field,
it's going to have a critique and it's going to have search.
So this is going to be the draft for our answer
and this is going to be happening here
in the draft note here.
And here we can see the answer
we have here is the base draft.
We have right over here.
We can see we have here the reflection,
which has the missing attribute with missing information
and superfluous attribute with information we should remove
and some search queries we should make here.
So right now the number of tool calls is going to be one,
up until now, 'cause this is going to be a tool call here
and we can even see if we'll go and open it,
let's see the sequence of the, when we called OpenAI.
And if we'll go here to the trace of the actual LLM call,
we can see here it was one invocation of a tool call
which invoked the answer question tool.
So we used the identical object as a tool here.
So this is really cool.
Alright, so this was one tool call.
Let's go and check out the architecture.
After this, we should go and execute the tools.
We should execute the search queries.
So here we went to the execute tool nodes
and this tool node here ran three search queries.
The first one was to query AI-powered
SOC startups venture fundings in 2025.
The second one was autonomous SOC market sizing
and use cases.
And the third one was comparative analysis
of AI SOC platforms here.
Now let me show you something cool here.
They all ran concurrently because tool nodes supports
running multiple tools concurrently here.
So if I'm going to check each tools here,
here we have the start time.
So for the different tools that ran,
the start time stayed the same
because they all ran concurrently.
So after we ran this node, we have now an AI message
with a bunch of tool call results.
So now we go to the revised node.
Let's go back to the architecture.
After we run the execute tools, we go to the reviser,
which has now in its history all of the tools execution,
all of the search results.
It has the first answer, it has also a critique
and now it's going to revise the answer
based on the search results.
And it's going to give another critique
and it's going to add a citation here.
So if I'm going to go back to LangSmith,
the reviser is going to be running now.
We can see here it has all of the history of the tool calls.
Let me go through that.
And we get here now as a response, a revised answer.
So this is also a tool call.
If I'm going to dive deeper here,
we can see this is eventually an LLM call.
If this is the input on first thing
the schema of the revised answer here
with all of the details that we want.
And here after the reviser change,
we have now the conditional loop.
If we counted more than two tool calls,
so we want to go and end.
And if we are not, we want to have another iteration
and we want to go and start executing the tools over here
because here we're going to have different search queries.
So let's go back now to LangSmith, here the event loop
and the event loop returned
that we should go to execute tools.
So we go now to execute tools here.
So notice here, now we have also the reference field
also in our tool call.
And here we have different search queries.
Here we have AI SOC ROI case studies,
autonomous SOC market size on 2025,
and industry adoption in AI SOC.
So those search queries are actually different
from the first search queries we had.
And then we go, we execute the tools.
So after the reviser, we go and execute tools again.
So let's go back to the diagram over here.
We go, we execute the tools,
we then make another revision here
and then we want to finish.
Because by the time we finish the reviser here,
we're going to have four tool calls.
So in the trace here we go and execute the tools.
We run three search queries again.
1, 2, 3 search queries all running concurrently.
Then we go and revise the answer.
And now we go to the event loop.
So after the second revision, let's go and open it.
We go here to the event loop.
And here you would actually expect that
we're going to be finishing the graph execution
because the number of tool calls is greater than two here.
However, let's go and count the number of tool calls.
This is one tool call from the beginning.
This is a second tool call and this is not a tool call.
And up until here we have two tool calls.
And the reason why we go to execute tools here,
because when we execute here, this event loop,
the revised node actually haven't finished yet.
So it hasn't really update the state here.
So if I'm going to go back to the code here,
the reviser node here, this is going to be updated
only after we are going to finish this event loop here.
So when we execute for the second time the event loop here,
the tool count is still less than the max_iteration.
So that's why we don't end.
So that's why after it, we go and do another execute tools.
And by this time after we execute the tools,
even though this is not a tool call over here,
I remind you this is a tool note here,
but we have already updated the state
that executed this note here, the revised note here.
So when we go after these execute tools,
so when we go to this revised note here,
we go back to the event loop.
So now the number is greater than two.
And yeah and now we go to the end here
and now we finish the graph here.
So actually according to the max iterations equals two,
it's actually going to be three iteration.
It's not going to be two iterations,
it's going to be three iterations.
And this was my mistake, so sorry about it.
So eventually we made here three iterations of revisions.
Cool. So this was the Reflexion architecture.
So we saw that actually timing the number of iterations
was actually not that simple.
In the next section,
we are going to be using an LLM as a judge.
So we're not going to rely on these max iterations here.
We are going to rely on an LLM, which is going to decide
whether to make another iteration or not.
And this is going to be in the agentic RAG section,
where you can find the docs in LangGraph agentic RAG,
which is going to be this architecture.
And this is what we're going to be covering
in the next section.