Hi, and welcome to this AI in C#
video on the Microsoft Agent Framework.
Today we're going to look into OpenTelemetry, where
we'll be looking into the console and application
insights.
And in a separate video, we're going to
look at it on .NET Aspire's dashboard as
well.
A little disclaimer, I'm not too good at
OpenTelemetry, so those of you who are experts
in it, sorry in advance, but at least
this is how you get going.
Because OpenTelemetry have a standard, more or less,
for AI.
It's called gen underscore AI, and the Agent
Framework follows this whenever you set up OpenTelemetry.
So let's go in and see that.
So here we are in the system, and
we are in the telemetry example here.
And what we need to do is, you
can see, these already have OpenTelemetry as a sub-nuget
packages, so we don't need to put anything
extra in here.
But what we need to do is we
need to make a source, a trace provider,
in order to trace whatever happens with every
AI.
For that, we need to give it a
name, so we just give it, in my
case, AI source here.
And then we are going to use OpenTelemetry,
where we add a source, and in my
case, we also add a console exporter, so
we can export the various data out to
the console.
And then I have an application inside set
up.
So I have an application inside set up
here.
Quickly grab this before we move on.
And I'm using one here, and I have
an instrumentation key that is in my user
secrets.
If you don't have that, you can still
run the application, but it won't log to
Azure then.
Then we build this trace provider.
So we have something that can be traced
and logged to two different, or exported to
two different sources.
And the only thing we need to do,
rather than the normal things we do, is
we need to just say, as builder, add
the middleware of OpenTelemetry, where we give the
source name.
And then we need to choose if we
want to enable sensitive data or not, meaning
should the actual answers back and forward be
there, or should it only be something about
what is the model, and how many tokens
was used, and such things like that, which
is non-sensitive.
Microsoft recommends that you don't put this on
in production, of course, but they can't stop
you in doing it.
So if it's for training or anything, you
might do this.
But beyond that, I'm doing nothing special.
The only thing I'm doing here is my
responses are coming out in yellow here.
And we might actually go to red, so
it's even easier to see where they are,
because as I have the telemetry console I
exported here, a lot of extra data will
come out, despite our three texts here.
So I'm telling it my name.
I'm asking for the capital of France, and
I'm asking again what my name was, just
in a normal thread.
So let's set a breakpoint and get this
going.
We are on the wrong project here, so
let's jump to that one instead.
And let's get this side-by-side like
we do.
So we set up the trace, and in
my case the Azure Monitor as well, or
in the application inside that is called as
well.
So that's ready.
We build our normal agents and hook it
up.
And in my case, I'm using the extra
data out, and then I'm making a thread
and do the first request.
And as you saw, a lot of things
came out, and then our answer.
Second request, a lot of things came out,
our answer, and the third one.
So what's in here?
So we see a bunch of trace IDs,
band ID, all kinds of things of what
this was that happened.
In this case, it was an invoke agent
event, and it had some tags about it.
It was a chat.
It was the GPT model that I used.
It was OpenAI from this address.
And in my case, the input message came
in, and the output message came out as
well.
We had the agent and an ID for
the agent.
And then a little more down here with
some of the same data.
And essentially the same with some different input
and output in this case.
And this is what we get also sent
up into Azure.
And what we can do up in Azure,
we can go in.
Azure don't really have our application inside, don't
really have anything as far as I know
for showing GNI data, but we can make
our own logs and our own workbooks if
we want to.
So all the data goes into a table
called dependencies, and most of it go into
something called custom dimensions, which is a JSON.
So I'm just projecting that out using KQL.
And if I run this, I need to
not stand on, so it thinks it's only
a sub layer part of it.
We should be able to see our agent
here where we asked, what is my name?
So that was the last one.
So this comes in descending order.
The second one was the thing about France.
And had we not set the sensitive data,
input and output data would have been empty,
and the rest would have been there.
So I will put this in the description
of the video, so you can grab this
and use it in your own logs.
Or you can use, since OpenTelemetry is open,
as the name implies, you can use it
in other places.
But this is how you do it up
in Application Insights.
In Azure Finder, you have direct integration, and
in Aspire Dashboards, you can get them there
as well.
But that was actually everything for this one.
It's fairly simple to set up.
You can, of course, add logging with normal
logging and so on, but that's normal OpenTelemetry
and not really anything to do with AI
as such.
So with that, see you on the next
one.