Hi, and welcome to this ANC SHARP video on the Microsoft Agent Framework.
Today we're going to look into a unique feature that Google Gemini have,
and that is integration into Google Maps.
I haven't seen any of the others have any maps integration,
but since Google owns Google Maps, of course,
you have the option to actually integrate into it.
So let's see that in action.
So here I am in a sample,
and that sample being the Google Gemini specific features and Google Maps.
And we make a Google Gen AI client as normal with Google,
and turn it into an agent like we do.
Normally we set tools using the normal tools and AI tools,
and AI function factory, and so on.
But these tools are very special to Google,
so we need to go in and do what is called breaking glass.
So inside the chat client agent options and chat options,
we need to set the raw representation factory.
And if we do that to generate content config,
we get access to some other tools, not AI tools, but the Google tools.
And then among those tools, we can make a tool called Google Maps.
We have very few options here,
but among one thing, we can enable a widget.
We'll talk a little more about what those are at some point.
But if you do this, and let's just run it to here,
we can then ask a question, for example, like,
what is the opening times of Hard Rock Cafe in New York?
And tell me if it's wheelchair access.
So if we do this, it will go off and talk with Google Maps.
And once it comes back,
we can see we get a response back.
Let me make this a little bit bigger so we can see.
So it talks about Hard Rock Cafe at Times Square,
where it's located, the opening hours, and how the wheelchair access is.
Apparently, there's two of them. I didn't actually know, but fair enough.
So what we can do in order to actually look a bit more into this,
instead of just what is being shown here,
which has been taken out of the response,
is we can go in and take our response,
get the raw representation factor of that,
which is a chat response, still Microsoft.
But if we go in and take the raw representation of that,
we get back to the generate content response,
so the output of this.
And once we have that, we can go to something called Candidates.
And inside the Candidates, there's only one in this case,
we can get grounding metadata.
So if we get that metadata,
we can get what is called the Google Maps Widget Context Token.
I haven't tried this out in real life,
but according to the documentation,
you can use this token together with their JavaScript API
to get something that looks a little like this
in your AI application, if you wish to.
Alternative, you can go in and get all the grounding chunks,
meaning that in this case, we have two grounding chunks,
one being for each of the two Hard Rock Cafes.
And if we go through each of them,
we get a URL to Google Maps, so we could go to this URL. Let's do that.
And we will see that we get to Hard Rock Cafe in Google Maps.
So we can get links to the thing,
and we can get a title, which is just the Hard Rock Cafe.
And we can get a bunch of extra information
about that specific location.
And we are talking really lots, type, restaurant, business, price level.
All the things you find in Google normally is also here,
including the wheelchair access,
and that's the reason why they were able to give us back that information.
And then there would be one more, and one more, and so on.
And for each of these, you can see these small one and two,
that is just referencing each of these grounding chunks,
so you can link the data to the grounding chunk.
But if we do it like this, it will give us back all this extra information
if we wish to also show that to the user in some way.
It only comes back in this format,
it's not in a structured format, unfortunately,
but it's still better than nothing that we have all this information.
And once we have gone through that,
getting the second Hard Rock Cafe here,
and all its information.
So if there's only one result, you only get one of these,
if there's multiple results, you get multiple of these.
A little about pricing of this,
it's actually the same as the web search,
you get 5,000 prompts for free, and then $14 per 1,000 queries.
So it's a really cool way if you have something
that needs to integrate with restaurants,
with sightseeing and such things in the system.
So that's everything, see you on the next one.