In today's video, let's clarify certain concepts.
We will begin with embedding model followed by vector embedding.
And then we will learn about vector search.
These are the concepts that would help you understand the move framework in detail.
Let's go through the background.
In case of the text data military and army, they have similar meaning.
Even though the words military and army are very different.
For semantic search to work effectively, representation of military and army must significantly capture
their semantic similarity.
This is where vector representations are used and why the derivation is so important.
If we want to compare two sentences, we don't want to compare the words they contain, but rather whether
or not they mean the same thing.
To preserve the data's meaning, we need to understand how to produce vectors where the relationship
between the vectors makes sense.
This is where the next obvious question comes up.
Vector embedding.
Embedding models are built by passing large amount of labeled data to a neural network.
We then train this neural network to perform all sorts of different tasks.
Embedding model processes input as natural language query and returns a multidimensional vector that
represents the contextual information.
Here on the left hand side, shop for table is a natural language query that is passed to an embedding
model, and we get a multidimensional vector representation that represents the contextual information.
In this case, it shop for data and refund fees.
Now how we can use this multidimensional vector is where we cover it in the next topic, which is vector
embedding.
Once trained, an embedding model can transform raw data into vector embeddings that we saw in the.
I'll give you the slide.
A vector embedding is an array of numbers that is used to describe an object.
For example, refund could have a vector here.
More likely the array would be a.
It's not longer than that, but this is a numeric representation of the natural language query.
These vectors of numbers are then used by machine learning model to understand the meaning of the text.
How does this all come together?
On the left hand side, we have best places to visit as the natural language query, which is injected
to the embedding model.
Embedding model processes the and creates a multidimensional vectors here.
What this means is all the similars matching vectors are all put together here, and a cluster and their
distance would be smaller.
Whereas here in this case a refund fees avoid debit card fees.
They would have vector representations which are there close to each other, but they are far from the
travel vectors.
This is what vector embeddings means.
Same applies for the dinner or cooking topic that we have here.
Now the obvious next question is how does this work with the search Similarity measures are a function
distance that takes two vectors as input and calculates the distance value between them.
The distance can take shapes.
It can be a geometric distance between two points.
It could be an angle between the vectors.
There are different algorithms that calculates the distance between the two vectors.
Ultimately, we use the calculated distance to judge how close or far apart two vector embeddings are.
In this situation, once the vectors were calculated and decided where they go on the two dimensional
vector or multidimensional vector.
When the user asks a query what are the best places to visit that would journey to Europe and those
multidimensional vector here?
When a user would ask how to make waffle, that would be closer to cook a dinner.
How to make a cake.
But it would be far from traveling or refunding fees.