This is in continuation with the exact transformation which we have known, so we will be implementing
the same.
So first of all, let us in both Fondas and No.
And this is the fire which is containing all the estimates from the elections.
So this is the fire which we have.
It has a listing of if the e-mail address is a scam or spam and it has the message in front of it,
and both of them are separated by the.
So while reading this fine, I have.
Ready, fiber, and then I provided the delimiter buildup and I was told that there is more ahead and
the needs of the hair that I have provided, I started on message.
So let us view the data.
So this is the data which we have.
This is the target value.
And this is the message value target value is basically the value which we will be predicting when we
will be working on machine learning algorithms.
So this is what we are predicting based on the messages.
So the main focus here is to.
Analyze the message and find out if the message is actually a ham or a spam.
So when we will be implementing some machine learning algorithm on top of this data.
It will provide us a target value on a new treatment, first of all, learn from this data that these
are a few messages and this is the level of time, this is the class which we have provided to it.
So whenever three and three is written, it will try to classify that as a spam.
So what will happen is it will learn from this particular data.
And when a new data unseeingly that would be shown to the model, which we will be training, it will
be able to classify if that this is actually a ham or a spam.
So right now, we will not be going very much in depth in how we will treat the model, but we will
only look at the data preparation part of it.
So right now we are just working on data and how we will actually train the model is the second reading.
So this is the of which we have.
So just ignore this particular step.
We will discuss it once, we'll start with the modern creation.
And.
These are different libraries, which are which we are including the first one is its feature extraction.
Using a scanner.
So it is this idea of victories in this idea of vectorized is what will convert our tax data into a
vector form, into a tabular form, which we are expecting.
Then from the end of the purpose, we are importing the Stopford, so the stop words are the words which
we want to remove from the text.
For example, we have OPIS.
You.
Then.
He.
I all of these are stoppered, so these are all stop words, these are the words which don't really
give a context to the bill.
They do not give a meaning to the doll, but they actually add the number of words.
And because we are creating a vector, we are creating a complete vector out of this data.
We don't want the extra words to be present here.
So that is why we are getting a list of stoppered so that we could remove them from the text, which
we will be having.
Then we are importing the water organizer from and we get organize this vote organizer will actually
convert the text into token forms in the world to conform.
So this entire message are joking with will be converted into OK, as a different word, I would be
a different word, would be a different book and so on.
Then we have more limited.
We can use Wordnik, memorize it also and ostomy method also.
Any of those methods could be used.
Limitation, as we have already discussed, is a more appropriate method because it keeps context in
mind.
So we will be using what mental amitiza here.
So we are creating an object of ordinance and amitiza.
Named as Lemmer.
And we are getting the sack of stuff put.
And we are getting these tough words from English language, you can get these words from any language.
And Lee, let me stop for you, let us see what the words are covered in the books.
So here we have a list of stop words that is E above after again, all because between below then four
did few had happened in is in must myself.
So all the words which don't really give a meaning to a sentence, but the connecting words are considered
good as stop.
But now there are several other things which cause issues like punctuations.
So we can consider punctuations also and I those punctuation to our stoppered so that those are also
removed from the sentences.
So what we are doing here is we are defining a particular function.
So what this function does is the functions name is split into us.
So this will get different limits to us.
So what we are doing is we are forcefully converting the entire message into Lorqess.
After the the messaging to lowercase, we are dividing the message into different words, different
word tokens.
After these words would have been organized, we are thinking of all the different words.
This is a blandest which we have created and we are taking in each word from the message one by one.
So from each and every word in the message we are reading on top of each and every word, and then we
are checking if the word is a part of the word, then we do not do anything.
We just continue with our Falu.
And in case the word is not found in the but then the IV that would pull this particular list, which
we have created.
And after we got on the list, all the words list from these words, the word on the memorized form
of these words.
So we basically keep a collection of all words except for the school books and then memorize them and
send them back.
So this is what we are doing and here what we have is liquid on this view.
So here we have as deeply and as deep this this is just a split of data, which we have done, don't
get into depth of it.
Let's just see what is there in the brain data.
So this is the data which we have the training data which has the target value and the message value.
And after this target value, what happens is the same similar thing is what hasn't been the best data
also.
We've just seen no rain to fight the effect of it, so this is a vectorized which will initially loan
from the data which we have.
So it will to follow on from the data which we have and known what type of birds do I need to have?
What type of awards should be considered so that we will be able to classify some as famous Victor?
They the effect that it is just learning from it now.
So what it does is we are just defining the object of this idea of victimizer and we are just telling
it that you will see this particular functions in dilemma's.
I'm considered the minimum document frequency to be doing the maximum government frequency to be three
thousand, which means that consider only those words which have at least 20 occurrences and it marks
3000 occurrences.
So what this means is, let's say we have a word C or let us see.
We have some word as you and that word is not very frequently occurring.
There was once in a lifetime.
Then I go to estimates of the child up Voynov.
So considering that as a criteria for classifying something as a bomb or harm would not be a good measure
somewhere, which is very used, so we will have to remove all the very rarely used words because it
will basically add to the list of words which we are considering.
It will add to the size of the features which we have.
So we will do those ones now on the other end, when we see maximum value in frequency is equal to three
thousand, which means is legacy.
My name is Satana.
And the name comes in each and every sentence, which I receive.
So whatever estimates I'm receiving, all of them, I'm having the name Taniya, that means that it
isn't even relevant, Wolf, to would you classify something to Hammerstein?
Because everyone has the information that my name is gone now and each and every person would be able
to find out that this person's name is Sonya.
So even the ball dismisses or any kind of misses or the five dismisses, all will be sending out the
message to me saying hi to Nafees by this.
Danielle, please get this.
So I don't want to consider these kind of words.
So this is the reason why we do not consider the maximum data frequency here.
So we will be considering only those words which have a limit to the maximum document frequency.
So we are saying that those words need the judge agreeing on a maximum document of three thousand documents,
whatever is occurring, more than three in more than three thousand dismissals don't consider that.
And right now, we're just specifying the criteria on which our estimates would be loaning, the organizer
would be on how this vectorized would be loaning, this DFI vectorized would be learning.
That is what we are defining at this moment.
So now what we do is.
Now, we'll run this.
And then we will slip this the idea of vectorized on the plane.
So we will saw this on this particular training day that we didn't like the idea of vectorized, please
learn from the messages which I already have and find out that vich message should be considered.
Endou which words should be considered while you are creating this vectorized.
So right now, I'm just straining my vectorized.
I'm just telling the vectorized to loan from this data and find out what does what does it actually
have to to the.
And then after I have done that, I will be.
Transforming.
These deep train.
Messages I will be transforming these messages from a continuous text form into the vectorized form.
So once I read this and then I transform this, I get this training data.
And similarly, I can do the same thing with the best thing they know.
So you don't need to worry about what training data is, what testing these days.
Don't worry about that as of now.
Just focus on that.
First of all, tell them what know learn how it needs to wake the guys from the training data, which
we have and what I want you to consider.
Let us see, for example, how you can understand this is legacy.
I am working on a geological project.
And the majority of Soumises, which I get, are related to geography.
It will be talking about it and then it will be talking about some of some options in some cities and
some regions, and most of them will do this while I have another friend who is working on a chemical
project and they will be having words like nitrogen, hydrogen and phosphorus, those kind of words
with.
Now, what I'm trying to say here is that the world which I will be considering to create a vectorized
would be different from the words my friend would be considering for creating his victories.
That is the reason why we are forced to all training, we are, first of all, footing this idea of
the visit on my brain in detail so that it can learn what what want more than what vocabulary does it
need to consider for creating the vectorized.
And then only it will transform this so that I have a proper vaporizer for my geography related project
and.
My friend has a different reason why his game is really good theology and so on.
So we just heard this, and here is one example of how we can actually visualize this, so although
the damage will be created, this idea of the victims, which will be.
Within a spice matrix form, so it will be creating something like there will be a lot of columns.
And all of those problems will be having some zeros, ones, dos, tres wars and the count of the words
which will be there.
And apart from that, there will be a blind space so that this matrix does not capture a lot of space.
So this is what all victimiser is using and how we create vectors using different.
Now, how we will use this vector, which we have generated into this classification, is what we will
be launching later and we will learn how we will see a different classification models using such datasets.
There to see how we can how this limited English is working, so I created one message that is my Red
Sox on the day of the Sox in the world, and the nation needs more Sox like this.
So we will just run.
This particular port, so I am going message dot law and then printing the message.
Then I'm converting these words into tokens, so each word has been completed and converted into tokens.
Now here I am just vibrating on the boards and then I am simply printing the words.
If they are skimped on appendix.
So you can see in my.
On the in the time, though, all of these words, which are part of the supports have been Skipp and
other words have been upended in the list, and then they are just using the words.
So these are the of the world, so you can see the socks have been converted into sock.
At all the places.
So what this will do is when we are done voting, I know when we are creating the the idea of victory,
then these will be considered as one single word.