OK, now let's have a look at the solution for the safe travel classification problem.
So here's a data science field.
It has basically gained strength and we have a lot of statistical libraries which can help us to compute
regression or classification of data.
Now, actually, what the people who the data scientist, which are present in insurance companies,
they were called actuaries a lot of time ago.
So they used to collect data from different sources and analyze that and claim data to identify a fraudulent
transactions that led them to classify or keep the premiums.
So if anything, data science technology of today has given far more tools to perform their analysis
very easily.
So the data here has a few ordinary categorical data that needs to be fudged and digitized properly
here.
Our goal is to predict a binary outcome of one which indicates a safe driver and zero which will indicate
that the drivers do that needs to be reviewed.
So if the driver is safe, then it means that this particular accident could be a genuine accident if
the driver is not thief.
And that could mean that there could be some problem with this particular data or there could be some
flaws in the data.
So this could be a fraudulent insurance claims.
So we will look at the Cartman's variables and we will fill in the missing data with the mean or median
in order to not you research now after cleaning up the data and filling the missing values.
We will look at the features and their correlations so that we can drop highly correlated data, which
may impact our research.
So what we are doing here is we are importing all the important Vitan libraries and packages and importing
farmers and nubby for data frames.
My third live for visualization and is known for statistical algorithms understanding of the need.
So these are all different inputs which we have done here for.
What we are doing is we are getting the same driver data set from this Excel sheet.
And next, we are finding out the data.
Their safe driver is less than zero, that the value is less than zero, and finding the sum of it,
checking if there are any values which have negative, negative values.
So these are different values.
So for that, the idea of having this value target has this value and these are different values and
don't really look much useful.
So next, we are checking if there are any time this particular data set.
So here this is the driver info.
So these are all the data that we present here.
Here you can see there are a lot of of in 64.
One, two, three, four, five, six, seven, eight, nine columns which are of object type.
Then there are seven in 64 and one fluid column.
Apart from that, these do not have any value.
So we are actually good at that particular front
here.
The count of miles driven and will bucket has only lesser income.
Now let us describe this particular data on the see the numerical variables.
So I.B. has values which could be dropped target.
You can see 25 percent to 50 percent.
There are a lot of zeros, but after 50 percent, you can see most of the values and unhappy ones in
them.
Next, we have Engeneic B, which you can see there is there are a few outliers in genetic.
Credit history looks good.
Your experience, again, looks fine.
This is also fine.
There seems to be a strange increase in the miles driven annually.
So this looks like there is some outlier present in miles driven, unwieldy size of the family.
Looks fine.
So now we will have a look at the credit history, so these this is basically the credit history.
So this is the plug for the credit history.
So here you can see.
Now we will check and see if we have an imbalanced class, so the truth claims is twenty one thousand
three 96 total number of records is 30000.
So the percentage of true claims is almost 71 percent.
So this means that our dataset is imbalanced and 29 percent of the detail will be having false claims
and 71 percent of the data will be having to claims.
So we will have to use some weighted classes here.
So we will balance later using this technique.
And the dataset contains several categorical data and that ends with a bucket that needs to be either
dropped or converted to numerical value.
So we need to get rid of the categorical deny.
Then we will drop those columns or we will convert them into dummy columns, as we have already discussed
how we actually do it.
So these are different categorical columns which are gender, marital status, Reagan type age bracket,
Indian market expedients, vukic miles driven annually by credit history, market state.
So among these categorical variables, we will retain gender, marital status and age bracket.
Regarding the engine bucket, your experience, -- miles driven market and credit history market.
They have a corresponding Continenza even if you see these values.
These have continuous values.
So what we do here is we will keep the age bracket as there is no continuous variable to be present
each.
So now we can split the data by state and analyze each state differently.
So for now, we will drop this date column and analyze the data across the entire nation.
So we have dropping these columns, which is in June is freemarket years, miles driven and credit history
working for now.
And we will analyze this data.
So here you can see this is the target, one zero one zero one gender.
If an engine HP credit history, your experience and you claim to see this vehicle like my instrument
size of family and age bracket and state.
Now, if.
We take the median of the legal papers truck as all the order number values for truculently.
So basically what it means is that if we are talking about a particular you have particular vehicle,
then for that we can take the median of see if we are talking about truck, then the median of the trucks
could be the price for the bus.
Similarly for car, the median of the cars would be price for the car.
So if you want to impute values, if you want, you know, value, so that you would have been sort
of legal status.
So if we have practic the median of trucks and that if you have got the median of cars and imputed that.
So these are the values.
So we have four cars.
You have targeted one truck.
These are these values in GeneDx before God is 140.
Truck is 150.
You these 130, one is 120.
So you can see that the engine h.b is also within the door likely.
When we talk about the credit history, it is also meeting with the Wakeling.
When we talk about car and truck, it is almost similar.
One utility and man, it is also similar amongst these.
When we talk about years of experience, God in Iraq has almost seven and four utility and man, it
is almost seven.
So you can see there is a little similarity between these two.
So when we talk about God in Iraq, the values are a little similar for these.
When we talk about utility man, the values, what these two are similar.
So you can see the relationship being created here.
So we will replace the not no values in miles driven and we leave with the made median value for prop.
There may be a better way to build these missing detail, but we have just it not a no sense out of
some 30000 plus rules, which is less than zero point three percent, same building with median does
not make much difference.
What we can do that very simple.
Next, we will check for the values.
Are there any other values?
Then we will impute them again.
So here we don't have any values, all the values have been imputed, so when we have a look at the
feature values, these different values above the range of values of each other within the.
Right, so you can see that the values are ranging a lot, this has won this as well is 695.
This is seven one one three one four seven.
This is four.
So these are not skewed accordingly.
So we will be using preprocessing Botkin from Escalon to scale all these columns accordingly.
And after scaling, you can see we have scaled these columns and we still have these categorical columns
which are still present here.
So next, we are using data visualization for finding out the distribution of the features and also
the correlation between different features.
Now, after that, we can drop one or two features based on the distribution of correlation accordingly.
So for now, we are dropping the target.
OK, and we are creating the scatter drop scatterplot for the bi weighted relationship.
So here we have different columns that is Engeneic be created, history, year of experience and will
claim miles driven.
And you can see how the relationship is present.
So this these are almost uniform.
This is uniform, this is uniform.
This is also uniform.
These are these are also no relationship here.
You can see between credit history and in general, there is some relationship.
So these lines are actually showing relationships, but we don't really know the magnitude of them.
So we can't really take a decision as of now.
So we will create the heat map or find out the correlation coefficient for this and still got an idea
that these are actually correlated.
But the magnitude we still need to.
So here we have the magnitude of the correlation coefficient.
So here you can see that the target I'm imagining to be has minus 22 096 Credit History and zero point
zero zero two one.
So the features are not highly correlated with the target, so we can keep the remaining features as
it is.
So we are keeping all of these features now next to what we are doing is we will have a look at the
relationship between a dependent variable with the category could be so beloved by many variables.
But now we will take into consideration the category of individuals.
So we will applaud the book's plot.
But the target variable, so these are the box plot.
So here you can see no much difference.
Again, no much difference.
This, again, has not much difference.
This is again, similar.
This is again, similar that only a reading one is the size of the family.
So with the with the size of the family, the dog is slightly really.
So as the size of the families increasing, the driver actually is changing, so you can see that the
box plot indicates that there are some outliers in the engine, be a lot of outliers, actually, credit
history and miles driven.
So here and miles driven again, there are outliers to entry, but we need to keep the outliers unless
they affect our results and take another look at them later.
So we will take care of it later.
Then we separate our feature set from liberal target.
We convert all of them into numeric variable and split the feature and do our training and test data.
So we will convert the categorical features into numeric by giving the best to each variable.
So here we have Jundah.
We are creating one two p.m. to Boomi.
Michael, she does one a single to us, Mark Wacol type.
We are using label encoder and for each bucket again we are using label.
Now we are not using dummies or one word encoder because this creates sparse matrices and increase the
dimensionality by giving one or two farseeing.
Those details, we are giving higher returns to Modig by assigning a value to.
Similarly, we are giving higher ratings to me here.
I'm.
Now, let us go hit these images of the damage we have now, and after converting these into a label,
encoders and numeric labels, this is the final data which we have here.
Now, we will drop the target column from the training data frame as that is available, and we will
create a new data frame by which will have the target value.
Now we will use all dummies to resolve the categorical data steeped in the numeric values.
And next, as we have already found, their target levels with just 70 percent failure.
That is a big driver where the value target is one and 30 percent success, where the person is a good
driver or a target is zero.
So now we will balance the class using smoke.
OK, so we are simply modeling smoke from erm I, I am beknown but oversampling.
So we are simply sampling the data for the.
So after sampling the deductible's, basically create equal dataset here.
So the length of oversample data is forty two thousand seven hundred and ninety two.
The number of negative plus is oversampled in oversampled data is twenty one thousand three ninety six
and number of positive plus in oversampled is twenty one thousand three ninety six.
So now we have created equal.
Basically we have created a balance between the classes.
So we have some oversampled one of the class so that we could have the same number of rows of data for
the class.
This is what this small has given us.
So we will have to find out how significant other features are in predicting our label.
So we will use the feature important method from the line of what is classified.
So after that, we will need the importance of the features using a.
Again, you can add a random randomly will in saying this and all and find out the central importance
for all the features which represent whatever columns are falling under or below the value of the random
column.
You can remove those columns and keep whatever is above the random column in the feature importance
list.
So this is the quickest and most stable way of doing so.
What we do here is we are finding out the random forest and from the random what is we are finding out
the feature importances and you can see these on having very low importances.
That is almost zero.
And these are highly important.
So these are my different variable importances.
So credit history is Hyman's driven and it's high in December expediencies, a little gem that is lower
than that age bracket vehicle, Gapminder Souters size of family, an annual theme is.
What we can do for the rest, we will run the different models on top of it, you can run the logistic
regression, then run the of random Flores decision tree and so on, and find out the best model that
you get out of it and then create the confusion with things accordingly.
From the confusion, we.
Again, you can broaden the scope here.
I have gotten zero scope here.
So here you can see this is a vehicle which has been generated.
So I get zero point six six percent out of it.
So our confusion matrix based on the decision does not look good.
So it is showing high number of false positives and false negatives.
So next, we are building a support vector classifier and from the support vector classifying yet again
would the bagenal itself, which again is a lower performance in comparison to the previous one.
So SVM classifier returns lesser, less than better results than the decision tree.
So we will try stochastic gradient descent, classify the system and the glass of wine, which we have.
And similarly, we can try different algorithms.
And from all these algorithms you can build and find out the best one.
Find out which one works the best.
I use that one.
So here I have Waldo precisions full of summaries for all of those and then the comparison of all of
these.
And you can see that the gradient is performing pretty well here.
So that is something which I will be using.
And here again, you can see gradient boosting is actually performing pretty well.
So that is the one which we will be picking up.
So we are using really interesting in this particular solution.
But you can again use any combination.
You can use gradient boosting your mood and maybe you can use or three also and create a stack out of
these and use them so that it's completely up to you how you want to solve the problem.
This is just one of the solutions.