After we state the null and alternative hypotheses.
The next step in the hypothesis testing procedure is to determine the level of significance.
Now, we said before that the level of significance is equal to or the same as the alpha level, which
remember is the complement to the confidence level.
So when we say choose a level of significance, we could also think about this as choosing a confidence
level.
The most common levels will choose will be a confidence level of 90%, a confidence level of 95%, or
a confidence level of 99%, which means then that the alpha values associated with these are 10%, 5%
and 1%.
So when we say choose a level of significance or choose an alpha value, we're talking about choosing
this value right here.
The level we choose depends on how strict we feel like we need to be with our test.
And a lot of that is dictated by our willingness to make what are called type one and type two errors.
So let's actually start there.
Let's say to take an example that we have stated null and alternative hypotheses this way.
Maybe we're running a shipping department at a warehouse and we are hypothesizing our alternative hypothesis
is that the mean order processing time is longer than 14 hours.
So from the time the customer places the order to the time that we send the shipment out of the warehouse
on its way to the customer, we're saying that the mean time to process that order is longer than 14
hours.
So that's what we're hypothesizing, which means the null hypothesis is the opposite statement that
the mean time is less than or equal to 14 hours.
So this is the pair of hypothesis statements that we're working with.
This was step one of the hypothesis testing procedure.
And now step two is to choose our significance level.
So to pick our significance level, we really have to be aware of these type one and type two errors.
So to understand these errors, let's start here.
In any hypothesis testing procedure, we can really end up with four scenarios and we're outlining those
in this table here.
We can either have the scenario where the null hypothesis is true or the scenario where the null hypothesis
is false.
And in both of those cases, we could reject the null hypothesis or we could accept the null hypothesis.
In other words, fail to reject the null.
Now two of these combinations are correct choices, right?
If the null hypothesis is true and we accept it, that's a correct decision.
And what we're saying here is that we may not know it based on the sample that we poll and the statistics
that we calculate associated with that sample.
But let's say in actuality, in the real world, whether we know it or not, the null hypothesis is
actually true.
And as a result of our testing procedure, we accept the null.
That's a correct decision.
We have no issue in that scenario.
We also have one other correct scenario.
This scenario is in actuality, in the real world, it is actually the case that the null hypothesis
is false, which means that we should reject it.
If it's false, hopefully we would reject the null.
That would be the correct thing to do.
And so if the null hypothesis is false and we do reject it, we make a correct decision here.
We'll come back to the idea of power in a second.
But all we want to say right now is that rejecting the null when it's false or accepting the null when
it's true are both correct decisions.
So there's no problem there.
But these other two scenarios are wrong decisions, right?
If the null hypothesis is actually true.
But we take a sample and we calculate statistics for that sample, and the sample leads us to believe
that the null hypothesis is actually false and therefore we reject it if we reject the null when it's
actually true, that's a wrong decision.
And we call that type of wrong decision a type one error.
So if we reject the null when it's actually true, we make a type one error or we commit a type one
error, and the probability of committing a type one error is equivalent to the alpha value, where
that alpha value is the complement of the confidence level, which we've talked about before and which
we outlined here, which means that when we decide on an alpha value, what we're really deciding on
is how much we want to risk committing a type one error.
In other words, if we choose an alpha value equal to 5%, we're saying that 5% of the time or one out
of 20 times, we will reject the null hypothesis when it's actually true 5% of the time will commit
a type one error.
Whereas if we choose a confidence level of 99% and therefore an alpha value of 1% or a level of significance
of 1%, what we're choosing there is that we are willing to commit a type one error, one out of 100
times 1% of the time.
Now, while it's true that we always want to be as confident as possible about our result, I.
Ideally, we would always choose 99% confidence instead of some lower confidence level.
Remember that picking a higher confidence level and therefore a lower alpha value comes at a cost because
the lower the alpha value and we looked at this before when we talked about confidence intervals, the
lower the alpha value, the wider the confidence interval.
In other words, as we decrease the alpha value by increasing the confidence level, it becomes more
difficult to reject the null hypothesis because the region of rejection is shrinking.
Remember before we looked at sort of a graph of the confidence level, in contrast with the alpha value,
we put our confidence level percentage in the middle of our normal distribution, and we said that these
two little regions on either side of this confidence level area together made up the alpha value.
So half of our alpha value is in this lower tail on the left and half of the alpha value is in the upper
tail on the right.
So that's why we say alpha over to alpha over two for each of these tails.
This area in the middle here is what we call the region of acceptance.
These two tails on either side are the regions of rejection.
And what we mean there is that if we find a Z score or a T score that puts us outside of either of these
boundaries here, this boundary on the left and this boundary on the right, if we find a Z score or
a T score, depending on which one we're supposed to be using, that puts us above this boundary over
here in this right side tail.
Then we're in the region of rejection.
We will reject the null hypothesis.
Similarly, if we find that Z score or T score, that puts us to the left of this boundary on the lower
edge, such that we're in the region of rejection on this lower tail, then we will reject the null
hypothesis.
But if our Z score or a T score puts us in this middle area here in between these two boundaries, we're
inside that region of acceptance and we will accept the null hypothesis, also known as we will fail
to reject the null.
And so the point is that as this confidence level increases, so as we widen this confidence level percentage
in the middle, this confidence interval and these two boundaries push out toward either side.
So as that confidence level grows, these boundaries push out away from the mean in the center and that
alpha value decreases.
So these tails get smaller, the area in these tails gets smaller.
That means that our region of rejection gets smaller, so we are less likely to end up with a Z score
or a T score that's actually in one of these regions of rejection.
We're more likely to wind up in the region of acceptance.
And so as this alpha value decreases or as this confidence level grows, we are less likely to reject
the null, which means we are less likely to make a type one error.
Now, the fourth scenario, we can end up with this one here where the null hypothesis is actually truly
false, but we accept it anyway when we shouldn't.
We call that, as you may expect, a type two error and the probability of making a type two error we
call beta.
So if the null hypothesis is false and we're hoping to make a correct conclusion, which of course we
are, then we need to fall in the region of rejection because we need to reject the null in order to
make a correct decision when that null is false.
But as the alpha value decreases, so as confidence level increases in alpha value decreases, and these
two boundaries here push out further away from the mean toward the tails, this region of acceptance
in the middle grows.
We are more likely to accept the null instead of reject it, because it's going to be harder for us
to find a Z score or T score that's going to put us in these smaller regions of rejection.
And so we're more likely to make a type two error.
What we realize then is that as we reduce the risk of committing a type one error, we increase the
risk of committing a type two error and vice versa.
So as alpha increases, beta decreases and vice versa.
And in fact, the only way to decrease the probability of committing both errors at the same time will
be to increase our sample size, which again is not always possible.
So we're always balancing these two risks against each other, alpha and beta, unless we can increase
our sample size, in which case we'll be able to reduce the risk of committing a type one error and
reduce the risk of committing a type two error at the same time.
Otherwise, alpha increases as beta decreases and vice versa, which means that most of the time, because
we're always trying to take the best sample that we can and in theory we're using the largest sample
we're able to.
We're always being forced to decide which type of error is more dangerous.
And the answer really depends on the situation.
What we really need to ask ourselves when we're trying to determine this level of significance is what's
our worst case scenario?
So to take an example, let's think about a factory that produces car parts.
So our.
Factory is making.
Parts for automobiles.
And let's say that this factory has a quality control process in place.
So as they produce parts, maybe some employees in the factory are responsible for quality control before
each part is approved.
In a situation like this one, the factory wants a low alpha value because that means they have to reject
fewer parts.
More parts are going to pass quality control.
They'll reject fewer parts, which means that the factory will save money.
It's going to cost less to make each part, and they'll be able to increase profit, which is definitely
an important goal.
But what that means is that more defective parts are likely to get through, which means they'll be
putting more defective parts onto cars and those cars are less safe for consumers.
So if we are the consumer, we actually might want a higher alpha value at that factory because of course
we want our cars as safe as possible.
But that might mean that we have to pay a little bit more money for the car because the car part factory
has a stricter quality control process in place.
So as the consumer, a higher alpha value means increased.
Safety.
But higher costs.
So we're balancing these two things against each other.
And what it really comes down to is the seriousness of the situation.
If we're making car parts or airplane parts or if we're testing medications for human beings, we're
probably going to want a very high confidence level, a very low alpha value, because the stakes are
really high.
It's really, really important we get this right.
But if we're conducting a survey about whether employees at our company prefer chunky peanut butter
versus smooth peanut butter, well, the stakes of that particular hypothesis testing procedure are
not very high.
We might be totally comfortable settling for a 90% confidence level, and we're very willing to commit
a type one error 10% of the time or one in ten times.
The stakes are not high.
It's okay to be less confident.
But if the stakes are very high, if it's something important, if it's a matter of health and safety,
we're probably going to tend toward this higher confidence level, which means this lower alpha value
or lower level of significance in order to reduce the probability that we commit a type one error,
which means reduce the probability that we reject the null hypothesis when it's actually true.
Now, before we wrap up here, we want to come back to really quickly this idea of power that we said
we would revisit.
We give this scenario a special name.
We call it the power of the hypothesis test, because this is really exactly what we want to do.
Remember here that the alternative hypothesis is kind of how we formulate our thinking, and then we
just state the opposite claim for the null.
So here in this simple example, we hypothesized that our order processing time is more than 14 hours.
So what we're looking to do in our hypothesis testing procedure is to pull a sample from the population.
So we would pull a sample of orders that are placed by our customers and then we would collect data
from that sample to see how long it took to process each of those orders in the sample that would allow
us to calculate a sample mean and a sample standard deviation.
And then from there we'd use those statistics and follow the rest of the hypothesis testing procedure,
which we'll look at in the next few lectures.
But the idea at the end of the procedure is that if we want to support this claim that's being made
by the alternative hypothesis, then what we're hoping for is that the null hypothesis is wrong.
It's false.
If we want this alternative hypothesis to be correct, to be true, that means we want the null hypothesis
to be false.
So we want the null hypothesis to be false and we want the sample that we took to support the fact that
the null hypothesis is false.
And if it does, if the statistics we pull from our sample give us enough evidence that we can reject
the null.
And again, we'll talk about what enough evidence means and how we can make a conclusion that we can
reject the null if the null hypothesis is false as we want and the sample supports our ability to reject
the null, then we make the correct decision here.
And the conclusion of our hypothesis test is that we give support to the idea that order processing
time is greater than 14 hours.
We lend support to the hypothesis we sought to investigate in the first place, and we call that the
power of the hypothesis test.
It's the probability that we will reject the null when it's false, which is exactly what we're hoping
to do.
That would be the best possible outcome of the whole hypothesis test.
So the higher the power of our test, the better off we are, and the power of the test is equal to
one minus beta.
Where beta is that probability that we make a type two error.
So this value becomes important to us as we go forward with our hypothesis testing procedure.
So just to recap, we said that step one of the procedure was to state null and alternative hypotheses.
We did that up here.
For our example, step two was to choose a level of significance, and that's what we've talked about
here.
Choosing a level of significance means choosing an alpha value, which in turn of course means picking
a confidence level.
If the stakes are high, it's important that we pick a higher confidence level.
If the stakes aren't as high, we might be able to get away with a lower confidence level.
So that's what we're considering when we choose a level of significance.
Of course, once we pick a confidence level that comes with the associated alpha value or level of significance.
And so we are consciously making this decision where based on the level of significance or alpha value
that we choose, we are defining our willingness for making a type one error, which means our willingness
to reject the null hypothesis when it's actually true.
So if we've chosen a level of significance of 10%, we're saying that we're willing to be wrong in this
specific way.
10% of the time, we're willing to reject the null when it's actually true 10% of the time, if that's
our chosen level of significance in this example with these hypothesis statements were hypothesizing
that order.
Processing time is longer than 14 hours if the null hypothesis is true.
It means that order processing time is actually less than or equal to 14 hours.
But rejecting the null hypothesis would mean saying that we found support.
That order processing time was longer than 14 hours, even though it actually isn't.
So what we're saying there is that 10% of the time to make the conclusion that order processing time
is longer than 14 hours when it actually isn't.
It's actually less than or equal to 14 hours.
More specifically, when we say we're willing to be wrong 10% of the time, what we mean is that we
are willing for one out of ten samples that we pull from our population to lead us to this specific
type of wrong conclusion.
So those are the first two steps of the hypothesis testing procedure.
Step one state our hypothesis statements.
Step two determine level of significance.
And once we've done those first two steps, our next step, which we'll look at in the next lecture,
will be calculating our test statistic.