Welcome back to Practical Time Series Analysis.
This is the last of our gentle review videos, where we're going over some of
the concepts you would have studied in your introductory statistics classes.
This one particular deals with correlation.
Correlation is a really critical topic for us.
As we study time series, very often we use perhaps the most important
graphical tool that we have, the ACF or auto correlation function.
In order to meaningfully interact with that,
we have to have a good understanding of what correlation is all about.
There are many ways to measure linear association, or
rather the association between two variables.
Linear association is very common and perhaps the most common of all
is Pearson's product moment correlation coefficient.
That's what we talk about in this video.
Specifically we'll review how to plot data
In such way as to make a quick visual interpretation about whether we think
there's a linear association between the underlying variables.
We'll look at the formula for co-variance and for
correlation and we'll try to understand where the definition comes from.
And I'll try to convince you that this is really the definition
you would come up with if you took some time to think about it.
It's nice to have an example to guide our thinking.
We'll look at the trees example,
which should be available to you just by opening R.
If you do the help command on trees, You'll see that we're looking at
a relationship between girth, height, and volume for these black cherry trees.
I think of this as volume speaking to the commercial utility of a particular tree.
How much lumber are we going to get out of the tree?
That's an interesting thing and
when you're out in the middle of the woods, it's hard to predict except
we could measure some variables like girth and height.
Those are things that you could get through with a tape measure or perhaps
a Biltmore stick, as I remember from my Earth Science class in middle school.
Those are very, very, easy to obtain and
the question is, can we use them to make predictions about volume?
The pairs plot that I've got here off of this data set with a couple plotting
commands, in particular you can see that I'm using red dots, tells the story.
Girth is very strongly associated with volume.
Girth is a really great predictor of volume in these trees.
The height of a tree is also a decent predictor.
Not surprisingly, as the height increases so
does the volume that you're going to obtain.
But really girth is the strong predictor.
Let's calculate the covariance.
This might be a little surprising.
The covariance between girth and volume is just a little under 50.
The covariance between height and volume is actually more.
That's not consistent with the pictures that we just saw
unless you start thinking about the units that are involved.
When we take a correlation
we try to look at the relationship without worry about the units.
As we switch from yards to feet to miles to kilometers,
we're going to change the covariance, but the correlation should remain the same.
And reassuringly here, we see that the correlation between girth and
volume is really quite high.
In these pictures, we try to understand where a formula measuring strength of
linear association might come from.
You'd probably agree that on the left we have a set of
data points which fall very close to a straight line.
Not so on the right.
I've created a sort of local set of axes here based upon the averages.
So if you take the average y value, you can put a horizontal line.
The average x value, you can put a vertical line.
And you see that the data is falling really predominately in the first and
third quadrant.
So think about what a deviation might look like.
This is an ordered pair, there's deviation from the x,
there's a deviation in terms of the y.
This data point is above average in x and it's above average in y as well.
So we'll look at the deviations, we'll have positive quantities for
the x deviation and the y deviation.
If we're going to multiply those together, positive times the positive is positive,
we would still get something positive.
We'll do that for every data point.
Down here,
where we also have quite a few data points, the x values are below average.
The y values are below average.
Your deviations in x and in y are both negative.
Negative times a negative is a positive.
So if we take some sort of cumulative measure
by say adding up all of the products of the deviations,
we're going to get something that's contributing coherently.
Look at the second quadrant and the fourth quadrant.
Here x values are below average, y values are above average,
negative times a positive is a negative.
Look in the fourth quadrant, the x values are above average, the y value is below.
And again, we would have when we take the product of the deviations,
we'll have something that gives us a negative value.
The positives clearly out number the negatives and
we would get strong contributions towards covariance.
In the figure on the right,
each of the quadrants seems pretty well equally stocked with the data points.
Positive contributions in quadrants one and four, negative contributions in
quadrants two, I'm sorry with one and three will get positive contributions.
In two and four we'll get negative contributions and they kind of cancel out.
We would expect a strong
covariance in the first picture and a weak covariance in the second.
The formulas that we typically use reflect the conversations we just had.
When you look at data, your covariance will be a sum of deviations in x and
y and we can even take an averaged quantity.
Instead of dividing by 1 over n, the number of data points,
we'll come up with an unbiased estimator by dividing by 1 / (n- 1).
The corresponding formula, a little bit more theoretically or
when considering random variables, is to look at the covariance as an average,
and expected value of the centered random variables.
The correlation moves the same sort of way.
For the correlation for random variables,
we look at an expected value and averaged quantity.
And here we do some centering and some scaling to get rid of those units.
You'll recall on your elementary stats course, that if you have a data point
minus a mean over a standard deviation, we're talking about standard units.
Very often, people use the letter z to represent that.
For data sets, no surprise here.
We'll estimate the standard deviation and we'll do our centering and scaling.
There are more compact formulas that we can come up with if we
introduce sum of squares notation.
We've seen this before.
There's a definitional formula, (x- x bar)(x- x bar).
But there's also a corresponding computational formula that you get
just by doing a little bit of algebra.
This allows us to write our covariance and our correlation much more compactly.
What we're doing here is substituting in the formula for
the standard deviation in terms of the sums of squares.
And then just noting that we can see a sums of squares up in the numerator here.
There's sums of squares in the bottoms.
We cancel all of the n – 1 terms to get rid of some clutter.
And at the end of the day,
the correlation can be expressed rather simply in terms of sums of squares.
In this video, we took some time to recall pairwise plotting.
That gave us a nice visual way of trying to assess strength of linear association.
We looked at the motivation behind the calculations for covariance and
correlation.
And I tried to convince you that the formulas really do make sense.