Welcome back to practical
time series analysis. We've been looking at simple stochastic
processes that can be used to generate the kinds of data that we see in science,
business, engineering. We've looked at moving average
processes and in the last lecture or two we're looking at
autoregressive processes. These are processes where the state of the
system depends upon some sort of noise, or innovation, or shock, together with
some recent history of the system. In this lecture, we'll work on expressing
an autoregressive process of order p. As a corresponding infinite
order moving average process. That sounds like we are about
to make things more complicated. But it's actually going to work for
us beautifully as a simplification. We'll use this to find the ACF
fo an auto regressive process. And we'll drill down and
the order aggressive process of order 1. And look at how
the autocovariance structure, the autocorrelation
structure depends upon phi. A little reminder, we take a series of
white noise running variables Z sub c. We'll take the mean of zero and sigma squared is how we'll
denote the common variance. The auto regressive
process looks like Z of T time T plus the linear combination
of states going back several lags. The Backshift Operator we've seen
is a way to express given a current position a longer are so I guess
suppresses look back by a position. So B times X sub t is going to give
you the system t to times t-1. If we apply B squared to Xt would
that the system t, t-2 etc. So if we're going to form an expression
for the auto regressive process, we can say x of t looks like noise
plus five one times shift on x of t. There's your x of t minus one all the way
down and we'll do it for several states. How's this going to help us? Certainly we can start writing
things more compactly. A polynomial that we'll
encounter often capital fi of b is just how we would write the noise if
we want to express noise as an operator on x t and this actually is
going to be useful for us. We could write Xt as 1 / 1- this
polynomial term right here. And then we make the important point
that we're going to try to express that as an infinite order moving average. So we'll have the time delays and
suitable coefficient operating on Zt. If that's a little too obstruct, let's
take a p equals 1 first order example. Current state at time t
is going to look like some noise plus pi times
the state at time t- 1. But we can just apply this to t-1,
this theta times t-1. So Xt is Z sub t +, now we'll take phi and we're going to expand t sub t-1,
the state of the system one period ago. As noise one period ago. Plus phi times C to
the system two periods ago. Nothing's stopping us we're doing
the same process for x sub t minus two. So we'll get to see that the system looks
like noise plus phi times noise one period ago plus phi squared
times noise two periods ago. And, of course, we're going to have to
accommodate the state of the system three periods ago,
when we do our expansion. I've only gone down for a few steps but
I think you can see the pattern. This might be a little easier to see
if we use the operative notation when we do the operator approach. We'll take system state at time
t looks like noise plus phi times the state one time period ago. And now we're going to treat our operator
almost like it was a number, and we're just going to do algebra on it. X sub t is 1 over 1- this term here,
times Z sub t. And let's expand to that around the way
we would through geometric series. I've got 1 over 1- pi times B. And I'll take that as pi B +
pi squared B squared, etc. And of course,
this is the formula that we'd be use. 1 over 1 minus a is the infinite sum 1 and then we'll keep raising our number
to great and greater powers. We're treating 5 times
b as the number a here. So why did we do all this? Because now our results are,
one would say almost trivial. The expected value of an order
aggressive process of order P will look at the expected value of
state of the system times t. That's the expected value of
these sum here of operators and proficients are reading on Z's of t. The expect value of course will
distribute over the sum will pull our coefficient through the expected value
operator and will also take the B's. And we'll apply them to the Z sub
t to the noise to get B time Z sub t is noise period to go B squared
Z sub t is noise two periods ago etc. We are left with the expected
value of X sub t looks like. The expected value of
the noise at the current time. Constant time's expected value
the noise one period ago etcetera. That's an infinite sum. But all these expected
value terms evaluate to 0. So the expected value of our auto
regressive process in fact is 0. The variances the same
approach basically the same. Manipulation except when we take the
variance and we apply it through our sum, we'll have the variance operated
distribute over terms in our sum. That's true and I've taken liberty of
applying the time shift operator as well. But constants come through the variance
operator as constants squared. So theta 1 will come through as theta 1
squared, theta 2 is theta 2 squared, etc. The variances are constant,
I'll call it sigma sub z here, squared Just to remind us that this is
variance of the noise random variable. And each of these will produce
a sigma squared term, so I'm going to pull those out. And we're left with sigma
squared times this sum. Evidently we have the necessarily
condition for stationarity. We would like that variance
to be constant, and so we need that infinite sum to converge. Moving to the autocovariance, and we'll get to the autocorrelation
in just a moment. We can take the autocovariance
looking as that constant variance and
then we have a sum of terms like this. Remember we're only going to get
contributions where the variables overlap. Four and A are a P process then,
we'll get sigma of K looking like constant term we're taking the order Q up
to infinity so the only thing that really changes moving from here to here
Is the infinite limit up top here. It turns out that this gives us a very
simple formula for the autocorrelation. We're going to get a fair
amount of cancelling and we'll have terms that look like this. Let's see some examples. If we have an AR(1) I have phi
here as a generic coefficient. When we apply our formulas, then the theta
i becomes 5 raised to the ith power. Theta i + k becomes phi
raised to the i + kth power. Our sum is on i. That's the variable at
the site of our sum. The K is constant, so I'm going to pull pi
raised to the Kth power out of the sum. You can see that right here. Leaving us inside of the sum with pi
raised to the I times pi raised to the I. So we have a simple formula for
the autocode variance We have an even simpler formula for
the auto correlation. Its just phi raise to the kth power. Now if we have our coefficients
looking like 0.9 and 0.2 we can see that when our coefficient
is 0.9 I've called it out for this graph, this is the phi for
the proceeding graph. We'll have something that
to case rather slowly. You can imagine as
the coefficient approaches one, that the decay will become slower and
slower. If we have alpha point two,
that's rather a lone number. The correlations are going to
dot off really quickly. What happens if we have
negative correlations? If our coefficient is negative as we
raise it to succeeding powers, we'll see that our auto-correlation function becomes
negative, positive, negative, positive. Here the correlate or
the coefficient is point 3. So you get rather rapid decay. When the coefficient is -.8, the alternating character
looks very pronounced. And the decay is slower. What we've learned in this
lecture is that an AR(p) process can be expressed as an infinite
order moving average process. This has the advantage of making some
theoretical results very easy to show. We're able to find the ACF Of
a what a regressive processes way. And we've seen how
the coefficient phi is going to determine the ACF for
first order auto regressive process.