Welcome back to
Practical Time Series Analysis. In these lectures, we're looking at
some of the fundamental building blocks, some of the fundamental stochastic
processes that give rise to the sorts of time series we're likely to encounter
in our professional practice. We've already seen moving
average processes, and now we explore autoregressive processes. When you're done with this lecture, you
should be able to describe to a friend or a colleague what an autoregressive
process is, what it's seeking to model. You should be able to simulate with AR or similar environment
an autoregressive process. You should be able to discuss
qualitatively what the ACF of some simple autoregressive
processes look like and we'll tie the autoregressive process
back to our concept of a random walk. Just to recall the notational
environment that we're in, for a moving average process,
we start with white noise, and we then take a linear combination
of several preceding terms in time of white noise, and
build our new state X sub t from those. We're going to do something
that looks similar, but is actually rather different with
the auto regressive process. We'll see what we mean by
looking similar in a moment. Right now let's say that X sub t is
going to be some sort of a piece that we can't model very well, an innovation,
a random shock to a system. Plus a term that depends upon
the history of the system. In other words,
the several states preceding. When we look at history what we mean
is take xt- 1 down through xt- p. We're going to look at these
several history terms. We'll form, again, a linear combination
by multiplying by coefficients. And we'll then put in the piece that
we don't know quite how to model, this random term Zt, in order to
create our new state of our system. The random walk is almost
trivially seem to be of this sort. The state of the system at time t looks
like what it looked like one period ago, so the coefficient in front of
the Xt- 1 can be just taken as a 1, plus some Z of t. We present this right now
just as a quick caution that the auto-regressive processes
won't necessarily be stationary. In fact, we'll try to come up with
some basic conditions that tell us when an auto-regressive
process is stationary. Moving onto simulation, and we can't
stress enough how important it is for you to run many simulations. Look at the resulting traces,
trajectories, realizations, to look at the resulting time series. And also look at the ACF
that you're generating. So we'll set a random seed so that everybody will have the same
data when they run their simulations. We'll take 1000 terms with Phi = 0.4. There;s a little bookkeeping to be done. We'll set up with z as a family of
independent high ID random variables. We're going to say x equals null so that we have the variable named x that
we're going to now start filling. We'll take our first x to be z1, and more importantly, how do we start
building states in our system? x at time t will look like
some noise component, plus phi xt-1, and
we'll fill in slots from time 2 to time n. A little bit of housekeeping or
bookkeeping on plotting, we'll create a time series
object as we've done before. We'll set up our plots so
that there are two rows and one column. We'll plot the time series and
we'll plot its estimated ACF. When phi = 0.4,
there is some dependence upon neighbors. You can look, and at this point,
we should have the intuition to see that this is not just noise, but
that there are some correlations. It's hard to get a quantitative
measure of that just be looking at a time series like this. So we're led to the ACF where we look at
correlations at different lag spacings. And we can see that after two or
three periods, the correlations seem to be
getting down to noise, but we do have a fairly health relationship
over a couple side periods. If we bring phi up to 1,
we're just moving past stationarity and also we'll discuss that later and
also you can see that the ACF, here there is some
various decay in the ACF. But the ACF would stay theoretically
right constant up there at 1. There's strong dependence on history. Let's look at an AR(2) process,
an auto regressive process of order 2, let's go back and look at the formula. We've got some random piece. And I'm taking 0.7 and
0.2 here as coefficients. But again, you should be running these
codes and varying the coefficients. And making observations on how the time
series looks, and how the ACF looks. We're trying to increase
our sophistication. So now we'll call arma.sim. This is a routine available
to us in the stance package. It's going to want a list of coefficients. And arima.sim, as the name suggests, will give you auto regressive
integrated moving average simulations. We haven't really talked about i yet. But we can put in autoregressive terms or
moving average terms and you can see we're putting our 0.7 and
0.2 in for the autoregressive piece. And then we'll do the same
sort of plotting that we do. You can look at the time series up here
and look at the ACF down here, and see that the correlations, you can see
that very strongly in different pieces, the correlations are going to
stick around for some time. The correlations decay at a slower rate
now, than they did just a moment ago. We like our time series to be
stationary for several reasons. There are conditions, and we'll develop conditions involving
the unit circle in a little bit. Right now the geometry suggested
is more of a triangle. And in order for
an AR(2) process to be stationary, we'll state here without proof that phi
2 should be between negative 1 and 1. And there's a relationship between phi 2
and phi 1 that must be satisfied as well. We'll simulate an AR(2) process. As we move here there's
a very small difference. We're going to have phi1 be 0.5. We'll let phi2 be negative now and
look at the time series in the ACF. We'll call arima.sim as before, but now we're going to put phi1 and
phi2 in as variables. That's, perhaps, obvious in that line. A little bit more interestingly,
when we do our plotting, we can put those variables right into
our plot command by using Paste. So Paste is going to create a nice
characterated feed to main, the title of our plot. We'll give it a character string and
then we'll separate by comma, but now we can actually put in phi1 and phi2 just
as variables into the plotting command. That'll save you a lot of time and
a lot of trouble by going back and hoping that you find every phi1 and
phi2 as an argument or as a label. When we look now, we can see that the time
series is jumping around quite a bit. In fact, by phi2 being negative, we're actually introducing negative
correlations into our ACF. So you can see the correlation of
neighbors one step away is positive. 2 and 3 negative. And then it's very hard to tell
if you don't have a formula, but it looks like we get into noise
pretty quickly after that. In this introductory lecture, we've
defined what an autoregressive process is. We've seen how to write it down. We've explored simulations. We've begun discussing
qualitative features of the ACF, but again, not to be pedantic or
redundant about it. Can't stress enough that running these
simulations yourself will give you a lot of insight into how these processes
behave, so try to run as many as you can.