Welcome back to
Practical Time Series Analysis. We're looking at stochastic processes and
their realizations called time series. And in these lectures, we're looking at
them through the lens of stationarity. Stationarity is a crucial concept for
us and it's a very important idea that allows us to try to say something
meaningful about the stochastic process, a complicated mathematical object
based upon a single realization or a time series. Perhaps a day that's set
that you have acquired. This is something you
can't do with a coin. If you have a coin and
you observe tails on one toss, you can't really say anything
meaningful about the coin or at least the distribution of heads and
tails. All you can really say is that yes,
this coin can give a tails but you can't say anything beyond that. So, stationarity really helps
us to get some good work done. We're looking at stationarity through some
very simple examples as we get started. These are more mathematically oriented. And as we move through the course,
we move more into data sets. Right now,
we're thinking about white noise. White noise will be trivially stationary. Random walks,
which will not be stationary. And we'll look at an introduction
to moving averages. These will be stationary processes. Recall the definition. Process is weakly stationary if
the mean function as we look up and down the stochastic process and look
at the average going on of each point, the mean function is constant. It is the same everywhere we look. The ACF, the autocovariance function,
but depends just upon lag spacing. Again, it doesn't matter where
you are along the process. If you have two random variables and
you would like to know their covariance, all you need to know is how
far away they're separated. Not where they are along the process. As promised, white noise is stationary. If you think of a random variable family,
let's say a set, a sequence of IID random variables,
they might be normally distributed but really they don't have to be. All we care about at the moment
is that they're independent, identically distributed with mean of 0 and
constant variance. Then the mean function,
as a function of index t is 0 everywhere, so of course it's constant. If you look at the autocovariance
function, gamma of t1 and t2, then we find that that's essentially a delta function,
it's 0 when t1 and t2 do not agree. In other words, when you have two
different random variables and as sigma squared, it reduces
the variance when the subscripts agree. So, almost trivially you could
say white noise is stationary. Random walks on the other
hand are not stationary. Let's build a random walk off of
a family of IID random variables. I'm using mu and sigma squared for
the mean, and the variance for each one of the random variables. Mu could be 0, but in general,
we'll go with a generic mu. We build a walk in t steps as your first position will be just where
you got to off of your first variable. Your second position is where you get
to by adding your first position and now taking another step of
size to be determined by Z2. And we continue in that way, moving to
the left or the right in random amounts. Your position at any time,
t then, is just the sum, the aggregate of all
the individual steps you took. A journey is really just the sum
of its individual steps. When we explore the expected value
as a function of index t here for our position x. Then and it can encourage enough
to think about expected value, not really as a number associated with
random variable but more as an operator that will make many variable manipulations
much, much simpler to comprehend. Then we take an expected value of sum
of these independent random variables. The expected value operator
moves through the sum. That is an appropriate independence
that just happens with random variables generically. And we find that the expected value
of position looks like t times Mu. In other words if mu is not zero,
the expected value is growing with time. Same for the variance. Since the X sub t is built on a family
of independent increments here, the Z of t, then that will allow the
variance upper to move through the sum. In general, it won't, now that there's
a dependency structure among Z. But here, we started with independent
identically distributed random variable. So, the variance operator moves
to the summation, no problem. Variance grows with time. Variance is increasingly
linearly with time. To have a meaningful process, we won't
take sigma squared equals to zero. So, you are seeing that
the variance is not constant. If the variance isn't constant,
your process is not stationary. Another one of the canonical
stochastic processes has to do with taking
a family of random variables. We'll work with IID, independent
identically distributed Z sub t. We'll give them zero mean and
constant variance. We'll define a moving average process of
order q as this called as X of t is equal to a linear combination
of the underlying Z's. You can center your notation
by looking at Z of t and then moving up and down along Z of t. But we'll follow the notation,
the convention that says that X is a function of index
t is equal to the noise at t plus the noise at t-1 and we're giving
a certain weighting as we move through. There are different sets of beta that
people like for different processes. You might have an image and
you might be smoothing it or you might be doing edge detection. There are varieties of
reasons people have for doing things like moving
average processes. We're not making the claim that
you see moving average processes just by themselves in
nature all that often. It's a little bit hard to come up with
an example of a naturally occurring moving average process just
in its simple form like this. But the procedure of taking components and
weighting them and adding together is really very basic, very common,
and so it's important to study this. We'll also see a relationship
later between moving average and auto-regressive processes
that'll make this worthwhile. We can do some nice theoretical things
with moving average processes to make our lives easier. We should look at a picture. White noise process up on top, no real
structure to speak of, it's just noise. Now, down below, we've created a moving
average process, where we let Q = 3. I did a simple moving average. So, we're just taking our components,
adding them together, and dividing by the number of components. So, with Q = 3,
we're dividing 4(Q+1) components where we're just taking
an average of four components. You can see that we're losing some
of our higher frequencies and gaining some low frequencies. What we're doing is seeing
structure between neighbors. If your random variables
are close together, there is actually going to be
a dependency structure now. That's with Q = 3. I'm going to show you now and we hope that it'll just layover perfectly,
Q = 9. So, let me move back. There's Q = 3. When I go to Q = 9,
we induce still longer scale correlations, relationships between neighbors. We're smoothing even more, and
I guess this just makes sense. We're including nine numbers,
or ten actually, numbers in our average rather than nine. In this video, we've looked at some very
basic examples of stochastic processes and we've studied their stationarity. White noise is stationary,
perhaps trivially so. Random walks, even if there's zero mean,
are not stationary. The variance grows with time. And we started looking at moving averages. In the next lecture, we'll actually
explore the autocoveriance structure of the moving average process and
look at its stationarity.