So at this point, you now understand how convolution is applied to images?
Well, we could use this approach for Time series.
The next step is to look more closely at one's convolution, which handles Time series directly.
So this lecture will look at one de convolution specifically for use on Time series.
Now we discuss the mathematical operation in the previous lectures, so we don't really need to discuss
that again.
But for completion sake, let's just review what Convolution is doing.
So suppose we have some multivariate input time, series of shape, TBD, if you had multiple samples,
this would be invited, but let's ignore that extra dimension for now.
Now, let's also suppose that we've chosen to have more output features and we are doing convolution
and same mode, thus our output time series, whatever the shape T by M.
So if we choose a kind of length of K than our kernel or filter, whatever the shape K by M and the
actual convolution would be done using the same multiplying sum that you've seen many times.
Now, the point of this lecture is not to do more math, but instead to more deeply understand the math
we've already seen.
So one of the first examples I use to explain convolution is the Gaussian blur.
I think it's pretty obvious what kind of effect this has when we use it on images.
But clearly this can also be applied to Time series.
So if we can evolve a Gaussian with a time series, what we get is a blurred version of that Time series.
Of course, when we talk about Time series, we don't call it blurring, but we call it smoothing.
So this is another way to implement something you've seen many times before.
As you recall, the simple moving average and exponential smoothing work in a similar way.
So you've seen that in learning what we call convolution is really just what everyone else calls cross
correlation.
So if you ever talk to a statistician or an engineer, they will think you are doing convolution backwards.
That's why another way to think of convolutional neural networks is that they are cross correlation
neural networks.
Well, let's recall that this is not the first time that we've seen a correlation in this course, auto
correlation plays a huge role in Arima.
We use the auto correlation to help us determine the Arima orders.
So at this point, we can kind of combine our knowledge of deep learning in Arima to more deeply understand
what correlation is doing.
As mentioned earlier, a convolutional filter in deep learning is like a pattern finder.
By doing convolution with a filter, you will get a spike whenever the filter matches the pattern in
the signal, whether that's an image or a time series.
So what is autocorrelation doing?
Well, it's going to give us a spike whenever the time series matches itself, specifically a lag version
of itself.
In other words, it's pattern matching different parts of the time series with each other and that can
tell you which parts data points are useful for making future predictions.
And that's because if they match, then they are predictive.
Another thing you can do is cross correlation with different time series, if you see a spike at a lag
other than zero, then that means one of these times series is predictive of the other.
Now, in my view, the most interesting way to relate convolution to Arima is this let's write down
the equation for an ERP process.
At this point, I shouldn't have to say anything else.
After watching the previous lectures, you should immediately recognize what this is.
In fact, this is nothing but convolution.
And actually this is real convolution, since we have minus K and not plus K.
That is to say, an AP model is like a mini CNN.
It's a CNN with only one layer and identity activation.
In other words, it's linear nonetheless.
It still is a CNN.
So that to me is a very interesting connection to make.
You can think of auto regressive cnes like an extension of IRP or you can think of IRP as a special
case of CNN's.
And again, note that there is so much theory behind Arima that we just don't have time for in this
course and that most students wouldn't be too keen on learning.
But to give you some idea, once you realize that this is just convolution, there is a lot of analysis
that you can do, especially involving the frequency domain and for what it transforms.