57. Covariance Stationary Processes#
57.1. Overview#
In this lecture we study covariance stationary linear stochastic processes, a class of models routinely used to study economic and financial time series.
This class has the advantage of being
simple enough to be described by an elegant and comprehensive theory
relatively broad in terms of the kinds of dynamics it can represent
We consider these models in both the time and frequency domain.
57.1.1. ARMA Processes#
We will focus much of our attention on linear covariance stationary models with a finite number of parameters.
In particular, we will study stationary ARMA processes, which form a cornerstone of the standard theory of time series analysis.
Every ARMA processes can be represented in linear state space form.
However, ARMA have some important structure that makes it valuable to study them separately.
57.1.2. Spectral Analysis#
Analysis in the frequency domain is also called spectral analysis.
In essence, spectral analysis provides an alternative representation of the autocovariance function of a covariance stationary process.
Having a second representation of this important object
shines light on the dynamics of the process in question
allows for a simpler, more tractable representation in some important cases
The famous Fourier transform and its inverse are used to map between the two representations.
57.1.3. Other Reading#
For supplementary reading, see.
* [LS18], chapter 2
57.2. Introduction#
Consider a sequence of random variables
Thus,
As in other fields, successful economic modeling typically assumes the existence of features that are constant over time.
If these assumptions are correct, then each new observation
For this reason, we will focus in what follows on processes that are stationary — or become so after a transformation (see for example this lecture and this lecture).
57.2.1. Definitions#
A real-valued stochastic process
Its mean
does not depend on .For all
in , the -th autocovariance is finite and depends only on .
The function
Throughout this lecture, we will work exclusively with zero-mean (i.e.,
The zero-mean assumption costs nothing in terms of generality, since working with non-zero-mean processes involves no more than adding a constant.
57.2.2. Example 1: White Noise#
Perhaps the simplest class of covariance stationary processes is the white noise processes.
A process
for some
(Here
White noise processes play the role of building blocks for processes with more complicated dynamics.
57.2.3. Example 2: General Linear Processes#
From the simple building block provided by white noise, we can construct a very flexible family of covariance stationary processes — the general linear processes
where
is white noise is a square summable sequence in (that is, )
The sequence
Equation (57.1) is said to present a moving average process or a moving average representation.
With some manipulations it is possible to confirm that the autocovariance function for (57.1) is
By the Cauchy-Schwartz inequality one can show that
Evidently,
57.2.4. Wold’s Decomposition#
Remarkably, the class of general linear processes goes a long way towards describing the entire class of zero-mean covariance stationary processes.
In particular, Wold’s decomposition theorem states that every
zero-mean covariance stationary process
where
is white noise is square summable can be expressed as a linear function of and is perfectly predictable over arbitrarily long horizons
For intuition and further discussion, see [Sar87], p. 286.
57.2.5. AR and MA#
General linear processes are a very broad class of processes.
It often pays to specialize to those for which there exists a representation having only finitely many parameters.
(Experience and theory combine to indicate that models with a relatively small number of parameters typically perform better than larger models, especially for forecasting)
One very simple example of such a model is the first-order autoregressive or AR(1) process
By direct substitution, it is easy to verify that
Hence
Applying (57.2) to the previous expression for
The next figure plots an example of this function for
using LaTeXStrings, Plots
using LinearAlgebra, Statistics
plt_1 = plot()
plt_2 = plot()
plots = [plt_1, plt_2]
for (i, phi) in enumerate((0.8, -0.8))
times = 0:16
acov = [phi .^ k ./ (1 - phi .^ 2) for k in times]
label = L"autocovariance, $\phi = %$phi$"
plot!(plots[i], times, acov, color = :blue, lw = 2, marker = :circle,
markersize = 3,
alpha = 0.6, label = label)
plot!(plots[i], legend = :topright, xlabel = "time", xlim = (0, 15))
plot!(plots[i], seriestype = :hline, [0], linestyle = :dash, alpha = 0.5,
lw = 2, label = "")
end
plot(plots[1], plots[2], layout = (2, 1), size = (700, 500))
Another very simple process is the MA(1) process (here MA means “moving average”)
You will be able to verify that
The AR(1) can be generalized to an AR(
Putting all of this together, we get the
57.2.6. ARMA Processes#
A stochastic process
where
An alternative notation for ARMA processes uses the lag operator
Def. Given arbitrary variable
It turns out that
lag operators facilitate succinct representations for linear stochastic processes
algebraic manipulations that treat the lag operator as an ordinary scalar are legitimate
Using
If we let
then (57.6) becomes
In what follows we always assume that the roots of the polynomial
This condition is sufficient to guarantee that the ARMA(
In fact it implies that the process falls within the class of general linear processes described above.
That is, given an ARMA(
The sequence
The function
57.3. Spectral Analysis#
Autocovariance functions provide a great deal of information about covariance stationary processes.
In fact, for zero-mean Gaussian processes, the autocovariance function characterizes the entire joint distribution.
Even for non-Gaussian processes, it provides a significant amount of information.
It turns out that there is an alternative representation of the autocovariance function of a covariance stationary process, called the spectral density.
At times, the spectral density is easier to derive, easier to manipulate, and provides additional intuition.
57.3.1. Complex Numbers#
Before discussing the spectral density, we invite you to recall the main properties of complex numbers (or skip to the next section).
It can be helpful to remember that, in a formal sense, complex numbers are just points
When
The modulus or absolute value of a complex number
The product of two complex numbers
When endowed with these notions of multiplication and addition, the set of complex numbers forms a field — addition and multiplication play well together, just as they do in
The complex number
The
Converted back to our first notation, this becomes
Complex numbers can be represented in the polar form
where
57.3.2. Spectral Densities#
Let
The spectral density
(Some authors normalize the expression on the right by constants such as
Using the fact that
It is not difficult to confirm that
real-valued
even (
), and -periodic, in the sense that for all
It follows that the values of
For this reason it is standard to plot the spectral density only on the interval
57.3.3. Example 1: White Noise#
Consider a white noise process
It is easy to check that in this case
As we will see, this can be interpreted as meaning that “all frequencies are equally present”.
(White light has this property when frequency refers to the visible spectrum, a connection that provides the origins of the term “white noise”)
57.3.4. Example 2: AR and MA and ARMA#
It is an exercise to show that the MA(1) process
With a bit more effort, it is possible to show (see, e.g., p. 261 of [Sar87]) that the spectral density of the AR(1) process
More generally, it can be shown that the spectral density of the ARMA process (57.5) is
where
is the standard deviation of the white noise processthe polynomials
and are as defined in (57.7)
The derivation of (57.12) uses the fact that convolutions become products under Fourier transformations.
The proof is elegant and can be found in many places — see, for example, [Sar87], chapter 11, section 4.
It is a nice exercise to verify that (57.10) and (57.11) are indeed special cases of (57.12).
57.3.5. Interpreting the Spectral Density#
Plotting (57.11) reveals the shape of the spectral density for the AR(1) model when
ar1_sd(phi, omega) = 1 ./ (1 .- 2 * phi * cos.(omega) .+ phi .^ 2)
omega_s = range(0, pi, length = 180)
plt_1 = plot()
plt_2 = plot()
plots = [plt_1, plt_2]
for (i, phi) in enumerate((0.8, -0.8))
sd = ar1_sd(phi, omega_s)
label = L"spectral density, $\phi = %$phi$"
plot!(plots[i], omega_s, sd, color = :blue, alpha = 0.6, lw = 2,
label = label)
plot!(plots[i], legend = :top, xlabel = "frequency", xlim = (0, pi))
end
plot(plots[1], plots[2], layout = (2, 1), size = (700, 500))
These spectral densities correspond to the autocovariance functions for the AR(1) process shown above.
Informally, we think of the spectral density as being large at those
To see the idea, let’s consider why, in the lower panel of the preceding figure, the spectral density for the case
Recall that the spectral density can be expressed as
When we evaluate this at
Hence the product is always large and positive, and hence the sum of the products on the right-hand side of (57.13) is large.
These ideas are illustrated in the next figure, which has
phi = -0.8
times = 0:16
y1 = [phi .^ k ./ (1 - phi .^ 2) for k in times]
y2 = [cos.(pi * k) for k in times]
y3 = [a * b for (a, b) in zip(y1, y2)]
# Autocovariance when phi = -0.8
plt_1 = plot(times, y1, color = :blue, lw = 2, marker = :circle, markersize = 3,
alpha = 0.6, label = L"\gamma(k)")
plot!(plt_1, seriestype = :hline, [0], linestyle = :dash, alpha = 0.5,
lw = 2, label = "")
plot!(plt_1, legend = :topright, xlim = (0, 15), yticks = [-2, 0, 2])
# Cycles at frequence pi
plt_2 = plot(times, y2, color = :blue, lw = 2, marker = :circle, markersize = 3,
alpha = 0.6, label = L"cos(\pi k)")
plot!(plt_2, seriestype = :hline, [0], linestyle = :dash, alpha = 0.5,
lw = 2, label = "")
plot!(plt_2, legend = :topright, xlim = (0, 15), yticks = [-1, 0, 1])
# Product
plt_3 = plot(times, y3, seriestype = :sticks, marker = :circle, markersize = 3,
lw = 2, label = L"\gamma(k) cos(\pi k)")
plot!(plt_3, seriestype = :hline, [0], linestyle = :dash, alpha = 0.5,
lw = 2, label = "")
plot!(plt_3, legend = :topright, xlim = (0, 15), ylim = (-3, 3),
yticks = [-1, 0, 1, 2, 3])
plot(plt_1, plt_2, plt_3, layout = (3, 1), size = (800, 600))
On the other hand, if we evaluate
phi = -0.8
times = 0:16
y1 = [phi .^ k ./ (1 - phi .^ 2) for k in times]
y2 = [cos.(pi * k / 3) for k in times]
y3 = [a * b for (a, b) in zip(y1, y2)]
# Autocovariance when phi = -0.8
plt_1 = plot(times, y1, color = :blue, lw = 2, marker = :circle, markersize = 3,
alpha = 0.6, label = L"\gamma(k)")
plot!(plt_1, seriestype = :hline, [0], linestyle = :dash, alpha = 0.5,
lw = 2, label = "")
plot!(plt_1, legend = :topright, xlim = (0, 15), yticks = [-2, 0, 2])
# Cycles at frequence pi
plt_2 = plot(times, y2, color = :blue, lw = 2, marker = :circle, markersize = 3,
alpha = 0.6, label = L"cos(\pi k/3)")
plot!(plt_2, seriestype = :hline, [0], linestyle = :dash, alpha = 0.5,
lw = 2, label = "")
plot!(plt_2, legend = :topright, xlim = (0, 15), yticks = [-1, 0, 1])
# Product
plt_3 = plot(times, y3, seriestype = :sticks, marker = :circle, markersize = 3,
lw = 2, label = L"\gamma(k) cos(\pi k/3)")
plot!(plt_3, seriestype = :hline, [0], linestyle = :dash, alpha = 0.5,
lw = 2, label = "")
plot!(plt_3, legend = :topright, xlim = (0, 15), ylim = (-3, 3),
yticks = [-1, 0, 1, 2, 3])
plot(plt_1, plt_2, plt_3, layout = (3, 1), size = (600, 600))
In summary, the spectral density is large at frequencies
57.3.6. Inverting the Transformation#
We have just seen that the spectral density is useful in the sense that it provides a frequency-based perspective on the autocovariance structure of a covariance stationary process.
Another reason that the spectral density is useful is that it can be “inverted” to recover the autocovariance function via the inverse Fourier transform.
In particular, for all
This is convenient in situations where the spectral density is easier to calculate and manipulate than the autocovariance function.
(For example, the expression (57.12) for the ARMA spectral density is much easier to work with than the expression for the ARMA autocovariance)
57.3.7. Mathematical Theory#
This section is loosely based on [Sar87], p. 249-253, and included for those who
would like a bit more insight into spectral densities
and have at least some background in Hilbert space theory
Others should feel free to skip to the next section — none of this material is necessary to progress to computation.
Recall that every separable Hilbert space
The nice thing about such a basis is that every
where
Thus,
The scalar sequence
In other words,
Consider an operator
The Fourier coefficients of
Using elementary results from Hilbert space theory, it can be shown that
is one-to-one — if and are distinct in , then so are their expansions in is onto — if then its preimage in is the sequence given by is a linear isometry — in particular
Summarizing these results, we say that any separable Hilbert space is isometrically isomorphic to
In essence, this says that each separable Hilbert space we consider is just a different way of looking at the fundamental space
With this in mind, let’s specialize to a setting where
is the autocovariance function of a covariance stationary process, and is the spectral density , where is the set of square summable functions on the interval , with inner product the orthonormal basis for given by the set of trigonometric functions
Using the definition of
In other words, apart from a scalar multiple, the spectral density is just an transformation of
In particular, it is an expansion of the autocovariance function with respect to the trigonometric basis functions in
As discussed above, the Fourier coefficients of
Transforming this inner product into its integral expression and using (57.16) gives (57.14), justifying our earlier expression for the inverse transform.
57.4. Implementation#
Most code for working with covariance stationary models deals with ARMA models.
Julia code for studying ARMA models can be found in the DSP.jl
package.
Since this code doesn’t quite cover our needs — particularly vis-a-vis spectral analysis — we’ve put together the module arma.jl, which is part of QuantEcon.jl package.
The module provides functions for mapping ARMA(
impulse response function
simulated time series
autocovariance function
spectral density
57.4.1. Application#
Let’s use this code to replicate the plots on pages 68–69 of [LS18].
Here are some functions to generate the plots
using QuantEcon, Random
# plot functions
function plot_spectral_density(arma, plt)
(w, spect) = spectral_density(arma, two_pi = false)
plot!(plt, w, spect, lw = 2, alpha = 0.7, label = "")
plot!(plt, title = "Spectral density", xlim = (0, pi),
xlabel = "frequency", ylabel = "spectrum", yscale = :log)
return plt
end
function plot_spectral_density(arma)
plt = plot()
plot_spectral_density(arma, plt = plt)
return plt
end
function plot_autocovariance(arma, plt)
acov = autocovariance(arma)
n = length(acov)
plot!(plt, 0:(n - 1), acov, seriestype = :sticks, marker = :circle,
markersize = 2, label = "")
plot!(plt, seriestype = :hline, [0], color = :red, label = "")
plot!(plt, title = "Autocovariance", xlim = (-0.5, n - 0.5),
xlabel = "time", ylabel = "autocovariance")
return plt
end
function plot_autocovariance(arma)
plt = plot()
plot_spectral_density(arma, plt = plt)
return plt
end
function plot_impulse_response(arma, plt)
psi = impulse_response(arma)
n = length(psi)
plot!(plt, 0:(n - 1), psi, seriestype = :sticks, marker = :circle,
markersize = 2, label = "")
plot!(plt, seriestype = :hline, [0], color = :red, label = "")
plot!(plt, title = "Impluse response", xlim = (-0.5, n - 0.5),
xlabel = "time", ylabel = "response")
return plt
end
function plot_impulse_response(arma)
plt = plot()
plot_spectral_density(arma, plt = plt)
return plt
end
function plot_simulation(arma, plt)
X = simulation(arma)
n = length(X)
plot!(plt, 0:(n - 1), X, lw = 2, alpha = 0.7, label = "")
plot!(plt, title = "Sample path", xlim = (0, 0, n), xlabel = "time",
ylabel = "state space")
return plt
end
function plot_simulation(arma)
plt = plot()
plot_spectral_density(arma, plt = plt)
return plt
end
function quad_plot(arma)
plt_1 = plot()
plt_2 = plot()
plt_3 = plot()
plt_4 = plot()
plots = [plt_1, plt_2, plt_3, plt_4]
plot_functions = [plot_spectral_density,
plot_impulse_response,
plot_autocovariance,
plot_simulation]
for (i, plt, plot_func) in zip(1:1:4, plots, plot_functions)
plots[i] = plot_func(arma, plt)
end
return plot(plots[1], plots[2], plots[3], plots[4], layout = (2, 2),
size = (800, 800))
end
quad_plot (generic function with 1 method)
Now let’s call these functions to generate the plots.
We’ll use the model
Random.seed!(42) # For reproducible results.
phi = 0.5;
theta = [0, -0.8];
arma = ARMA(phi, theta, 1.0)
quad_plot(arma)
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
┌ Warning: Invalid limits for x axis. Limits should be a symbol, or a two-element tuple or vector of numbers.
│ xlims = (0, 0, 90)
└ @ Plots ~/.julia/packages/Plots/Ec1L1/src/axes.jl:642
57.4.2. Explanation#
The call
arma = ARMA(phi, theta, sigma)
creates an instance arma
that represents the ARMA(
If phi
and theta
are arrays or sequences, then the interpretation will
be
phi
holds the vector of parameterstheta
holds the vector of parameters
The parameter sigma
is always a scalar, the standard deviation of the white noise.
We also permit phi
and theta
to be scalars, in which case the model will be interpreted as
The two numerical packages most useful for working with ARMA models are DSP.jl
and the fft
routine in Julia.
57.4.3. Computing the Autocovariance Function#
As discussed above, for ARMA processes the spectral density has a simple representation that is relatively easy to calculate.
Given this fact, the easiest way to obtain the autocovariance function is to recover it from the spectral density via the inverse Fourier transform.
Here we use Julia’s Fourier transform routine fft, which wraps a standard C-based package called FFTW.
A look at the fft documentation shows that the inverse transform ifft takes a given sequence
Thus, if we set
For
(You can check the last equality)
In view of (57.14) we have now shown that, for