Soci709 (formerly 209) Module 14  AUTOCORRELATION IN TIME SERIES DATA
Resources:
ALSM5e pp 481498; ALSM4e pp 497516
Hamilton 2006 pp. 339360 (especially commands tsset date and prais y
x1 x2)
1. AUTOCORRELATION OF ERRORS: NATURE OF
THE PROBLEM
Time series data are observed on the same unit
(individual, country, firm, etc.) at n points in time. EX:

yearly divorce rate in the U.S. from 1922 to present

quarterly profits of a company

income inequality in the U.S. measured each year
from 1964 to present

daily atmospheric pollen count in Chapel Hill
since first recorded, etc.
In regression models using time series data the
errors are often correlated over time (they are said to be autocorrelated
or serially correlated).
NKNW illustrate the problems caused by correlated
errors with simulated data generated with the model:

Y_{t} = b_{0}
+ b_{1}X_{t}
+ e_{t}

e_{t} = e_{t1}
+ u_{t}

X_{t} represents "time", so that X_{1}=1,
X_{2}=2, etc.

b_{0} = 2;
b_{1}
= .5

e_{0} (e_{t}
prior to beginning the process) = 3
The simulated data are shown in the next exhibit.
As seen in the next exhibit, the errors e_{t}
are positively correlated.
Because of serial correlation

the OLS and true regression lines may differ sharply
from sample to sample depending on the initial disturbance e_{0}
(compare (a), (b) and (c) in next exhibit)

MSE may underestimate true variance of e_{t}
(compare variability of residuals around regression line in (a) and (b)
in next exhibit); thus standard errors of estimate of the regression coefficients
may also be underestimated
These patterns can be seen in the next exhibit
In general, serial correlation of the disturbances
may have the following effects with OLS estimation

estimated regression coefficients are still
unbiased but no longer minimum variance (= inefficient)

MSE (the OLS estimate of s^{2})
may underestimate the true variance of errors

s{b_{k}} may underestimate true standard
error of estimate

thus, statistical inference using t and F is no
longer justified
2. AUTOCORRELATION DIAGNOSTICS
1. Plot of Residuals Against Time or Sequential
Order
An informal diagnostic of autocorrelation of errors
is to plot the residuals from the OLS regression against time or against
the sequential order of the observation in the file (after checking that
observations are in fact arranged in chronological order!). Connecting
the points with a dotted line makes any pattern of autocorrelation more
conspicuous. Look for evidence of "tracking", in which residuals
corresponding to adjacent time points have similar values. (Some
people say to look for a pattern like that made by bullets fired from a
machine gun.)
Exhibit: Index plot
(= time plot) of residuals for Blaisdell data
2. The WaldWolfowitz Runs Test
The WaldWolfowitz runs test is a nonparametric
test that detects serial patterns in a run of numbers. Applied to
the residuals of the OLS regression, a significant test indicates the presence
of sequences of positive or negative residuals longer than expected by
chance alone. Such long sequences of residuals above or below zero
is what one would expect if the errors are "tracking" because of autocorrelation.
For the Blaisdell data the test is significant
(p=.006) so one concludes that the errors are correlated.
3. The DurbinWatson Test
The DurbinWatson (DW) test is the most commonly
used test of autocorrelation of residuals.
The DW D statistic is calculated from the
ordinary OLS residuals e_{t} = Y_{t}  ^Y_{t} as
D = S_{t=2
to n} (e_{t}  e_{t1})^{2} / S_{t=1
to n} e_{t}^{2}
where n is the number of cases.
To understand the DW formula consider that

when e_{t} and e_{t1} are correlated
they have similar values

thus when e_{t} and e_{t1} are
correlated the terms (e_{t}  e_{t1})^{2} are
small and the numerator of D is small (while the denominator is the same
no matter how much autocorrelation there is)

thus small values of D (close to zero) indicate
serial correlation
The DW test setup is
H_{0}: r
= 0
H_{1}: r
> 0
Table B7 gives critical values d_{L} and
d_{U} such that
if D > d_{U} conclude H_{0}
(r = 0)
if d_{L} <= D <= d_{U}
the test is inconclusive
if D < d_{L} conclude H_{1}
(r > 0)
Example: SYSTAT routinely reports the DW
D statistic with every regression (D has no meaning unless observations
are sequentially ordered). For the Blaisdell data D = .735.
Table B7 for n=20 and p1=1 gives d_{L}=.95 and d_{U}=1.15.
Since .735 < .95 = d_{L} one concludes H_{1} (errors
are autocorrelated).
3. REMEDIAL MEASURES FOR AUTOCORRELATION
1. Add Omitted Predictors to Model
Autocorrelation is caused by unmeasured variables
that have similar values from period to period. Identifying these
variables and including them in the model may eliminate the serial correlation.
Some of these substantive omitted variables may be "simulated" by adding
to the model

a linear or exponential trend

seasonal indicators
If adding a trend or seasonal indicators gets
rid of the autocorrelation, this is by far the best solution to the problem.
2. The FirstOrder Autoregressive Error
Model With Generalized Least Squares Estimation
1. FirstOrder Autoregressive Error Model
The model is
Y_{t} = b_{0}
+ b_{1}X_{t1}
+ b_{2}X_{t2}
+ ... + b_{p1}X_{t,p1}
+ e_{t}
e_{t} =
re_{t1}
+ u_{t}
where
r
< 1 (r
is Greek "rho" and denotes the autocorrelation parameter)
u_{t} is i.i.d. ~ N(0, s^{2})
One can show the following consequences of model
assumptions (see ALSM4e pp. 501502; try to express these relationships in
words):

E{e_{t}}
= 0

s^{2}{e_{t}}
= s^{2}/(1r^{2})

s{e_{t},
e_{t1}}
= r(s^{2}/(1r^{2}))

r{e_{t},
e_{t1}}
= s{e_{t},
e_{t1}}/(s{e_{t}}s{e_{t1}})
= r

r{e_{t},
e_{ts}}
= r^{s}
Thus the variancecovariance matrix of e
is nondiagonal with a specific structure; s^{2}{e}
=
k 
kr 
kr^{2} 
... 
kr^{n1} 
kr 
k 
kr 
... 
kr^{n2} 
... 
... 
... 
... 
... 
kr^{n1} 
kr^{n2} 
kr^{n3} 
... 
k 
where
k = s^{2}/(1r^{2})
( k is Greek
"kappa")
(This is why the model is called "generalized",
as in "generalized least squares"; see Module 12.)
Even though the firstorder autoregressive
model is simple, it is often a good approximation of actual situations.
2. Generalized Least Squares Estimation
Using Transformed Variables
Assume (for the sake of argument) that one knows
the value of r.
Define the transformed variables
Y_{t}' = Y_{t}  rY_{t1}
X_{tk}' = X_{tk}  rX_{t1,k}
Then one can show that the regression
Y_{t}' = b_{0}'
+ ... + b_{k}'X_{tk}'
+ ... + u_{t}
based on the transformed variables has error term
u_{t} which is no longer serially correlated, and that b_{k}
= b_{k}'
except that b_{0}'
= b_{0}(1r)
(see NKNW pp. 508509). Thus if one knows r
one can get rid of the serial correlation by using OLS with the transformed
data.
(This transformation can be derived from the
application of GLS estimation to the nondiagonal variancecovariance matrix
of e generated
by the autocorrelation. So the transformation is a special case of
GLS estimation.)
In practice the value of r
is unknown. The 3 classical methods of estimation in the presence
of autocorrelation discussed next (CochraneOrcutt, HildrethLu, first
differences) are all based on transforming the variables, using alternative
ways of estimating r.
3. CochraneOrcutt Procedure
The Cochraneorcutt procedure is

do an OLS regression of Y_{t} on the X_{tk}
and calculate the residuals e_{t}

estimate the autocorrelation r
as
r = S_{t=2
to n} e_{t1}e_{t} / S_{t=2
to n} e_{t1}^{2}

use r to transform the variables into Y_{t}'
and X_{tk}' using formula above; do an OLS regression of Y_{t}'
on the X_{tk}'

if the DW test still indicates serial correlation,
reestimate r using residuals computed using the original variables Y_{t}
and X_{tk} and the regression coefficients estimated from the (last)
transformed regression; go to 3.
The following exhibits show the CochraneOrcutt
procedure with the Blaisdell data.
4. HildrethLu Procedure
The HildrethLu procedure searches for the estimate
of r that minimizes
the sum of squared errors of the transformed regression, i.e.
SSE = S(Y_{t}'
 ^Y_{t}')^{2}
(HilderthLu is similar to the BoxCox procedure
to estimate the parameter l
of a power transformation of Y.)
One can search for the optimal r
by calculating the transformed regression for closely spaced values of
r
and choosing the one with smallest SSE, as shown in NKNW.
One can also estimate r
and the regression coefficients simultaneously using iterative methods
(nonlinear least squares). This can be done using the NONLIN module
of SYSTAT, as shown in the exhibit analyzing the Blaisdell data.
Exhibit: (REPEAT) Replication
of CochraneOrcutt, HildrethLu, & first differences procedures for
Blaisdell data
5. First Differences Procedure
First differences is the simplest transformation
procedure as it implicitly assumes r
= 1. This assumption is often approximately justified because

estimates of r
are often close to 1

the relationship of SSE with r
is often "flat" (as seen in the HildrethLu procedure: see ALSM4e Table 12.5
p. 513) for values of r
near the optimum, so the estimate of r
does not need to be exact
The first differences transformation is thus
Y_{t}' = Y_{t}  Y_{t1}
X_{tk}' = X_{tk}  X_{t1,k}
The first differences procedure involves two
regressions with the transformed data:

a first regression without a constant term
to estimate the regression coefficients (since the first differences transformation
"wipes out" the constant term)

a second regression with a constant term
to recalculate the DW D statistic only (because the DW formula requires
a constant in the model)
6. Comparison of the 3 Transformation Methods
Results of the 3 transformation methods (compared
with OLS) are shown in the following table.
Regression results for 4 estimation methods
(SYSTAT)  Blaisdell data (compare with ALSM5e <>, ALSM4e Table 12.7 p. 516  some figures differ slightly)

b_{1} 
s{b_{1}} 
tratio 
r 
MSE 
CochraneOrcutt 
.1738 
.0029 
59.42 
.626 
.004515 
HildrethLu (nonlinear LS) 
.1605 
.0079 
20.24 
.959 
.004479 
First differences 
.1685 
.0051 
33.06 
1.0 
.004815 
OLS 
.1763 
.0014 
122.0 
0.0 
.007406 
7. STATA Commands
4. COMPREHENSIVE EXAMPLE: U.S. DIVORCE RATE
19201970, 19201997
1. SYSTAT Analysis
The following exhibit present examples of the
CochraneOrcutt, HildrethLu (using nonlinear least squares), and first
differences methods applied to an analysis of the divorce rate in the U.S.
from 1920 to 1970.
As a substantive epilogue the next 3 exhibits relate to a model of the
divorce rate that is more elaborate than one previously shown, as it includes
a measure of the birth rate (women 1544) and military personnel per 1,000
population. Only OLS results are shown.
2. STATA Analysis
To be added later.
5. SUMMARY & RECOMMENDATIONS
Analysis of the Blaisdell data and the divorce rates data illustrate the
following approach:
 set up your data as a timeseries (i.e., identify variable that
represents time, or the sequential order of observations) is required by
your software (e.g., tsset in STATA)
 do an OLS regression and test for autocorrelation of residuals (using
WaldWolfowitz run test and DurbinWatson test)
 if autocorrelation is significant consider adding to the models
variables that might contribute to the autocorrelation
 do a PraisWinston regression, twostep and iterated
 if this solution fails investigate more complicated error structures
(beyond this module)
Last modified 24 Apr 2006