Module 1: Memory, Noise, and Why Time Order Matters

Econ 6376 — Applied Time Series Econometrics

How to Use These Notes

This chapter has one main reading path and three optional layers.

• Core material is the main text. It contains the concepts, notation, interpretations, and applied skills expected of everyone.
• Deeper Dive sections explain why a result works or develop it more fully. They are useful, but they can be skipped on a first reading.
• Technical Note sections state qualifications or formal details that matter for precise reasoning.
• Looking Ahead sections introduce an idea that will be taught formally in a later module.

If you missed lecture, read the main text, run the Core code, and complete the Core Practice problems. Then return to the optional sections that address your questions or interests.

Core Learning Objectives

By the end of this module, you should be able to:

1. Distinguish time-series data from cross-sectional and panel data and explain why time ordering matters.
2. Use the master equation to locate the AR(1) model within the broader course.
3. Distinguish an innovation, \(\epsilon_t\), from a fitted residual, \(e_t\).
4. Explain “memory” in an AR(1) process and connect it to the autoregressive parameter \(\phi\).
5. Build and interpret an AR(1) simulator in R.
6. State the three conditions for weak stationarity.
7. Define and interpret autocovariance, autocorrelation, and a correlogram.
8. Explain why a random walk is nonstationary.
9. Use simulation to demonstrate the danger of spurious regression.

---

1.1 What Is Time-Series Data?

Three Common Data Structures

Most applied economic data have a cross-sectional dimension, a time dimension, or both.

Cross-sectional data

\[ \{y_i,\mathbf{x}_i\}, \qquad i=1,\ldots,N, \]

where \(y_i\) is the outcome for unit \(i\) and \(\mathbf{x}_i\) is a \(K\times 1\) vector of features.

• The index identifies units such as households, firms, counties, or countries.
• The ordering of rows is usually not substantively meaningful.
• Introductory econometrics commonly begins with an iid sampling benchmark.
• A typical example is a survey of household income and education in one year.

Time-series data

\[ \{y_t,\mathbf{x}_t\}, \qquad t=1,\ldots,T, \]

where the index records when an observation occurred.

• Ordering is meaningful. Shuffling the rows destroys temporal information.
• Past values such as \(y_{t-1}\) and \(y_{t-2}\) may help us understand or predict \(y_t\).
• Dependence across periods is common and often substantively important.
• A typical example is the monthly U.S. unemployment rate.

Time-series data do not have to be dependent. An iid white-noise sequence is a valid time series. The important point is that we cannot treat periods as interchangeable without first considering whether time order contains information.

Panel data

\[ \{y_{it},\mathbf{x}_{it}\}, \qquad i=1,\ldots,N,\quad t=1,\ldots,T. \]

Panel data combine unit and time dimensions. They may consist of the same units followed over time, repeated samples of different units, or other mixed structures. Panel methods must account for both dimensions. They are not the focus of this course.

Why the Change of Index Matters

Moving from \(i\) to \(t\) is not merely changing a subscript. Suppose unemployment is 5.2 percent this month. It would be surprising if it were 1 percent next month and 14 percent the month after that. Economic states tend to persist.

That persistence changes the questions we ask:

• How informative is the past about the present?
• How quickly does the influence of a shock fade?
• Are the probabilistic rules stable over time?
• How should uncertainty be measured when observations are dependent?
• Can a pattern that looks meaningful be produced by persistence alone?

This course organizes those questions around one plain-language idea:

Core idea: How much does the past matter?

We will measure temporal dependence with tools such as the autocorrelation function, represent it with models such as ARMA and VAR systems, and use it for forecasting.

Technical Note — Independence, identification, and inference

The iid assumption is a useful introductory benchmark, not a universal identifying assumption for regression. Cross-sectional observations can be dependent, and valid methods exist for clustered or otherwise dependent data.

It is also important to distinguish dependence in the observed outcome from serial correlation in a regression error. A persistent outcome need not imply that the errors from every model of that outcome are serially correlated. Conversely, if a fitted time-series regression leaves serial correlation in its residuals, conventional iid standard errors are generally not appropriate.

For this course, the practical lesson is simple: with time-indexed data, dependence must be investigated rather than assumed away.

Deeper Dive — The effective sample-size problem

Standard time-series arguments often study what happens as \(T\rightarrow\infty\). Economic history, however, gives us only one observation per period and many important series span decades rather than centuries.

A sample of 80 quarterly observations represents 20 years of data. That may sound historically substantial, but it can still be small for distinguishing a highly persistent stationary process from a unit root or for estimating a model with many parameters. Consequently, asymptotic approximations can perform poorly in realistic samples.

More frequent measurement does not automatically create more independent information. Daily observations may be more numerous than monthly observations, but they can also be more strongly related to one another. Throughout the course, sample size must be interpreted together with dependence.

---

1.2 The Master Equation

The course’s central organizing equation is:

\[ y_t = \alpha+\delta t +\sum_{j=1}^{p}\phi_j y_{t-j} +\sum_{\ell=1}^{q}\theta_\ell\epsilon_{t-\ell} +\epsilon_t. \]

Think of this equation as a mixing board. Each term is a channel that can be switched on or off.

Component	Role	Where it returns
\(\alpha\)	Intercept	Available throughout
\(\delta t\)	Deterministic time trend	Trend stationarity and unit-root testing
\(\phi_j y_{t-j}\)	Memory of past outcomes	AR and ARMA models
\(\theta_\ell\epsilon_{t-\ell}\)	Memory of past shocks	MA and ARMA models
\(\epsilon_t\)	New innovation at time \(t\)	Every stochastic model

Later in the course, dynamic regression extends this spine by adding covariates such as \(\beta x_t\) and their lags. Keeping that extension separate for now lets the master equation foreground the two kinds of memory developed first: memory of outcomes and memory of shocks.

The equation is a roadmap, not something you need to estimate in full today. In Module 1 we turn on one autoregressive term:

\[ y_t=\alpha+\phi y_{t-1}+\epsilon_t. \]

The DGP and the Fitted Model

We use notation to distinguish the process that generated the data from the model estimated using one observed sample.

Data-generating process

\[ y_t=\alpha+\beta x_t +\sum_{j=1}^{p}\phi_jy_{t-j}+\epsilon_t. \]

The parameters are unhatted because they describe the unknown DGP.

Fitted equation

\[ y_t=\widehat{\alpha}+\widehat{\beta}x_t +\sum_{j=1}^{p}\widehat{\phi}_j y_{t-j}+e_t, \]

where

\[ e_t=y_t-\widehat{y}_t. \]

The distinction between \(\epsilon_t\) and \(e_t\) is foundational:

• \(\epsilon_t\) is an innovation in the DGP. Under the model, it is the new shock at time \(t\) relative to the information already available.
• \(e_t\) is a residual produced by a fitted model. It is observed after estimation.
• A well-specified model should leave residuals whose behavior is compatible with the innovations assumed by that model.
• A residual is not literally the innovation. It is a sample-dependent approximation whose quality depends on the model and estimated parameters.

Core checkpoint: Unhatted parameters and \(\epsilon_t\) describe the DGP. Hatted parameters and \(e_t\) describe the fitted model.

Looking Ahead — Residual diagnostics

In Module 5, we will ask whether fitted residuals retain predictable structure. If a model is intended to capture the conditional mean, remaining residual autocorrelation is evidence that the mean specification is incomplete.

Constant variance and normality are separate modeling choices. Weak stationarity does not, by itself, guarantee constant conditional variance, and normality is not required for every estimation or forecasting task.

---

1.3 Memory and the AR(1) Process

The AR(1) as the “Hello World” Model

The first-order autoregressive process is

\[ y_t=\alpha+\phi y_{t-1}+\epsilon_t, \qquad \epsilon_t\sim WN(0,\sigma^2). \]

For the simulations below, we impose the stronger assumption

\[ \epsilon_t\overset{\mathrm{iid}}{\sim}N(0,\sigma^2). \]

Each component has a distinct role:

• \(\alpha\) is the intercept. When the process is stationary, \(\alpha\) and \(\phi\) jointly determine its long-run mean.
• \(\phi\) is the autoregressive or memory parameter.
• \(y_{t-1}\) is the previous observation.
• \(\epsilon_t\) is the new innovation.

The coefficient \(\phi\) determines how the past enters:

Value of \(\phi\)	Behavior
\(\phi=0\)	No linear memory. With iid innovations, observations are iid around the mean \(\alpha\).
\(0<\phi<1\)	Positive persistence. Shocks fade gradually without changing sign.
\(-1<\phi<0\)	Alternating persistence. Shock effects change sign as they fade.
\(\phi=1\)	Unit root. Shock effects do not decay.
\(\phi=-1\)	A nondecaying effect that alternates sign.
\(|\phi|>1\)	Explosive behavior.

The stationary AR(1) case is \(|\phi|<1\). The closer \(|\phi|\) is to one, the more persistent the process.

White Noise

A weak white-noise sequence \(\{\epsilon_t\}\) satisfies

\[ \mathbb{E}[\epsilon_t]=0,\qquad \operatorname{Var}(\epsilon_t)=\sigma^2<\infty,\qquad \operatorname{Cov}(\epsilon_t,\epsilon_s)=0\quad(t\neq s). \]

White noise is the basic “new information” component from which we construct the models in this course.

Technical Note — Uncorrelated is not the same as independent

General white noise need only be serially uncorrelated. It need not be independent or normally distributed.

• Weak white noise: mean zero, constant finite unconditional variance, and zero autocovariance at nonzero lags.
• Independent white noise: adds independence across dates.
• Gaussian white noise: is jointly Gaussian and serially uncorrelated; joint Gaussianity then implies independence across dates.

A process can be serially uncorrelated while still being dependent through higher moments. Conditional heteroskedasticity is an important example: the sign of tomorrow’s shock may be unpredictable even though the magnitude of tomorrow’s shock depends on recent volatility. We return to this distinction when studying ARCH and GARCH models.

Why \(\phi\) Measures Persistence

Suppose an innovation of size one occurs at time \(t\), with subsequent innovations held at zero. Its effects are

Date	Effect
\(t\)	\(1\)
\(t+1\)	\(\phi\)
\(t+2\)	\(\phi^2\)
\(t+k\)	\(\phi^k\)

Thus, \(|\phi|\) controls how quickly the effect decays. The sign of \(\phi\) controls whether the effect decays smoothly or alternates.

For example:

• If \(\phi=0.5\), only \(0.5^{10}\approx0.001\) of a shock remains after 10 periods.
• If \(\phi=0.9\), about \(0.9^{10}\approx0.35\) remains.
• If \(\phi=0.99\), about \(0.99^{10}\approx0.90\) remains.
• If \(\phi=1\), the effect remains exactly one.

This is the operational meaning of memory in an AR(1).

Deeper Dive — Recursive substitution and the MA(\(\infty\)) representation

Begin with

\[ y_t=\alpha+\phi y_{t-1}+\epsilon_t. \]

Substitute for \(y_{t-1}\):

\[ y_t=\alpha(1+\phi)+\phi^2y_{t-2} +\phi\epsilon_{t-1}+\epsilon_t. \]

After \(k\) substitutions,

\[ y_t = \alpha\sum_{i=0}^{k-1}\phi^i +\phi^k y_{t-k} +\sum_{i=0}^{k-1}\phi^i\epsilon_{t-i}. \]

If \(|\phi|<1\), then \(\phi^k\rightarrow0\) and the geometric sum converges:

\[ y_t = \frac{\alpha}{1-\phi} +\sum_{i=0}^{\infty}\phi^i\epsilon_{t-i}. \]

This is the causal MA(\(\infty\)) representation of the stationary AR(1). It makes the memory interpretation exact: \(y_t\) is composed of present and past innovations with geometrically declining weights.

Looking Ahead — Wold decomposition

The AR(1) representation foreshadows the Wold decomposition. Roughly, the purely nondeterministic part of a covariance-stationary process can be represented as an infinite moving average of uncorrelated innovations. We will use this idea later; Module 1 does not require the formal theorem.

The Lag Operator

The lag operator \(L\) shifts a series back one period:

\[ Ly_t=y_{t-1},\qquad L^ky_t=y_{t-k}. \]

The AR(1) can therefore be written

\[ y_t-\phi y_{t-1}=\alpha+\epsilon_t \]

or

\[ (1-\phi L)y_t=\alpha+\epsilon_t. \]

The difference operator is a special lag polynomial:

\[ \Delta y_t = y_t-y_{t-1} = (1-L)y_t. \]

This notation will appear throughout the course. At this stage, fluency means being able to translate between subscripts and lag-operator notation.

Looking Ahead — Roots of the lag polynomial

The AR(1) polynomial is \(\Phi(z)=1-\phi z\). Its root is \(z=1/\phi\). Stationarity requires the root to lie outside the unit circle:

\[ \left|\frac{1}{\phi}\right|>1 \quad\Longleftrightarrow\quad |\phi|<1. \]

For an AR(\(p\)), all roots of \(\Phi(z)=0\) must lie outside the unit circle. Module 3 develops that condition.

Building an AR(1) Simulator in R

The following function translates the DGP directly into code:

ar1_simulator <- function(
    n = 100,
    alpha = 0,
    phi = 0,
    sigma = 1,
    init = 0,
    burn_in = 500,
    start_date = c(1955, 11),
    frequency = 12) {

  stopifnot(
    length(n) == 1, n >= 1, n == as.integer(n),
    length(phi) == 1,
    length(sigma) == 1, sigma > 0,
    length(burn_in) == 1, burn_in >= 0,
    burn_in == as.integer(burn_in)
  )

  total_n <- n + burn_in
  y <- numeric(total_n + 1)
  y[1] <- init  # This is y_0.

  for (t in seq_len(total_n)) {
      epsilon_t <- rnorm(1, mean = 0, sd = sigma)
      y[t + 1] <- alpha + phi * y[t] + epsilon_t
  }

  # y[2] is the first generated value, so retained observations begin
  # one index after the corresponding simulation period.
  keep <- seq.int(from = burn_in + 2, to = total_n + 1)
  stats::ts(
    y[keep],
    start = start_date,
    frequency = frequency
  )
}

The central line is

y[t] <- alpha + phi * y[t - 1] + epsilon_t

It is the equation

\[ y_t=\alpha+\phi y_{t-1}+\epsilon_t \]

written as a loop.

The innovations are forced to have mean zero. A nonzero mean for rnorm() would be absorbed into the intercept and would blur the interpretation of \(\alpha\).

Deeper Dive — What burn-in does and does not do

The initial value is chosen by the programmer, not drawn by nature. For a stationary AR(1), its effect after \(b\) periods is proportional to \(\phi^b\). Discarding early observations reduces dependence on that arbitrary initialization.

The needed burn-in grows sharply near the unit-root boundary:

\[ 0.5^{100}\approx 7.9\times10^{-31}, \qquad 0.99^{100}\approx0.37, \qquad 0.99^{500}\approx0.0066. \]

For \(\phi=1\), no burn-in can make the process settle into a stationary distribution because no such distribution exists. Discarding a prefix of a random walk merely changes which part of the same nonstationary path is displayed.

What Memory Looks Like

Use the same seed before each simulation so differences are driven by \(\phi\) rather than by different shock sequences:

pacman::p_load(forecast, ggplot2, patchwork)

simulate_with_common_shocks <- function(phi) {
  set.seed(8675309)
  ar1_simulator(n = 819, phi = phi, burn_in = 500)
}

p0 <- autoplot(simulate_with_common_shocks(0.0)) +
  theme_bw(base_size = 15) + labs(title = expression(phi == 0), y = "")

p5 <- autoplot(simulate_with_common_shocks(0.5)) +
  theme_bw(base_size = 15) + labs(title = expression(phi == 0.5), y = "")

p9 <- autoplot(simulate_with_common_shocks(0.9)) +
  theme_bw(base_size = 15) + labs(title = expression(phi == 0.9), y = "")

p1 <- autoplot(simulate_with_common_shocks(1.0)) +
  theme_bw(base_size = 15) + labs(title = expression(phi == 1), y = "")

(p0 | p5) / (p9 | p1)

You should see:

• \(\phi=0\): no visible linear persistence.
• \(\phi=0.5\): short runs above or below the mean, followed by relatively quick reversion.
• \(\phi=0.9\): long, smooth-looking excursions that eventually revert.
• \(\phi=1\): wandering with no fixed mean or variance to which the process returns.

Core checkpoint: \(\phi=0.9\) and \(\phi=1\) can look similar in a short sample even though they have qualitatively different long-run behavior. A plot alone cannot reliably distinguish them.

---

1.4 Stationarity

A Process and One Realization

A stochastic process is a collection of random variables indexed by time:

\[ \{Y_t:t\in\mathbb{Z}\}. \]

The data in a spreadsheet are one realization, or sample path, from that process:

\[ y_1,y_2,\ldots,y_T. \]

The course usually writes \(y_t\) for both the theoretical series and its observed values. In this section only, uppercase \(Y_t\) emphasizes a random variable in the process and lowercase \(y_t\) denotes a realized value.

Had different innovations occurred, we would have observed a different path generated by the same probabilistic rules. Time-series analysis uses one observed path to learn about those rules.

Technical Note — Stationarity is not enough for every inferential result

Stationarity says that important probabilistic features are stable over time. By itself, it does not guarantee that sample averages converge to population averages. Consistency of sample moments generally also requires an ergodic or weak-dependence condition.

At this level, think of stationarity as a necessary part of the standard toolkit, not as a magical assumption that makes one realization equivalent to many independent samples.

Weak Stationarity: Three Conditions

A process \(\{Y_t\}\) is weakly stationary, or covariance stationary, if:

1. Its mean is constant

   \[ \mathbb{E}[Y_t]=\mu \qquad\text{for every }t. \]

2. Its variance is finite and constant

   \[ \operatorname{Var}(Y_t)=\sigma_Y^2<\infty \qquad\text{for every }t. \]

3. Its autocovariance depends on separation, not calendar time

   \[ \operatorname{Cov}(Y_t,Y_{t-k})=\gamma_k, \]

   where \(\gamma_k\) depends on the lag \(k\) but not on \(t\).

Stationarity does not mean the series is flat or motionless. A stationary series can fluctuate substantially. Stationarity means that the probabilistic rules governing those fluctuations do not change with calendar time.

Technical Note — Strict stationarity

A process is strictly stationary if every finite-dimensional joint distribution is invariant to a common shift in time:

\[ (Y_{t_1},\ldots,Y_{t_m}) \overset{d}{=} (Y_{t_1+h},\ldots,Y_{t_m+h}) \]

for all collections of dates and all shifts \(h\). Strict stationarity implies weak stationarity when the relevant second moments exist. Weak stationarity does not generally imply strict stationarity.

This course uses “stationary” to mean weakly stationary unless otherwise stated.

Properties of a Stationary AR(1)

For

\[ Y_t=\alpha+\phi Y_{t-1}+\epsilon_t, \qquad |\phi|<1, \]

with white-noise innovations of variance \(\sigma^2\), the stationary moments are:

\[ \mathbb{E}[Y_t] = \mu = \frac{\alpha}{1-\phi}, \]

\[ \operatorname{Var}(Y_t) = \gamma_0 = \frac{\sigma^2}{1-\phi^2}, \]

and

\[ \operatorname{Cov}(Y_t,Y_{t-k}) = \gamma_k = \phi^k\gamma_0 \qquad(k\geq0). \]

None of these quantities depends on calendar time. The autocovariance depends only on the lag. Thus, all three weak-stationarity conditions hold.

Deeper Dive — Deriving the mean and variance

Under stationarity,

\[ \mathbb{E}[Y_t]=\mathbb{E}[Y_{t-1}]=\mu. \]

Taking expectations gives

\[ \mu=\alpha+\phi\mu, \]

so

\[ \mu=\frac{\alpha}{1-\phi}. \]

From the MA(\(\infty\)) representation,

\[ Y_t-\mu=\sum_{i=0}^{\infty}\phi^i\epsilon_{t-i}. \]

Because the innovations are uncorrelated and have common variance \(\sigma^2\),

\[ \operatorname{Var}(Y_t) = \sigma^2\sum_{i=0}^{\infty}\phi^{2i} = \frac{\sigma^2}{1-\phi^2}. \]

The geometric sum exists only when \(|\phi|<1\).

It is tempting to substitute \(\phi=1\) into the stationary formula and say that the random walk has “infinite variance.” More precisely, the stationary variance formula is not valid at \(\phi=1\). A random walk has a finite variance at each finite date, but that variance depends on time and grows without bound as time advances.

Looking Ahead — Trend stationarity and difference stationarity

A trend-stationary process can be written as

\[ Y_t=a+bt+u_t, \]

where \(u_t\) is stationary. Removing the deterministic trend leaves a stationary process.

A difference-stationary process satisfies

\[ \Delta Y_t\text{ is stationary}, \]

even though \(Y_t\) is not. A random walk is the basic example.

These processes can look similar in a short plot but require different treatment. Module 2 develops the evidence used to distinguish them.

---

1.5 Measuring Memory: Autocovariance and the ACF

Autocovariance

For two dates \(t\) and \(s\), define

\[ \gamma(t,s) = \operatorname{Cov}(Y_t,Y_s) = \mathbb{E}\!\left[ (Y_t-\mathbb{E}[Y_t]) (Y_s-\mathbb{E}[Y_s]) \right]. \]

For a stationary process, the mean is constant and the covariance depends only on the lag. We therefore write

\[ \gamma_k = \operatorname{Cov}(Y_t,Y_{t-k}) = \mathbb{E}[(Y_t-\mu)(Y_{t-k}-\mu)]. \]

Important properties include:

• \(\gamma_0=\operatorname{Var}(Y_t)\).
• \(\gamma_k=\gamma_{-k}\).
• \(|\gamma_k|\leq\gamma_0\).

The sample autocovariance is

\[ \widehat{\gamma}_k = \frac{1}{T} \sum_{t=k+1}^{T} (y_t-\bar y)(y_{t-k}-\bar y). \]

Technical Note — Why some formulas use \(T-k\)

Both \(1/T\) and \(1/(T-k)\) appear as divisors in sample-autocovariance formulas. They have different finite-sample properties. R’s acf() uses a common-sample normalization equivalent to dividing by the number of observations for complete data. That convention helps ensure the estimated autocovariance sequence has the appropriate nonnegative-definiteness properties.

For Module 1, interpreting the pattern matters more than memorizing the divisor.

The Autocorrelation Function

Autocovariance depends on measurement units. Autocorrelation removes scale:

\[ \rho_k=\frac{\gamma_k}{\gamma_0}. \]

Consequently,

\[ -1\leq\rho_k\leq1,\qquad \rho_0=1,\qquad \rho_k=\rho_{-k}. \]

For a stationary AR(1),

\[ \boxed{\rho_k=\phi^k.} \]

This is the key Module 1 result.

\(\phi\)	\(\rho_1\)	\(\rho_2\)	\(\rho_5\)	\(\rho_{10}\)	Pattern
\(-0.7\)	\(-0.700\)	\(0.490\)	\(-0.168\)	\(0.028\)	Alternating decay
\(0.3\)	\(0.300\)	\(0.090\)	\(0.002\)	\(\approx0\)	Very fast decay
\(0.5\)	\(0.500\)	\(0.250\)	\(0.031\)	\(0.001\)	Fast decay
\(0.9\)	\(0.900\)	\(0.810\)	\(0.590\)	\(0.349\)	Slow decay
\(0.95\)	\(0.950\)	\(0.903\)	\(0.774\)	\(0.599\)	Very slow decay

The ACF is a useful fingerprint:

• A geometrically decaying ACF is consistent with a stationary AR process.
• Alternating decay suggests a negative autoregressive coefficient.
• Very slow decay is consistent with high persistence and may indicate nonstationarity.
• An ACF alone does not prove that a unit root is present.

Reading a Correlogram in R

A correlogram plots the sample autocorrelations \(\widehat{\rho}_k\) against the lags:

pacman::p_load(forecast, ggplot2)

set.seed(8675309)
y <- ar1_simulator(n = 819, alpha = 0, phi = 0.5)

ggAcf(y, lag.max = 24) +
  theme_bw(base_size = 16)

Read the plot in stages:

1. What is the sign of the first few correlations?
2. Does the ACF decay, remain persistently high, alternate, or cut off?
3. How quickly does any decay occur?
4. Are there spikes at substantively meaningful lags, such as 12 for monthly data?
5. Could the visible spikes be sampling variation?

The dashed bands are commonly drawn at approximately

\[ \pm\frac{1.96}{\sqrt{T}}. \]

They are approximate, individual-lag reference bands under an iid white-noise null. A bar outside the bands is evidence against zero autocorrelation at that lag, but:

• examining many lags creates a multiple-comparison issue;
• the formula is not a general confidence interval for every dependent process;
• one isolated crossing among many lags is not necessarily surprising.

Looking Ahead — AR and MA fingerprints

In Modules 3 and 4, the difference between an ACF that tails off and one that cuts off helps distinguish autoregressive from moving-average behavior. Module 1 asks only that you recognize persistence and describe its decay.

The Intercept Changes Level, Not Memory

For a stationary AR(1),

\[ \mu=\frac{\alpha}{1-\phi}, \qquad \rho_k=\phi^k. \]

Changing \(\alpha\) changes the long-run mean. Holding \(\phi\) fixed leaves the theoretical ACF unchanged.

set.seed(8675309)
s1 <- ar1_simulator(n = 5000, alpha = 2, phi = 0.5)

set.seed(8675309)
s2 <- ar1_simulator(n = 5000, alpha = 4, phi = 0.5)

set.seed(8675309)
s3 <- ar1_simulator(n = 5000, alpha = 6, phi = 0.5)

The realized ACFs can differ slightly because they are estimated from finite samples, but their population ACFs are identical.

Core checkpoint: \(\alpha\) determines where a stationary AR(1) lives; \(\phi\) determines how shocks propagate.

---

1.6 Perfect Positive Memory: The Random Walk

The Random-Walk Equation

Set \(\phi=1\) and \(\alpha=0\):

\[ Y_t=Y_{t-1}+\epsilon_t. \]

Recursive substitution gives

\[ Y_t=Y_0+\sum_{i=1}^{t}\epsilon_i. \]

Every past innovation remains in the current level with weight one. With a fixed initial value \(Y_0\) and independent innovations,

\[ \mathbb{E}[Y_t]=Y_0 \]

and

\[ \operatorname{Var}(Y_t)=t\sigma^2. \]

The variance is finite for every finite \(t\), but it changes with time and grows without bound. The random walk therefore violates weak stationarity.

Adding an intercept gives a random walk with drift:

\[ Y_t=\alpha+Y_{t-1}+\epsilon_t = Y_0+\alpha t+\sum_{i=1}^{t}\epsilon_i. \]

Stationary AR(1) Versus Random Walk

Property	Stationary AR(1), \(|\phi|<1\)	Random walk, \(\phi=1\)
Mean	\(\alpha/(1-\phi)\)	\(Y_0+\alpha t\)
Variance	\(\sigma^2/(1-\phi^2)\)	\(t\sigma^2\) for fixed \(Y_0\)
Shock effect	Decays as \(\phi^k\)	Persists with weight one
Long-run behavior	Mean reverting	No fixed mean-reverting level
Stationary theoretical ACF	\(\rho_k=\phi^k\)	Not defined
Typical sample ACF	Geometric decay	Often starts high and decays slowly

The phrase “the ACF of a random walk is approximately one” is common shorthand, but it must be interpreted carefully. Because a random walk is not stationary, it has no theoretical ACF depending only on the lag.

Technical Note — Time-dependent correlation in a random walk

For a zero-drift random walk with fixed \(Y_0=0\),

\[ \operatorname{Cov}(Y_t,Y_{t-k})=(t-k)\sigma^2, \]

and

\[ \operatorname{Corr}(Y_t,Y_{t-k}) = \sqrt{\frac{t-k}{t}}. \]

This correlation depends on both \(t\) and \(k\), not only on \(k\). That dependence on calendar time is exactly why it is not a stationary ACF.

Simulation

set.seed(8675309)
rw <- ar1_simulator(
  n = 500,
  alpha = 0,
  phi = 1,
  burn_in = 0
)

ts_plot <- autoplot(rw) +
  theme_bw(base_size = 14) +
  labs(title = "Random walk", y = "")

acf_plot <- ggAcf(rw, lag.max = 48) +
  theme_bw(base_size = 14) +
  labs(title = "Sample ACF of the random walk")

ts_plot | acf_plot

You will usually see a wandering series and a slowly decaying sample ACF. Those are warning signs, not a formal unit-root test.

Looking Ahead — Differencing

For a random walk,

\[ \Delta Y_t = Y_t-Y_{t-1} = \epsilon_t. \]

The level is nonstationary, but its first difference is white noise. Module 2 turns this observation into an applied diagnostic workflow.

---

1.7 Spurious Regression

The Problem

Suppose \(X_t\) and \(Y_t\) are independent random walks:

\[ X_t=X_{t-1}+u_t, \qquad Y_t=Y_{t-1}+v_t, \]

where the two innovation sequences are independent.

There is no population relationship between the processes. Nevertheless, a conventional regression of \(Y_t\) on \(X_t\) can produce a large \(R^2\), an apparently large \(t\) statistic, and a very small reported \(p\)-value. This is the classic spurious-regression problem associated with Granger and Newbold (1974).

One Simulation

set.seed(42)
n <- 100

x <- cumsum(rnorm(n))
y <- cumsum(rnorm(n))

spurious_fit <- lm(y ~ x)
summary(spurious_fit)

Call:
lm(formula = y ~ x)

Residuals:
   Min     1Q Median     3Q    Max 
-8.484 -2.980 -1.065  2.463  9.537 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  -7.3812     0.5431 -13.590  < 2e-16 ***
x             0.5187     0.1533   3.384  0.00103 ** 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 4.204 on 98 degrees of freedom
Multiple R-squared:  0.1046,    Adjusted R-squared:  0.09548 
F-statistic: 11.45 on 1 and 98 DF,  p-value: 0.001029

The numerical result depends on the realized paths. That is the point: two unrelated wandering paths can line up by chance for long stretches, and conventional regression output treats that low-frequency alignment as stronger evidence than it really is.

Why Conventional Inference Fails

The problem is deeper than “the two variables happen to trend.”

• The regressor and regression disturbance are integrated rather than stationary.
• Their sample cross-products accumulate persistent shocks.
• The usual normalization behind conventional OLS \(t\) tests is no longer valid.
• The reported \(t\) statistic does not have the familiar large-sample reference distribution and tends to become more extreme as the sample grows.

It is therefore too simple to describe this only as a small downward bias in a standard error. The conventional standard error and critical value belong to an asymptotic theory that does not apply to this levels regression.

Technical Note — What happens asymptotically

In a regression between independent random walks, the slope estimator has a nonstandard limiting distribution involving functions of Brownian motion. The conventional \(t\) statistic diverges rather than converging to a standard normal distribution. Phillips (1986) provides a formal treatment.

You do not need that derivation in Module 1. You do need to understand its implication: collecting more observations does not rescue the conventional test. The false-rejection problem can become worse.

Monte Carlo: This Is Systematic

Repeat the experiment many times:

set.seed(42)

n_sims <- 1000
n_obs <- 100

reject_rate <- mean(replicate(n_sims, {
  x <- cumsum(rnorm(n_obs))
  y <- cumsum(rnorm(n_obs))

  conventional_p <-
    coef(summary(lm(y ~ x)))["x", "Pr(>|t|)"]

  conventional_p < 0.05
}))

reject_rate

[1] 0.769

With these settings, the rejection rate is typically around three quarters rather than 5 percent. With set.seed(42) in the current course environment, it is approximately 0.77. The precise number depends on the sample length, simulation count, random-number stream, and regression specification. The qualitative conclusion is stable: conventional inference rejects far too often.

Try changing n_obs to 50, 200, and 500. Unlike a well-calibrated test, the false-rejection rate generally rises rather than settling near 5 percent.

A Stationary Contrast

Now repeat the exercise with two independent stationary AR(1) processes:

set.seed(42)

reject_rate_stationary <- mean(replicate(n_sims, {
  x <- arima.sim(model = list(ar = 0.5), n = n_obs)
  y <- arima.sim(model = list(ar = 0.5), n = n_obs)

  conventional_p <-
    coef(summary(lm(y ~ x)))["x", "Pr(>|t|)"]

  conventional_p < 0.05
}))

reject_rate_stationary

[1] 0.122

This is not the classic unit-root spurious regression: the slope estimator is centered on the true value of zero and has standard stationary-process asymptotics under suitable weak-dependence conditions. However, the usual lm() standard error still treats the regression disturbance as serially uncorrelated. Because \(Y_t\) is autocorrelated, the reported test can still be miscalibrated.

This contrast separates two lessons:

1. Unit roots can destroy the conventional levels-regression asymptotics.
2. Even stationary serial dependence must be handled when estimating uncertainty.

Later modules introduce dynamic specifications and dependence-robust inference.

Looking Ahead — Cointegration is the important exception

Not every regression between nonstationary variables is meaningless. If a linear combination of the variables is stationary, they may be cointegrated and the levels relationship can represent a genuine long-run equilibrium.

Do not translate the spurious-regression lesson into “always difference every nonstationary variable.” The correct workflow is to diagnose the stochastic properties of the series and then choose a model consistent with those properties. Cointegration is developed later in the course.

The Practical Takeaway

Before trusting a conventional time-series regression:

1. Plot the variables.
2. Examine persistence and possible structural changes.
3. Determine whether a stationary specification is plausible.
4. Test and transform thoughtfully rather than mechanically.
5. Consider whether a long-run relationship such as cointegration is substantively and statistically plausible.

Core checkpoint: A high \(R^2\) and a small conventional \(p\)-value do not, by themselves, establish a meaningful relationship between persistent time series.

---

1.8 Core Application: U.S. Unemployment

This application completes the Module 1 sequence: Concept \(\rightarrow\) Math \(\rightarrow\) Simulate \(\rightarrow\) Real Data.

The code uses the FRED series UNRATE, the seasonally adjusted monthly civilian unemployment rate.

To pull the series live, store your FRED key in the environment (never hardcode it in course files) and run:

pacman::p_load(fredr, forecast, ggplot2, patchwork)

# Store the key in the environment; never hardcode it in course files.
fredr_set_key(Sys.getenv("FRED_KEY"))

unrate <- fredr(
  series_id = "UNRATE",
  observation_start = as.Date("1948-01-01")
)

unrate <- unrate[order(unrate$date), ]

start_year <- as.integer(format(min(unrate$date), "%Y"))
start_month <- as.integer(format(min(unrate$date), "%m"))

unrate_ts <- ts(
  unrate$value,
  start = c(start_year, start_month),
  frequency = 12
)

So that these notes render identically for everyone — with or without an API key or an internet connection — the code below builds the same unrate_ts object from the course’s cached copy in data/UNRATE.csv. Everything downstream is identical either way.

pacman::p_load(forecast, ggplot2, patchwork)

unrate <- read.csv("../data/UNRATE.csv")  # cached FRED pull; see data/README.md
names(unrate)[names(unrate) == "observation_date"] <- "date"
unrate$date <- as.Date(unrate$date)

# The cache has one missing month (2025-10, a federal data-release gap).
# Keep the complete run from 1948-01 through the last month before the gap
# so the monthly ts index stays aligned.
first_na <- which(is.na(unrate$UNRATE))
if (length(first_na) > 0) unrate <- unrate[seq_len(min(first_na) - 1), ]

unrate <- unrate[order(unrate$date), ]

start_year <- as.integer(format(min(unrate$date), "%Y"))
start_month <- as.integer(format(min(unrate$date), "%m"))

unrate_ts <- ts(
  unrate$UNRATE,
  start = c(start_year, start_month),
  frequency = 12
)

length(unrate_ts)

[1] 933

p_time <- autoplot(unrate_ts) +
  theme_bw(base_size = 14) +
  labs(
    title = "U.S. civilian unemployment rate",
    x = NULL,
    y = "Percent"
  )

p_acf <- ggAcf(unrate_ts, lag.max = 48) +
  theme_bw(base_size = 14) +
  labs(
    title = "Sample ACF",
    x = "Lag",
    y = "Autocorrelation"
  )

p_time | p_acf

Questions to ask:

1. Does unemployment behave like white noise?
2. How persistent are month-to-month changes in the level?
3. Does the ACF decay quickly or slowly?
4. Do recessions appear as isolated shocks or sustained episodes?
5. Is the mean plausibly constant across the entire sample?
6. Could structural change or nonlinear behavior complicate an AR(1) interpretation?

The plot and ACF establish that unemployment is persistent. They do not, by themselves, determine whether the series contains a unit root. A highly persistent stationary process, a unit-root process, structural breaks, and nonlinear mean reversion can produce similar visual evidence in finite samples.

That ambiguity is the handoff to Module 2.

---

Common Pitfalls and Misconceptions

1. “Stationary means the series does not move.” No. Stationarity concerns stable probabilistic properties, not a flat realized path.

2. “All time series must be dependent.” No. White noise is a time series. Dependence is common and often the reason time order matters, but it is not part of the definition of time-indexed data.

3. “A persistent outcome means every regression error must be serially correlated.” No. Persistence in \(Y_t\) and serial correlation in the error of a particular model are different claims.

4. “The intercept controls memory.” In a stationary AR(1), \(\alpha\) changes the long-run mean and \(\phi\) controls the ACF and shock propagation.

5. “\(\epsilon_t\) and \(e_t\) are interchangeable.” \(\epsilon_t\) is a DGP innovation; \(e_t\) is a fitted residual.

6. “\(\phi=0.99\) and \(\phi=1\) are basically the same.” They may look similar in a short sample, but at \(0.99\) shocks eventually decay and a stationary distribution exists. At \(1\), shocks do not decay and the variance changes with time.

7. “A slowly decaying sample ACF proves there is a unit root.” No. It is evidence of persistence, not a formal verdict.

8. “A random walk has infinite variance at every date.” With a fixed initial value, its variance is \(t\sigma^2\): finite at each finite date, time-varying, and unbounded as \(t\) grows.

9. “Any regression involving nonstationary variables is spurious.” No. Cointegrated variables are the principal exception. Deterministic components and the complete model specification also matter.

10. “Stationary variables make ordinary lm() p-values automatically valid.” No. Stationary serial correlation can still require a dynamic model or dependence-robust standard errors.

---

Practice Problems

Core Practice

1. Classify each dataset as cross-sectional, time series, or panel. Explain whether rearranging its rows would destroy information.

   • A survey of 5,000 households in 2025
   • Monthly inflation from 1960 through 2025
   • Annual income for the same 1,000 people over 10 years

2. For an AR(1) with \(\phi=0.7\), compute \(\rho_1\), \(\rho_2\), \(\rho_5\), and \(\rho_{10}\). At approximately what lag does the autocorrelation fall below 0.05?

3. Repeat Problem 2 for \(\phi=-0.7\). What changes and what remains the same?

4. State the three conditions for weak stationarity in both notation and plain language.

5. Use ar1_simulator() to generate paths for

   \[ \phi\in\{-0.7,0,0.5,0.9,0.99,1\}. \]

   Describe the persistence, sign pattern, and mean-reverting behavior of each.

6. Write the following using the lag operator:

   • \(Y_t=0.3Y_{t-1}+\epsilon_t\)
   • \(Y_t=0.5Y_{t-1}-0.2Y_{t-2}+\epsilon_t\)
   • \(\Delta Y_t=Y_t-Y_{t-1}\)

7. A zero-drift random walk begins at \(Y_0=0\) and has innovation variance \(\sigma^2=4\). Compute \(\mathbb{E}[Y_{25}]\) and \(\operatorname{Var}(Y_{25})\). Which stationarity condition fails?

8. Explain why a large \(R^2\) between two persistent series is not sufficient evidence of a meaningful relationship.

Deeper-Dive Practice

9. Starting from the AR(1), use recursive substitution to derive the MA(\(\infty\)) representation for \(|\phi|<1\).

10. Derive the stationary mean and variance of the AR(1). Identify exactly where the condition \(|\phi|<1\) enters.

11. Run the spurious-regression Monte Carlo for \(T\in\{50,100,200,500\}\). Plot the rejection rate against \(T\). Explain why this pattern is the opposite of what we expect from a correctly calibrated test.

12. Compare the sample ACFs of a stationary AR(1) with \(\phi=0.95\) and a random walk using samples of length 50, 200, and 2,000. When does the difference become visually clearer?

Technical Practice

13. Construct two random variables that are uncorrelated but dependent. Explain why this distinction matters for weak white noise.

14. Show algebraically that

    \[ (1-L)^2Y_t=Y_t-2Y_{t-1}+Y_{t-2}. \]

15. For a zero-drift random walk with fixed \(Y_0=0\), derive

    \[ \operatorname{Corr}(Y_t,Y_{t-k}) = \sqrt{\frac{t-k}{t}}. \]

Looking-Ahead Practice

16. Simulate

    \[ Y_t=0.05t+u_t,\qquad u_t=0.7u_{t-1}+\epsilon_t, \]

    and a random walk with drift. Explain why the plots can look similar and why detrending and differencing are not interchangeable operations.

---

Technical Appendix A: The Lag Operator as a Matrix

This appendix is optional. It makes the finite-sample action of a lag polynomial concrete.

Let

\[ \mathbf{y} = \begin{bmatrix} y_1&y_2&\cdots&y_T \end{bmatrix}', \]

and consider the zero-intercept AR(1)

\[ y_t=\phi y_{t-1}+\epsilon_t. \]

Define

\[ \mathbf{\Phi} = \begin{bmatrix} 1 & 0 & 0 & \cdots & 0\\ -\phi & 1 & 0 & \cdots & 0\\ 0 & -\phi & 1 & \cdots & 0\\ \vdots & \ddots & \ddots & \ddots & \vdots\\ 0 & \cdots & 0 & -\phi & 1 \end{bmatrix}. \]

Then

\[ \mathbf{\Phi}\mathbf{y} = \boldsymbol{\epsilon} +\begin{bmatrix} \phi y_0&0&\cdots&0 \end{bmatrix}'. \]

The boundary term matters. The simpler statement

\[ \mathbf{\Phi}\mathbf{y}=\boldsymbol{\epsilon} \]

is exact only when \(y_0=0\) or when the first observation is defined to absorb the initial condition.

In R:

build_phi_matrix <- function(phi, n) {
  stopifnot(n >= 1, n == as.integer(n))

  phi <- as.numeric(phi)
  p <- length(phi)
  out <- diag(1, n)

  if (p == 0 || n == 1) {
    return(out)
  }

  for (j in seq_len(min(p, n - 1))) {
    index <- cbind(
      (j + 1):n,
      1:(n - j)
    )
    out[index] <- -phi[j]
  }

  out
}

build_phi_matrix(phi = 0.7, n = 6)

     [,1] [,2] [,3] [,4] [,5] [,6]
[1,]  1.0  0.0  0.0  0.0  0.0    0
[2,] -0.7  1.0  0.0  0.0  0.0    0
[3,]  0.0 -0.7  1.0  0.0  0.0    0
[4,]  0.0  0.0 -0.7  1.0  0.0    0
[5,]  0.0  0.0  0.0 -0.7  1.0    0
[6,]  0.0  0.0  0.0  0.0 -0.7    1

build_phi_matrix(phi = c(0.5, 0.3), n = 6)

     [,1] [,2] [,3] [,4] [,5] [,6]
[1,]  1.0  0.0  0.0  0.0  0.0    0
[2,] -0.5  1.0  0.0  0.0  0.0    0
[3,] -0.3 -0.5  1.0  0.0  0.0    0
[4,]  0.0 -0.3 -0.5  1.0  0.0    0
[5,]  0.0  0.0 -0.3 -0.5  1.0    0
[6,]  0.0  0.0  0.0 -0.3 -0.5    1

For an AR(\(p\)), the matrix has one main diagonal and up to \(p\) nonzero subdiagonals. It is lower triangular and banded. This representation becomes useful in likelihood calculations, state-space methods, and models with structured covariance matrices.

---

References and Further Reading

• Cryer, J. D., and Chan, K.-S. Time Series Analysis: With Applications in R, Chapters 1–2.
• Enders, W. Applied Econometric Time Series, Chapters 1–2 and the discussion of spurious regression.
• Granger, C. W. J., and Newbold, P. (1974). “Spurious Regressions in Econometrics.” Journal of Econometrics 2: 111–120.
• Hamilton, J. D. Time Series Analysis, Chapter 3, for a more formal treatment of stochastic processes and stationarity.
• Phillips, P. C. B. (1986). “Understanding Spurious Regressions in Econometrics.” Journal of Econometrics 33: 311–340.

---

Key Takeaways

1. Time ordering can contain information. Treating periods as interchangeable can discard the dependence we want to understand.
2. The master equation is the course roadmap; the AR(1) turns on one autoregressive channel.
3. The innovation \(\epsilon_t\) belongs to the DGP. The residual \(e_t\) belongs to a fitted model.
4. In an AR(1), \(\phi\) governs shock propagation. For \(|\phi|<1\), effects decay as \(\phi^k\).
5. Weak stationarity requires a constant finite mean, a constant finite variance, and autocovariance that depends only on the lag.
6. For a stationary AR(1), \(\rho_k=\phi^k\).
7. A random walk is nonstationary because its variance is time-dependent and unbounded as time advances. It does not possess a stationary theoretical ACF.
8. A slowly decaying sample ACF indicates persistence but does not, by itself, prove that a unit root is present.
9. Regressing unrelated random walks can produce highly misleading conventional regression evidence.
10. Stationary dependence can also invalidate iid standard errors, while cointegration is an important exception to the claim that every nonstationary levels regression is meaningless.