Back to the … Time Series?

Module 1 · Memory, Stationarity, and the Ghost of Spurious Regression

Gary Cornwall

Econ 6376 · The George Washington University

Act I · What makes time series different

Time series is a different animal

A cross-section is a crowd photographed once. A time series is one subject, photographed again and again, where order is everything.

  • Marty goes back to 1955 and shifts one punch in a parking lot — and his entire present rewrites itself.
  • Shuffle a cross-section: nothing lost. Shuffle a time series: the story is gone.
  • The central question of the course: how much memory does a series carry?
  • Everything downstream is just: how much? how do we measure it? what do we do about it?

Past · present · future of a single process — one dashboard.

Three shapes of data

Cross-section \[\{y_i,\ x_i\},\quad i = 1,\dots,N\] One snapshot, many subjects. Order doesn’t matter — shuffle the rows and nothing is lost.

Time series \[\{y_t,\ y_{t-l}\},\quad t = 1,\dots,T\] One subject, many snapshots. Order is everything. That dependence is the course.

Panel / longitudinal \[\{y_{it},\ x_{it}\}\] Many subjects through time:

  • Pooled cross-section — ignore time.
  • Repeated cross-section — new people each period, time matters.
  • Panel — the same people tracked over time.

We live in the middle column — but the machinery travels: a panel is time series stacked across subjects.

You never have enough time

Cross-section asymptotics lean on \(N \to \infty\). Time series leans on \(T \to \infty\) — and time is stingy.

  • People on Earth: \(\approx 8\times10^{9}\).   Stars in view: \(\approx 10^{24}\).
  • Quarterly US GDP since 1947: \(\approx 310\) observations.
  • The universe is only \(\approx 1.4\times10^{10}\) years old.

. . .

So our estimators must work in short, dependent samples. That constraint shapes every choice in this course.

Depending on what you measure, you may need A LOT of data before the asymptotics kick in.

The master equation — one mixing board

\[y_t = \underbrace{\alpha}_{\text{level}} + \underbrace{\delta t}_{\text{trend}} + \underbrace{\textstyle\sum_{j=1}^{p}\phi_j y_{t-j}}_{\text{AR · memory}} + \underbrace{\textstyle\sum_{l=1}^{q}\theta_l \epsilon_{t-l}}_{\text{MA}} + \underbrace{\epsilon_t}_{\text{innovation}}\]

Every model in this course is this one equation with some faders pushed to zero. Lecture by lecture we raise new channels.

Today only three faders are up: \[y_t = \alpha + \phi_1 y_{t-1} + \epsilon_t\]

Model Channels live
White noise \(\epsilon_t\)
AR(1) — today \(\alpha,\ \phi_1 y_{t-1},\ \epsilon_t\)
ARMA(p,q) M3–M5

Two equations, never confuse them

The DGP — the truth (unobserved): \[y_t = \alpha + \phi\, y_{t-1} + \epsilon_t\]

  • Parameters unhatted: \(\alpha,\ \phi\).
  • \(\epsilon_t\) — the innovation. The genuine surprise.

Your model — estimated (observed): \[y_t = \hat\alpha + \hat\phi\, y_{t-1} + e_t\]

  • Parameters hatted: \(\hat\alpha,\ \hat\phi\).
  • \(e_t\) — the residual. What’s left over.

If your model is right, the residual \(e_t\) behaves like the innovation \(\epsilon_t\). When it doesn’t, your model is wrong. Every diagnostic in this course is that one sentence.

Act II · Memory, made literal

Turn off everything but the memory

\[y_t = \underbrace{\alpha}_{\text{where it lives}} + \underbrace{\phi_1}_{\text{memory}} \cdot \underbrace{y_{t-1}}_{\text{the past}} + \underbrace{\epsilon_t}_{\text{the surprise}}, \qquad \epsilon_t \sim N(0,\sigma^2)\]

  • \(\phi_1\) is the star — how much of yesterday survives into today.
  • \(y_{t-1}\) is the series looking back at itself.
  • \(\epsilon_t\) is something genuinely new — a shock.

How much does the past remember?

There’s φ — now feel it.

The grey dashed line is the same shocks with no memory (φ = 0) — the raw surprises. Drag φ toward 0 and the blue line collapses onto it; toward 1 and it pulls away, leaning back only slowly after each surprise. Same shocks — only the memory changes, and that lag is the forgetting.

Four regimes of memory

  1. \(\phi_1 = 0\) — no memory. Today owes nothing to yesterday. White noise.
  2. \(\phi_1 \in (-1,1)\) — fading memory. The past matters, but its grip decays.
  3. \(|\phi_1| = 1\) — perfect memory. Every shock remembered forever.
  4. \(|\phi_1| > 1\) — explosive. Rare in economics; set aside.

A random walk (\(\phi=1\)) is Sheldon: eidetic memory, every embarrassment kept forever. A stationary series is the rest of us — vivid now, faded by next month.

The for-loop is the equation

Each point is assembled the same way: take φ of where you were (blue ring), add a fresh shock (orange), land at \(y_t\). The bars are the two contributions. Step +1 to build it one period at a time.

… which is exactly this R code

ar1_simulator <- function(n = 100, alpha = 0, phi = 0,
                          mu = 0, sigma = 1, burn_in = 100) {
  y <- numeric(burn_in + n)
  for (t in 2:(burn_in + n)) {
    y[t] <- alpha + phi * y[t-1] + rnorm(1, mu, sigma)
  }
  ts(y[-(1:burn_in)])          # drop burn-in
}

\[y_t = \alpha + \phi\, y_{t-1} + \epsilon_t\]

  • The loop line is the equation. No translation.
  • Burn-in: throw away the first 100 — let the process forget its arbitrary start and settle into its stationary distribution.

Production code uses arima.sim(); we hand-build the loop once, to feel the DGP in our fingers.

Two operators you’ll see every week

Lag operator \(L\) — shift back one period: \[L\,y_t = y_{t-1}, \qquad L^k y_t = y_{t-k}\]

Rewrite the AR(1): \[(1 - \phi_1 L)\,y_t = \alpha + \epsilon_t\]

Stationarity will live in the roots of that polynomial. Course convention: \(\Phi(L) = 1 - \phi_1 L - \cdots\) (minus signs) — flag it if you read Enders/Hamilton.

Difference operator \(\Delta\): \[\Delta y_t = y_t - y_{t-1} = (1 - L)\,y_t\]

  • Shows up everywhere: differencing for stationarity, \(\Delta\log\) returns, ECMs.
  • It’s just \((1-L)\) in disguise.

The lag operator, in code: embed()

The lag operator is math. embed() is its R twin — it stacks a series beside its own lags.

y <- c(10, 12, 15, 13, 17, 14)
embed(y, 3)     # cols: [y_t, y_{t-1}, y_{t-2}]
#      [,1] [,2] [,3]
# [1,]   15   12   10
# [2,]   13   15   12
# [3,]   17   13   15
# [4,]   14   17   13
  • Row 1 is \((y_3, y_2, y_1) = (15, 12, 10)\) — the present laid beside its own past.
  • You lose the first \(k-1\) rows: you can’t form a lag you don’t have yet.
  • That block is the design matrix for an AR regression — regress column 1 on the rest.

Act III · The limits of memory

Stationarity: the rules don’t change

A series is weakly stationary when:

  1. \(\mathbb{E}[y_t] = \mu\) for all \(t\)    (constant mean)
  2. \(\operatorname{Var}(y_t) = \sigma^2\) for all \(t\)    (constant variance)
  3. \(\operatorname{Cov}(y_t, y_{t-k}) = \gamma_k\) — depends on the gap \(k\), not on when    (stable autocovariance)

Stationarity doesn’t mean the series sits still. It means the rules governing its movement don’t change over time. That’s what makes the past a usable guide to the future.

Strict vs. weak — which rules hold still?

Strict stationarity \[P_t(Y) = P(Y)\quad \forall\, t\] The entire distribution is frozen in time — every moment, every joint shape.

Weak (covariance) stationarity

Only the first two moments must hold still: constant mean, constant variance, and autocovariance that depends on the gap, not the date.

Strict \(\Rightarrow\) weak, but weak \(\not\Rightarrow\) strict. We work with weak stationarity throughout — it’s exactly what our estimators and tests actually require.

What φ buys you — and where it breaks

When \(|\phi_1| < 1\): \[\mathbb{E}[y_t] = \frac{\alpha}{1-\phi_1}, \qquad \operatorname{Var}(y_t) = \frac{\sigma^2}{1-\phi_1^{2}}\]

Both are constants — no \(t\). Conditions 1 and 2 hold.

Socratic: what happens to that variance as \(\phi_1 \to 1\)?

\(1 - 1^2 = 0\) → the variance blows up. \(|\phi_1|<1\) isn’t a convenience — it is the boundary of stationarity.

The autocorrelation function — memory you can plot

Autocovariance is covariance of a series with its own past: \[\gamma_k = \mathbb{E}\!\left[(y_t-\mu)(y_{t-k}-\mu)\right], \qquad \rho_k = \frac{\gamma_k}{\gamma_0}.\]

For a stationary AR(1) this collapses to something beautiful: \[\boxed{\ \rho_k = \phi^{\,k}\ } \qquad \text{geometric decay.}\]

\(\phi\) \(\rho_1\) \(\rho_2\) \(\rho_3\) \(\rho_{10}\) memory
0.5 .50 .25 .13 .001 short
0.9 .90 .81 .73 .35 long
1.0 1 1 1 1 never decays

Watch the memory decay

Each bar is \(\rho_k = \phi^{k}\). Drag φ: toward 0 the memory collapses in a step or two; toward 1 the bars refuse to fall — the slow-decay fingerprint of a unit root (bars flip orange when \(\phi<0\), the tell-tale oscillation).

The intercept moves the level, not the memory

Three series, same \(\phi = 0.5\), three different \(\alpha\). They live at different heights — yet their ACFs land right on top of each other. \(\alpha\) sets where it lives; \(\phi\) sets how it moves.

Shocks: temporary, or forever?

Hit ⚡ Shock. The dashed gray line is the same series without the shock — so the gap between them is the shock’s lingering effect. At φ < 1 the gap fades as \(\phi^k\); at φ = 1 it never closes. That gap is memory, made visible (and your first impulse–response — we’ll formalize it for VARs in Module 11).

When memory is perfect: the random walk

Set \(\phi = 1\): \[y_t = y_{t-1} + \epsilon_t = y_0 + \sum_{i=1}^{t}\epsilon_i.\]

  • The series is literally the running total of every shock it has ever seen.
  • Nothing decays. \(\operatorname{Var}(y_t) = t\,\sigma^2\) — it grows without bound.
  • Condition 2 of stationarity is dead.

The cloud that never converges

Thirty random walks off the same starting line, each shocked independently. The dashed envelope is \(\pm 2\sigma\sqrt{t}\) — the spread grows without bound. A stationary series would hug a fixed band; this cloud never settles.

Stationary vs. random walk

Property Stationary \(\,|\phi|<1\) Random walk \(\,\phi=1\)
Mean \(\dfrac{\alpha}{1-\phi}\) (constant) \(y_0\) (fragile)
Variance \(\dfrac{\sigma^2}{1-\phi^2}\) (constant) \(t\,\sigma^2\) (grows!)
ACF \(\rho_k=\phi^k\) (decays) \(\rho_k \approx 1\) for many lags
Shocks temporary — fade at \(\phi^k\) permanent — kept forever

The random walk is Sheldon, in data form. Eidetic memory.

Could you tell them apart?

Series A is the stationary \(\phi = 0.9\); Series B is the random walk. But in a 50-observation sample, strong-but-fading memory and a true unit root look almost identical.

Eyeballing is where you start, not where you stop — so we need a formal verdict. That test is exactly where the next module begins.

Act IV · Why we check first

The trap: spurious regression

Two non-stationary series can look deeply related when they share nothing.

  • This is spurious regression — unless the series are genuinely cointegrated (M10).
  • Granger & Newbold (1974): the result that made a generation of econometricians paranoid — rightly.
  • OLS standard errors assume stationarity. Break that, and your \(t\)-stats lie.

Watch it happen

Regress \(y\) on \(x\) — OLS, \(n = 500\):

Term Coef. Std. err. \(t\) \(p\)-value
Intercept −3.922 0.350 −11.21 \(3.7\times10^{-26}\)
x 0.211 0.029 7.30 \(1.1\times10^{-12}\)

\(R^2 = 0.097\)  ·  \(\bar R^2 = 0.095\)  ·  \(F_{1,498} = 53.3\)  ·  resid. s.e. \(= 5.89\)

A huge \(t\), a \(p\)-value on the floor, a “real-looking” \(R^2\) — and there is no relationship. Both series merely wander, and anything that wanders looks correlated with anything else that wanders over a long enough time span.

It is not a fluke

1000 regressions of independent random walks. We reject “no relationship” 89% of the time at the 5% level. Your Type I error rate is nowhere near what you think it is.

The punchline

This is why stationarity is the first thing we check.


When a time-series regression hands you a beautiful \(R^2\), your first thought should be:

is this real — or is this Granger & Newbold?

Key takeaways

  1. Time series = temporal dependence. Order matters; independence is gone.
  2. \(\phi\) controls memory. \(|\phi|<1\): stationary. \(\phi=1\): random walk.
  3. The ACF measures memory: fast decay → short memory; no decay → unit root.
  4. Stationarity = constant mean, constant variance, gap-only autocovariance.
  5. Non-stationary series breed spurious regressions unless cointegrated — so check first.
  6. \(\epsilon_t\) (innovation) \(\neq\) \(e_t\) (residual). This distinction runs the whole course.

Next time

  • How do we formally test for a unit root? The Dickey–Fuller test and its augmented cousin.
  • Building a case: visual inspection \(+\) ACF \(+\) formal test \(+\) rules of thumb.
  • Fixing non-stationarity: differencing, and the order of integration \(I(d)\).

See you in 1985.