Back to the … Time Series?

Module 1 · Memory, Stationarity, and the Ghost of Spurious Regression

Gary Cornwall

Econ 6376 · The George Washington University

Act I · What makes time series different

Time series is a different animal

A cross-section is a crowd photographed once. A time series is one subject, photographed again and again, where order is everything.

  • Marty goes back to 1955 and shifts one punch in a parking lot — and his entire present rewrites itself.
  • Shuffle a cross-section: nothing lost. Shuffle a time series: the story is gone.
  • The central question of the course: how much memory does a series carry?
  • Everything downstream is just: how much? how do we measure it? what do we do about it?

Past · present · future of a single process — one dashboard.

Three shapes of data

Cross-section {yi,xi},i=1,…,N\{y_i,\ x_i\},\quad i = 1,\dots,N One snapshot, many subjects. Order doesn’t matter — shuffle the rows and nothing is lost.

Time series {yt,yt−k},t=1,…,T\{y_t,\ y_{t-k}\},\quad t = 1,\dots,T One subject, many snapshots. Order is everything. That dependence is the course.

Panel / longitudinal {yit,xit}\{y_{it},\ x_{it}\} Many subjects through time:

  • Pooled cross-section — ignore time.
  • Repeated cross-section — new people each period, time matters.
  • Panel — the same people tracked over time.

We live in the middle column — but the machinery travels: a panel is time series stacked across subjects.

You never have enough time

Cross-section asymptotics lean on N→∞N \to \infty. Time series leans on T→∞T \to \infty — and time is stingy.

  • People on Earth: ≈8×109\approx 8\times10^{9}.   Stars in view: ≈1024\approx 10^{24}.
  • Quarterly US GDP since 1947: ≈310\approx 310 observations.
  • The universe is only ≈1.4×1010\approx 1.4\times10^{10} years old.

. . .

So our estimators must work in short, dependent samples. That constraint shapes every choice in this course.

Depending on what you measure, you may need A LOT of data before the asymptotics kick in.

The master equation — one mixing board

yt=α⏟level+δt⏟trend+∑j=1pϕjyt−j⏟AR · memory+∑l=1qθlϵt−l⏟MA+ϵt⏟innovationy_t = \underbrace{\alpha}_{\text{level}} + \underbrace{\delta t}_{\text{trend}} + \underbrace{\textstyle\sum_{j=1}^{p}\phi_j y_{t-j}}_{\text{AR · memory}} + \underbrace{\textstyle\sum_{l=1}^{q}\theta_l \epsilon_{t-l}}_{\text{MA}} + \underbrace{\epsilon_t}_{\text{innovation}}

Every model in this course is this one equation with some faders pushed to zero. Lecture by lecture we raise new channels.

Today only three faders are up: yt=α+ϕ1yt−1+ϵty_t = \alpha + \phi_1 y_{t-1} + \epsilon_t

Model Channels live
White noise ϵt\epsilon_t
AR(1) — today α,ϕ1yt−1,ϵt\alpha,\ \phi_1 y_{t-1},\ \epsilon_t
ARMA(p,q) M3–M5

Two equations, never confuse them

The DGP — the truth (unobserved): yt=α+ϕyt−1+ϵty_t = \alpha + \phi\, y_{t-1} + \epsilon_t

  • Parameters unhatted: α,ϕ\alpha,\ \phi.
  • ϵt\epsilon_t — the innovation. The genuine surprise.

Your model — estimated (observed): yt=α̂+ϕ̂yt−1+ety_t = \hat\alpha + \hat\phi\, y_{t-1} + e_t

  • Parameters hatted: α̂,ϕ̂\hat\alpha,\ \hat\phi.
  • ete_t — the residual. What’s left over.

If your model is right, the residual ete_t behaves like the innovation ϵt\epsilon_t. When it doesn’t, your model is wrong. Every diagnostic in this course is that one sentence.

Act II · Memory, made literal

Turn off everything but the memory

yt=α⏟where it lives+ϕ1⏟memory⋅yt−1⏟the past+ϵt⏟the surprise,ϵt∼N(0,σ2)y_t = \underbrace{\alpha}_{\text{where it lives}} + \underbrace{\phi_1}_{\text{memory}} \cdot \underbrace{y_{t-1}}_{\text{the past}} + \underbrace{\epsilon_t}_{\text{the surprise}}, \qquad \epsilon_t \sim N(0,\sigma^2)

  • ϕ1\phi_1 is the star — how much of yesterday survives into today.
  • yt−1y_{t-1} is the series looking back at itself.
  • ϵt\epsilon_t is something genuinely new — a shock.

How much does the past remember?

There’s φ — now feel it.

The grey dashed line is the same shocks with no memory (φ = 0) — the raw surprises. Drag φ toward 0 and the blue line collapses onto it; toward 1 and it pulls away, leaning back only slowly after each surprise. Same shocks — only the memory changes, and that lag is the forgetting.

Four regimes of memory

  1. ϕ1=0\phi_1 = 0 — no memory. Today owes nothing to yesterday. White noise.
  2. ϕ1∈(−1,1)\phi_1 \in (-1,1) — fading memory. The past matters, but its grip decays.
  3. |ϕ1|=1|\phi_1| = 1 — perfect memory. Every shock remembered forever.
  4. |ϕ1|>1|\phi_1| > 1 — explosive. Rare in economics; set aside.

A random walk (ϕ=1\phi=1) is Sheldon: eidetic memory, every embarrassment kept forever. A stationary series is the rest of us — vivid now, faded by next month.

The for-loop is the equation

Each point is assembled the same way: take φ of where you were (blue ring), add a fresh shock (orange), land at yty_t. The bars are the two contributions. Step +1 to build it one period at a time.

… which is exactly this R code

ar1_simulator <- function(n = 100, alpha = 0, phi = 0,
                          mu = 0, sigma = 1, burn_in = 100) {
  y <- numeric(burn_in + n)
  for (t in 2:(burn_in + n)) {
    y[t] <- alpha + phi * y[t-1] + rnorm(1, mu, sigma)
  }
  ts(y[-(1:burn_in)])          # drop burn-in
}

yt=α+ϕyt−1+ϵty_t = \alpha + \phi\, y_{t-1} + \epsilon_t

  • The loop line is the equation. No translation.
  • Burn-in: throw away the first 100 — let the process forget its arbitrary start and settle into its stationary distribution.

Production code uses arima.sim(); we hand-build the loop once, to feel the DGP in our fingers.

Two operators you’ll see every week

Lag operator LL — shift back one period: Lyt=yt−1,Lkyt=yt−kL\,y_t = y_{t-1}, \qquad L^k y_t = y_{t-k}

Rewrite the AR(1): (1−ϕ1L)yt=α+ϵt(1 - \phi_1 L)\,y_t = \alpha + \epsilon_t

Stationarity will live in the roots of that polynomial. Course convention: Φ(L)=1−ϕ1L−⋯\Phi(L) = 1 - \phi_1 L - \cdots (minus signs) — flag it if you read Enders/Hamilton.

Difference operator Δ\Delta: Δyt=yt−yt−1=(1−L)yt\Delta y_t = y_t - y_{t-1} = (1 - L)\,y_t

  • Shows up everywhere: differencing for stationarity, Δlog\Delta\log returns, ECMs.
  • It’s just (1−L)(1-L) in disguise.

The lag operator, in code: embed()

The lag operator is math. embed() is its R twin — it stacks a series beside its own lags.

y <- c(10, 12, 15, 13, 17, 14)
embed(y, 3)     # cols: [y_t, y_{t-1}, y_{t-2}]
#      [,1] [,2] [,3]
# [1,]   15   12   10
# [2,]   13   15   12
# [3,]   17   13   15
# [4,]   14   17   13
  • Row 1 is (y3,y2,y1)=(15,12,10)(y_3, y_2, y_1) = (15, 12, 10) — the present laid beside its own past.
  • You lose the first k−1k-1 rows: you can’t form a lag you don’t have yet.
  • That block is the design matrix for an AR regression — regress column 1 on the rest.

Act III · The limits of memory

Stationarity: the rules don’t change

A series is weakly stationary when:

  1. 𝔼[yt]=μ\mathbb{E}[y_t] = \mu for all tt    (constant mean)
  2. Var(yt)=σ2\operatorname{Var}(y_t) = \sigma^2 for all tt    (constant variance)
  3. Cov(yt,yt−k)=γk\operatorname{Cov}(y_t, y_{t-k}) = \gamma_k — depends on the gap kk, not on when    (stable autocovariance)

Stationarity doesn’t mean the series sits still. It means the rules governing its movement don’t change over time. That’s what makes the past a usable guide to the future.

Strict vs. weak — which rules hold still?

Strict stationarity F(yt1,…,ytn)=F(yt1+h,…,ytn+h)∀hF(y_{t_1},\dots,y_{t_n}) = F(y_{t_1+h},\dots,y_{t_n+h})\ \ \forall\, h Every finite-dimensional joint distribution is unchanged by a common time shift — the entire distribution is frozen in time, every moment, every joint shape.

Weak (covariance) stationarity

Only the first two moments must hold still: constant mean, constant variance, and autocovariance that depends on the gap, not the date.

Strict ⇒\Rightarrow weak, but weak ⇏\not\Rightarrow strict. We work with weak stationarity throughout — it’s exactly what our estimators and tests actually require.

What φ buys you — and where it breaks

When |ϕ1|<1|\phi_1| < 1: 𝔼[yt]=α1−ϕ1,Var(yt)=σ21−ϕ12\mathbb{E}[y_t] = \frac{\alpha}{1-\phi_1}, \qquad \operatorname{Var}(y_t) = \frac{\sigma^2}{1-\phi_1^{2}}

Both are constants — no tt. Conditions 1 and 2 hold.

Socratic: what happens to that variance as ϕ1→1\phi_1 \to 1?

1−12=01 - 1^2 = 0 → the variance blows up. |ϕ1|<1|\phi_1|<1 isn’t a convenience — it is the boundary of stationarity.

The autocorrelation function — memory you can plot

Autocovariance is covariance of a series with its own past: γk=𝔼[(yt−μ)(yt−k−μ)],ρk=γkγ0.\gamma_k = \mathbb{E}\!\left[(y_t-\mu)(y_{t-k}-\mu)\right], \qquad \rho_k = \frac{\gamma_k}{\gamma_0}.

For a stationary AR(1) this collapses to something beautiful: ρk=ϕkgeometric decay.\boxed{\ \rho_k = \phi^{\,k}\ } \qquad \text{geometric decay.}

ϕ\phi ρ1\rho_1 ρ2\rho_2 ρ3\rho_3 ρ10\rho_{10} memory
0.5 .50 .25 .13 .001 short
0.9 .90 .81 .73 .35 long
1.0 1 1 1 1 never decays

Watch the memory decay

Each bar is ρk=ϕk\rho_k = \phi^{k}. Drag φ: toward 0 the memory collapses in a step or two; toward 1 the bars refuse to fall — the slow-decay fingerprint of a unit root (bars flip orange when ϕ<0\phi<0, the tell-tale oscillation).

The intercept moves the level, not the memory

Three series, same ϕ=0.5\phi = 0.5, three different α\alpha. They live at different heights — yet their ACFs land right on top of each other. α\alpha sets where it lives; ϕ\phi sets how it moves.

Shocks: temporary, or forever?

Hit ⚡ Shock. The dashed gray line is the same series without the shock — so the gap between them is the shock’s lingering effect. At φ < 1 the gap fades as ϕk\phi^k; at φ = 1 it never closes. That gap is memory, made visible (and your first impulse–response — we’ll formalize it for VARs in Module 12).

When memory is perfect: the random walk

Set ϕ=1\phi = 1: yt=yt−1+ϵt=y0+∑i=1tϵi.y_t = y_{t-1} + \epsilon_t = y_0 + \sum_{i=1}^{t}\epsilon_i.

  • The series is literally the running total of every shock it has ever seen.
  • Nothing decays. Var(yt)=tσ2\operatorname{Var}(y_t) = t\,\sigma^2 — it grows without bound.
  • Condition 2 of stationarity is dead.

The cloud that never converges

Thirty random walks off the same starting line, each shocked independently. The dashed envelope is ±2σt\pm 2\sigma\sqrt{t} — the spread grows without bound. A stationary series would hug a fixed band; this cloud never settles.

Stationary vs. random walk

Property Stationary |ϕ|<1\,|\phi|<1 Random walk ϕ=1\,\phi=1
Mean α1−ϕ\dfrac{\alpha}{1-\phi} (constant) y0y_0 (fragile)
Variance σ21−ϕ2\dfrac{\sigma^2}{1-\phi^2} (constant) tσ2t\,\sigma^2 (grows!)
ACF ρk=ϕk\rho_k=\phi^k (decays) ρk≈1\rho_k \approx 1 for many lags
Shocks temporary — fade at ϕk\phi^k permanent — kept forever

The random walk is Sheldon, in data form. Eidetic memory.

Real data: does unemployment remember?

Before the plot — commit. U.S. unemployment, monthly since 1948. Is its ACF (a) flat at zero, (b) decaying quickly to zero, or (c) barely decaying at all?

ρ̂1=0.97\hat\rho_1 = 0.97, and the ACF is still outside the band at lag 48 — four years of memory. Recessions are not isolated shocks; they are sustained episodes.

But slow decay alone does not settle it. A highly persistent stationary process, a true unit root, and a series with structural breaks all look like this in a finite sample. We cannot yet tell which one we are holding.

Could you tell them apart?

Series A is the stationary ϕ=0.9\phi = 0.9; Series B is the random walk. But in a 50-observation sample, strong-but-fading memory and a true unit root look almost identical.

Eyeballing is where you start, not where you stop — so we need a formal verdict. That test is exactly where the next module begins.

Act IV · Why we check first

The trap: spurious regression

Two non-stationary series can look deeply related when they share nothing.

  • This is spurious regression — unless the series are genuinely cointegrated (M10).
  • Granger & Newbold (1974): the result that made a generation of econometricians paranoid — rightly.
  • OLS standard errors assume stationarity. Break that, and your tt-stats lie.

Watch it happen

Regress yy on xx — OLS, n=500n = 500:

Term Coef. Std. err. tt pp-value
Intercept −3.922 0.350 −11.21 3.7×10−263.7\times10^{-26}
x 0.211 0.029 7.30 1.1×10−121.1\times10^{-12}

R2=0.097R^2 = 0.097  ·  R‾2=0.095\bar R^2 = 0.095  ·  F1,498=53.3F_{1,498} = 53.3  ·  resid. s.e. =5.89= 5.89

A huge tt, a pp-value on the floor, a “real-looking” R2R^2 — and there is no relationship. Both series merely wander, and anything that wanders looks correlated with anything else that wanders over a long enough time span.

It is not a fluke

1000 regressions of independent random walks, T=500T = 500 each. We reject “no relationship” 89% of the time at the 5% level. Your Type I error rate is nowhere near what you think it is — and it gets worse as TT grows, not better.

The punchline

This is why stationarity is the first thing we check.


When a time-series regression hands you a beautiful R2R^2, your first thought should be:

is this real — or is this Granger & Newbold?

Key takeaways

  1. Time series = temporal dependence. Order matters; independence is gone.
  2. ϕ\phi controls memory. |ϕ|<1|\phi|<1: stationary. ϕ=1\phi=1: random walk.
  3. The ACF measures memory: fast decay → short memory; no decay → unit root.
  4. Stationarity = constant mean, constant variance, gap-only autocovariance.
  5. Non-stationary series breed spurious regressions unless cointegrated — so check first.
  6. ϵt\epsilon_t (innovation) ≠\neq ete_t (residual). This distinction runs the whole course.

Next time

  • How do we formally test for a unit root? The Dickey–Fuller test and its augmented cousin.
  • Building a case: visual inspection ++ ACF ++ formal test ++ rules of thumb.
  • Fixing non-stationarity: differencing, and the order of integration I(d)I(d).

See you in 1985.