Week 8: Fitting and Breaking Models

What happens when you extrapolate beyond your data?

Introduction

In the content session you fitted regressions and checked diagnostics. Now you’ll experience model failure first-hand. Today’s goals:

Fit linear and exponential models to real UK energy data
Extrapolate both models into the future
Discover why both predictions are wrong — and what the models can’t capture

The extrapolation exercise is for everyone, regardless of whether your group project uses regression.

Exercise 1: Renewable growth

UK wind and solar generation have grown dramatically since 2010. Let’s compute the total and plot it.

Create a new column renewable_twh (= wind_twh + solar_twh), filter to 2010–2024, and plot the trend.

Hint 2

elec$renewable_twh <- elec$wind_twh + elec$solar_twh
recent <- elec[elec$year >= 2010, ]
ggplot(recent, aes(x = year, y = renewable_twh)) +
  geom_point() +
  labs(x = "Year", y = "Renewable generation (TWh)",
       title = "UK wind + solar generation, 2010–2024")

Solution

elec$renewable_twh <- elec$wind_twh + elec$solar_twh
recent <- elec[elec$year >= 2010, ]
ggplot(recent, aes(x = year, y = renewable_twh)) +
  geom_point() +
  labs(x = "Year", y = "Renewable generation (TWh)",
       title = "UK wind + solar generation, 2010–2024")

Renewable generation grew from about 10 TWh in 2010 to nearly 98 TWh in 2024 — roughly a tenfold increase. The growth looks fairly linear from 2015 onward, though the early years (2010–2014) show faster proportional growth from a small base.

Exercise 2: Linear extrapolation

Fit a linear model to the renewable generation trend and use it to predict 2030 and 2040. Are those predictions plausible?

Solution

lin_model <- lm(renewable_twh ~ year, data = recent)
summary(lin_model)
pred_2040 <- predict(lin_model, newdata = data.frame(year = 2040))
pred_2040

The linear model predicts about 213 TWh in 2040. The R² is very high (~0.97) — the linear fit is excellent for the historical data. The slope is about 6.7 TWh per year of additional renewable generation.

But linear extrapolation assumes nothing changes: no grid limits, no planning constraints, no slowdown as the best sites fill up. It projects the recent trend forward without asking whether the trend can continue.

Exercise 3: The exponential model

Early renewable growth (2010–2015) was more exponential — small base, rapid proportional increases. What if we fit an exponential model instead?

An exponential model in R: fit lm(log(y) ~ x), then back-transform predictions with exp().

Hint 2

exp_model <- lm(log(renewable_twh) ~ year, data = recent)
log_pred_2040 <- predict(exp_model, newdata = data.frame(year = 2040))
exp_pred_2040 <- exp(log_pred_2040)
exp_pred_2040

Solution

exp_model <- lm(log(renewable_twh) ~ year, data = recent)
summary(exp_model)
log_pred_2040 <- predict(exp_model, newdata = data.frame(year = 2040))
exp_pred_2040 <- exp(log_pred_2040)
exp_pred_2040

The exponential model predicts about 1,471 TWh in 2040 — more than 5 times total UK electricity generation. This is physically impossible. The model captures the early rapid growth (2010–2015) and projects it forward, ignoring every real-world constraint: finite suitable sites, grid integration limits, planning permission, and the fact that growth has already slowed.

Both models “fit” the historical data reasonably well. Neither predicts the future correctly. A model’s quality in-sample tells you nothing about its quality out-of-sample.

Quick check

Both models fit the 2010–2024 data well (linear R² ≈ 0.97; exponential R² ≈ 0.89). Yet the 2040 predictions differ by a factor of ~7. What does this tell you about extrapolation?

There’s no single right answer here — just type your thought.

Exercise 4: Reflection — your project

Think about your own group project. Every analysis has blind spots — things the model or test structurally cannot represent.

Write one sentence: “In my project, the biggest thing my analysis might be missing is…”

This isn’t graded. It’s a habit worth building: the analyst who asks this question is the one whose work a policy-maker can trust.

Extension: Visual comparison

This exercise is optional — for students who finish early.

Overlay both models on the same plot, with the data. Add a horizontal reference line at 285 TWh (total UK electricity in 2024) to give the exponential prediction some context.

Save your work

Copy the code you’re most proud of into your week8.R file. Commit and push via GitHub Desktop. Write a commit message that describes what you learned — not just “week 8.”