Week 8: Models and Their Limits

Linear regression, diagnostics, and the art of knowing what your model can’t tell you

Introduction

A model is a deliberate simplification — it captures some features of reality and ignores others. This session’s exercises will have you:

  • Fit a linear regression and interpret the output
  • Use diagnostic plots to spot what a model misses
  • See what happens when you trust a model too much (overfitting)

Every model you build raises the same question: what is this model not capturing?

Exercise 1: Scatter plot and regression

Solar panels convert sunlight to electricity — but does temperature affect their output? The solar data frame has 50 observations of panel temperature (°C) and electrical output_w (watts).

Make a scatter plot. Then add a regression line using geom_smooth(method = "lm"). What direction is the relationship?

NoteHint 1

Start with ggplot(solar, aes(x = temperature, y = output_w)) and add geom_point(). Then add geom_smooth(method = "lm") on a second layer.

NoteHint 2 
ggplot(solar, aes(x = temperature, y = output_w)) +
  geom_point() +
  geom_smooth(method = "lm")
TipSolution 
ggplot(solar, aes(x = temperature, y = output_w)) +
  geom_point() +
  geom_smooth(method = "lm")

The line slopes downward — output decreases as temperature rises. This is counterintuitive but real: crystalline silicon panels lose about 0.4–0.5% efficiency per °C above 25°C. Hotter days mean more sunlight but less efficient conversion.

Exercise 2: Interpret the model

Now fit the regression with lm() and look at the summary. Extract the slope — what does it mean in plain English?

NoteHint 1

The model formula is output_w ~ temperature. The slope is the second coefficient — use coef(solar_model)[2] or coef(solar_model)["temperature"].

NoteHint 2 
solar_model <- lm(output_w ~ temperature, data = solar)
summary(solar_model)
slope <- coef(solar_model)["temperature"]
TipSolution 
solar_model <- lm(output_w ~ temperature, data = solar)
summary(solar_model)
slope <- coef(solar_model)["temperature"]
slope

The slope is about −0.35: for each 1°C increase in temperature, output drops by roughly 0.35 watts. The R² is about 0.19 — so temperature explains only about 19% of the variation. Other factors (cloud cover, panel age, angle) matter more.

Quick check

The solar model’s R² is about 0.19. What percentage of the variation in output is unexplained by temperature?

Exercise 3: HolmesCo’s diagnostics

HolmesCo monitored groundwater levels and rainfall at a site near a quarry for 5 years (60 months). They fitted a linear model and reported:

“Rainfall is the dominant control on groundwater levels (R² = 0.49, p < 0.001).”

The model is pre-fitted below as gw_model. Your job: check whether HolmesCo should be so confident. Make two diagnostic plots:

  1. Residuals vs time (month number) — is there a pattern?
  2. Residuals vs fitted values — is the spread constant?
NoteHint 1

The residuals are stored in groundwater$residuals and the month number in groundwater$month. Map month to x and residuals to y.

NoteHint 2 
ggplot(groundwater, aes(x = month, y = residuals)) +
  geom_point() +
  geom_hline(yintercept = 0, linetype = "dashed") +
  labs(x = "Month", y = "Residual (m)",
       title = "Residuals vs time")

After making this plot, try a second one with x = fitted to check for non-constant spread.

TipSolution 
# Residuals vs time
ggplot(groundwater, aes(x = month, y = residuals)) +
  geom_point() +
  geom_line(alpha = 0.3) +
  geom_hline(yintercept = 0, linetype = "dashed") +
  labs(x = "Month", y = "Residual (m)",
       title = "Residuals vs time")

# Residuals vs fitted
ggplot(groundwater, aes(x = fitted, y = residuals)) +
  geom_point() +
  geom_hline(yintercept = 0, linetype = "dashed") +
  labs(x = "Fitted value (m)", y = "Residual (m)",
       title = "Residuals vs fitted values")

The residuals vs time plot reveals a seasonal oscillation — the residuals cycle up and down roughly every 12 months. There’s also a slow downward drift (the “temporary pumping phase” HolmesCo didn’t mention). The simple gw_level ~ rainfall model misses the seasonal recharge cycle and the declining trend. “Dominant control” is a stretch when 51% of the variation is unexplained and the residuals show clear structure.

Quick check

HolmesCo says R² = 0.49 means rainfall is the “dominant control.” What percentage of the variation in groundwater level is unexplained by their model?

Extension: The overfitting trap

This exercise is optional — for students who finish early.

A more complex model should fit the data better, right? Let’s test that. The code below fits two models to the solar data:

  • A simple linear model: output_w ~ temperature
  • A degree-15 polynomial: output_w ~ poly(temperature, 15)

The polynomial has 15 terms instead of 1. It will fit the training data more closely — but would you trust it for a new observation?

TipSolution

Run the code as-is and examine the plot. The polynomial follows every bump and dip in the data — it’s memorised the noise. At the edges (below 5°C or above 42°C), it oscillates wildly. The linear model misses some detail but gives sensible predictions everywhere.

More complex ≠ more useful. The polynomial has a higher R² on this data, but it would perform terribly on new observations. This is overfitting: the model captures noise, not signal.

Save your work

Copy the code you’re most proud of into your week8.R file. Commit and push via GitHub Desktop. Write a commit message that describes what you learned — not just “week 8.”