Models and their limits

Research Methods — Week 8

Recap

Where we are

You can now test hypotheses, compare groups, and quantify effect sizes.

This week: models.

A model is a simplified story about how the world works. The question is always: how much does it capture, and what does it leave out?

• [5 min] Frame the session: models as simplifications. Every equation in science is a model. Every test contains an implicit model.

Questions?

Submit questions anonymously:

PollEv.com/geol

text geol to 07480 781235

• Open the Poll Everywhere Q&A activity before the session and leave it active throughout.
• Questions appear in your Poll Everywhere presenter view; upvoted questions rise to the top.
• Address during transitions or at the end of concept blocks, not mid-explanation.
• Reset (archive) the activity between sessions so questions don’t carry over from the previous week.

Linear regression

🎓💻 Concept block 1

What is a model?

A deliberate simplification that captures some features of reality and ignores others.

Every statistical test you’ve run contains an implicit model.

The t-test models data as two normal distributions with equal variance.

Now we make the model explicit.

• [20 min block] Linear regression — concept and live demo.

Linear regression

\[y = \beta_0 + \beta_1 x + \varepsilon\]

“y increases by β₁ for every one-unit increase in x, plus noise.”

That’s the whole model. It’s powerful — and limited.

Live demo

🖥️ Switching to WebR

model <- lm(output ~ windspeed, data = turbine_df)
summary(model)

• Show the output: coefficients, R², p-values. Plot with geom_point() + geom_smooth(method = "lm").
• Interpret: “Wind speed explains ~60% of the variation. For each 1 m/s increase, output increases by ~X kWh.”

Reading the output

Element	Meaning
Intercept (β₀)	Predicted y when x = 0
Slope (β₁)	Change in y per unit change in x
R²	Proportion of variance explained
p-value (slope)	Is the slope distinguishable from zero?

• Emphasise R² interpretation: “R² = 0.6 means 60% of the variation is explained. 40% is something else.”

Fit and interpret

✏️💻 Exercise 1

Solar panels and temperature

Solar panel output vs temperature — counterintuitive.

1. Make a scatter plot
2. Fit lm()
3. Interpret the slope and R²
4. Plot the fit

Is the relationship linear? Does the model capture the pattern?

• [15 min] Output decreases at high temperatures for crystalline silicon panels. Students discover a negative slope where they expected positive. Good moment to ask “Did the data surprise you?”

Assumptions and diagnostics

🎓 Concept block 2

The four assumptions

1. Linearity — the relationship is actually linear
2. Independence — residuals don’t influence each other
3. Homoscedasticity — constant variance of residuals
4. Normality of residuals (for inference)

• [20 min block] Assumptions, diagnostic plots, what to do when violated.

Diagnostic plots

plot(model)

Four plots, two that matter most:

Plot	What to look for
Residuals vs Fitted	Pattern = non-linearity or heteroscedasticity
QQ plot	Departures from the line = non-normal residuals

• Walk through each plot. “No pattern in residuals vs fitted = good. Points on the QQ line = good.”

HolmesCo’s regression

✏️💬 Exercise 2

The scenario

HolmesCo

“Geological Solutions Since 2019”

Groundwater Analysis Report

Linear model: groundwater level ~ rainfall. R² = 0.45.

“Rainfall is the dominant control on groundwater levels.”

Now look at the diagnostic plots…

• [15 min] Show diagnostic plots with a clear seasonal pattern in the residuals. There’s a time-lag and seasonal cycle the model isn’t capturing.

What’s wrong?

• The residuals show a seasonal pattern — the model is missing a time-dependent structure
• R² = 0.45 means 55% of the variation is unexplained
• “Dominant” is a stretch when your model misses more than it captures

• Key question: what is the model not capturing? (Seasonal recharge, time lag between rainfall and groundwater response, temperature effects on evapotranspiration.)

The three failures of models

🎓💬 Concept block 3

Failure 1: Overfitting

A model that fits the training data perfectly but fails on new data.

It’s memorised the noise.

• [25 min block] Three failures — each a different form of hubris. Demo the polynomial overfitting live.

Demo: polynomial degree 15

🖥️ Live demo

Fit a degree-15 polynomial to 20 data points. It goes through every point but oscillates wildly between them.

Compare to the straight line: fits worse, predicts better.

More complex ≠ more useful.

• Live-code: geom_smooth(method = "lm", formula = y ~ poly(x, 15)) vs y ~ x on the same plot. The visual contrast is striking.

Failure 2: Extrapolation

A model that works within the observed range but breaks down outside it.

Example: UK solar capacity has grown roughly exponentially since 2010. Extrapolate to 2050 → solar exceeds the entire UK grid.

What’s the model not capturing?

Planning constraints, grid limits, diminishing suitable sites.

HolmesCo’s extrapolation

HolmesCo

“Geological Solutions Since 2019”

“Our linear trend in groundwater decline predicts the aquifer will be empty by 2028.”

They didn’t account for seasonal recharge or the fact that the decline was caused by temporary pumping.

Failure 3: Blind spots

Things the model structurally cannot represent.

• A single-turbine power curve has no term for wake interference → overpredicts wind farm output by 10–20%
• A carbon accounting model that counts biomass as zero structurally cannot see the emissions it ignores

You can only fix a blind spot if you know it’s there.

The new refrain

“What is the model not capturing?”

Overfitting: too much faith in your data.

Extrapolation: too much faith in your model.

Blind spots: too much faith in your framework.

• These are three forms of the same mistake: confusing the map for the territory. Each is a different flavour of hubris.

Quick reflection

✏️ One sentence

Write one sentence

“In my project, the biggest thing my analysis might be missing is…”

• [5 min] Quick written reflection. Students keep this for their reports.

Wrap-up

Key points

1. A model is a simplification — powerful, but limited
2. Always check the diagnostics: residuals don’t lie
3. Three failures: overfitting, extrapolation, blind spots
4. Always ask: “What is my model not capturing?”

Next time

Application session: “Fitting and breaking models”

You’ll fit models to your data — and deliberately try to break them.

• The application session includes both project work and a structured extrapolation exercise that everyone does.