Week 6: Check Your Assumptions

When a significant result isn’t what it seems

Introduction

In the content session, you’ve learned how to run a t-test and interpret the output. But every statistical test rests on assumptions — and when those assumptions are violated, the test can give you a misleading answer.

In these exercises, you’ll work with data from a HolmesCo borehole survey. HolmesCo measured temperatures at depth in two geological formations — the Whin Sill (a dolerite intrusion) and the Stainmore Formation (sedimentary) — to assess geothermal potential in County Durham.

You’ll discover that a “significant” result can be an artefact of violating a single assumption.

Goals:

Visualise distributions before testing
Check the normality assumption
See how log-transformation changes the conclusion
Build the reflex: always check before you test

Exercise 1: Visualise the groups

Before running any test, look at the data. Make histograms of temperature for each formation. What do you notice about the shape of the distributions?

Solution

ggplot(borehole, aes(x = temperature_c)) +
  geom_histogram(bins = 12) +
  facet_wrap(~ formation)

Both distributions are right-skewed — there’s a long tail of high temperatures. The Stainmore Formation is especially skewed, with a few very hot boreholes pulling the mean upward. This is typical of geoscience measurements like temperature at depth, permeability, and grain size — they often follow a log-normal distribution.

The t-test assumes the data within each group are approximately normal. Are they?

Exercise 2: Run the t-test on raw data

Run a t-test comparing temperature between the two formations. Store the p-value in a variable called raw_p.

Solution

raw_test <- t.test(temperature_c ~ formation, data = borehole)
raw_test

raw_p <- raw_test$p.value

The p-value is about 0.03 — “significant” at the conventional α = 0.05 threshold. HolmesCo would write this up as evidence that the two formations have different geothermal properties.

But wait. You saw in the histograms that the data are skewed. The t-test assumes normality. Let’s check that formally.

Exercise 3: Transform and re-test

Log-transform the temperature data and run the t-test again. Store the new p-value in log_p.

The log transformation pulls in the long right tail, making skewed data more symmetric — and often closer to normal.

Solution

borehole$log_temp <- log(borehole$temperature_c)
log_test <- t.test(log_temp ~ formation, data = borehole)
log_test

log_p <- log_test$p.value

The p-value is now about 0.22 — nowhere near significant.

This is exactly what happened to HolmesCo with their permeability data. They ran a t-test on raw values (p = 0.04, “significant!”) but on log-transformed values (p = 0.23, not significant). The original result was driven by a few extreme observations in the skewed tail, not by a genuine difference between formations.

The lesson: a p-value is only as trustworthy as the assumptions behind the test.

Quick check

Did the p-value increase or decrease after log-transformation?

Extension: QQ plots before and after

This exercise is optional — for students who finish early.

A QQ plot compares your data’s distribution to a theoretical normal distribution. If the points fall close to the diagonal line, the data are approximately normal. Deviations at the ends indicate skewness or heavy tails.

Make QQ plots for the Stainmore Formation, before and after log-transformation. Which looks more normal?

Solution

stainmore_raw <- borehole$temperature_c[borehole$formation == "Stainmore"]
stainmore_log <- log(stainmore_raw)

par(mfrow = c(1, 2))
qqnorm(stainmore_raw, main = "Raw temperature")
qqline(stainmore_raw)
qqnorm(stainmore_log, main = "Log temperature")
qqline(stainmore_log)

The raw data QQ plot curves away from the line at the upper end — the classic signature of right-skewness. The log-transformed version hugs the line much more closely.

You can also confirm this with the Shapiro-Wilk test: shapiro.test(stainmore_raw) gives p ≈ 0.03 (reject normality), while shapiro.test(stainmore_log) gives p ≈ 0.20 (can’t reject normality).

Save your work

Copy the code you are most proud of into your week6.R file. Commit and push via GitHub Desktop. Write a commit message that describes what you did — not just “week 6”.