Week 6: First Tests

Applying hypothesis tests to real data

Introduction

This session is about applying what you learned in the content session to real data — starting with a guided walkthrough, then switching to your own project data.

Today’s goals:

  • Work through a complete t-test workflow step by step
  • Practice writing a results sentence for a policy audience
  • Learn the non-parametric alternative for when assumptions fail
  • Apply these tools to your group’s project data

The walkthrough uses the borehole temperature data. After that, you’ll work with your group on your own project data.

Exercise 1: Complete t-test workflow

Work through the full workflow. Each step builds on the previous one.

Step 1: Visualise. Side-by-side boxplots are the quickest way to compare two groups.

NoteHint

Use geom_boxplot().

TipSolution 
ggplot(borehole, aes(x = formation, y = temperature_c)) +
  geom_boxplot()

The Stainmore Formation has a higher median and much more spread. There are some high outliers. Already you should be thinking about skewness.

Step 2: Check normality. Use the Shapiro-Wilk test on each group.

TipSolution 
shapiro.test(borehole$temperature_c[borehole$formation == "Whin Sill"])
shapiro.test(borehole$temperature_c[borehole$formation == "Stainmore"])

Whin Sill passes (p ≈ 0.28) but Stainmore fails (p ≈ 0.03). Since one group departs from normality, we should either transform the data or use a non-parametric test.

Step 3: Test (with appropriate transformation).

TipSolution 
borehole$log_temp <- log(borehole$temperature_c)
result <- t.test(log_temp ~ formation, data = borehole)
result

p ≈ 0.22. Not significant. The confidence interval for the difference in means includes zero.

Step 4: Extract the confidence interval.

TipSolution 
result$conf.int

The 95% confidence interval spans from about −0.33 to 0.08 (on the log scale). Because this interval includes zero, we can’t conclude the formations differ. If the interval had been entirely above or below zero, we’d have evidence for a real difference.

Exercise 2: Write a results sentence

Based on the workflow above, write one sentence that reports the result for a policy audience. Your sentence should mention:

  • What was compared
  • The direction and size of the difference (or lack thereof)
  • The p-value or confidence interval
  • The conclusion in plain English

There’s no code to write here — type your sentence as a comment.

TipSolution

Here’s a model sentence:

Borehole temperatures did not differ significantly between the Whin Sill and Stainmore formations (t-test on log-transformed data: t = 1.25, p = 0.22, 95% CI for the difference: −0.33 to 0.08 on the log scale), suggesting that the choice of formation is unlikely to be the dominant factor in geothermal potential at these sites.

Key elements: what was tested, the statistical result, the confidence interval, and what it means in context. Notice that we mention the log-transformation — this is part of honest reporting.

Exercise 3: Non-parametric alternative

When assumptions are violated and transformation doesn’t help (or isn’t appropriate), you can use a non-parametric test. wilcox.test() is the rank-based alternative to the t-test — it doesn’t assume normality.

Run it on the raw (untransformed) borehole data:

NoteHint

Same formula interface as t.test(): wilcox.test(temperature_c ~ formation, data = borehole)

TipSolution 
wilcox.test(temperature_c ~ formation, data = borehole)

The Wilcoxon test gives p ≈ 0.09 — not significant, consistent with the log-transformed t-test (p ≈ 0.22). Both approaches agree: the raw t-test’s “significant” result (p ≈ 0.03) was an artefact of skewness.

Three approaches, one story:

Method p-value Significant?
Raw t-test 0.03 Yes — but assumptions violated
Log-transformed t-test 0.22 No
Wilcoxon (non-parametric) 0.09 No

When the result depends on whether you check assumptions, that’s a sign the original test can’t be trusted.

Extension: Sample size and significance

This exercise is optional — for students who finish early.

With enough data, even a trivially small difference becomes “significant.” This exercise demonstrates why effect size matters as much as the p-value. (You’ll learn about effect sizes formally next week.)

Simulate two groups with a tiny true difference (means of 25 vs 25.5) and see how the p-value changes as sample size increases:

TipSolution 
set.seed(7042)
sizes <- c(10, 30, 100, 500, 2000)

for (n in sizes) {
  group_a <- rnorm(n, mean = 25, sd = 5)
  group_b <- rnorm(n, mean = 25.5, sd = 5)
  p <- t.test(group_a, group_b)$p.value
  cat("n =", sprintf("%5d", n), " p =", round(p, 4), "\n")
}

With a small sample, the tiny difference (0.5°C out of 25°C) is undetectable. But with 2000 observations per group, it becomes “highly significant” — even though the effect is negligible for any practical purpose.

This is why a significant p-value is the start of the conversation, not the end. You must always ask: is this a big number?

Now: your project data

The rest of this session is for your group. Using your own project data:

  1. Load and visualise — boxplots, histograms, or scatter plots
  2. Check assumptions — are the data normal? Skewed? Do you need to transform?
  3. Run the test — t.test() (or wilcox.test() if assumptions are violated)
  4. Interpret — what’s the p-value? What’s the confidence interval? What does it mean?
  5. Write it up — one paragraph of results, suitable for your report

If your project has more than two groups, preview: next week you’ll learn ANOVA. For now, pick your two most interesting groups and compare them.

Save your work

Copy your analysis code into your week6.R file. Commit and push via GitHub Desktop. Include both the walkthrough exercises and any project analysis you completed.