Week 7: Comparing Groups

ANOVA and the multiple comparisons problem

Introduction

Last week you learned to compare two groups with a t-test. But what if you have three, four, or more groups? You could run lots of pairwise t-tests — but as the content session just showed you, that inflates your false positive rate. With 4 groups, you’d need 6 t-tests, and you’d have a 26% chance of at least one false positive even if there’s no real difference.

ANOVA solves this by testing whether any group differs from the others in a single test. If it’s significant, post-hoc tests (like TukeyHSD) tell you which pairs differ — with correction for multiple comparisons.

In these exercises, you’ll analyse wind farm capacity factors across 4 UK regions and discover whether geography really matters for wind energy output.

Goals:

Visualise multi-group data with boxplots
Run and interpret an ANOVA
Use TukeyHSD to find which groups differ
Understand what the multiple comparisons correction buys you

Exercise 1: Boxplots by region

Start by visualising the data. Make boxplots of capacity factor by region. Which regions look different? Which look similar?

Solution

ggplot(wind, aes(x = region, y = capacity_pct)) +
  geom_boxplot()

The boxplots suggest two clusters:

Higher capacity: Scotland and Northern England (medians around 29–30%)
Lower capacity: Southern England and Wales (medians around 23–25%)

This makes geographical sense — northern and western UK sites tend to be windier. But are these differences statistically significant, or could they be due to sampling variation?

Exercise 2: Run the ANOVA

Use aov() to test whether capacity factor differs across regions. Save the ANOVA result so you can use it for post-hoc tests.

Solution

wind_aov <- aov(capacity_pct ~ region, data = wind)
summary(wind_aov)

The output shows:

F value ≈ 14 — the between-group variation is about 14 times larger than the within-group variation.
p < 0.001 — very strong evidence that at least one region differs from the others.

But ANOVA only tells you something differs. It doesn’t say which regions differ from which. For that, we need post-hoc tests.

Exercise 3: Which pairs differ?

Run TukeyHSD() on your ANOVA result. Look at the p adj column — which pairs of regions have significantly different capacity factors?

Solution

tukey_result <- TukeyHSD(wind_aov)
tukey_result

The pattern:

Comparison	Significant?	Interpretation
Scotland vs Northern England	No (p ≈ 0.99)	Similar — both windy
Southern England vs Scotland	Yes (p < 0.001)	Scotland produces more
Wales vs Scotland	Yes (p ≈ 0.001)	Scotland produces more
Southern England vs Northern England	Yes (p < 0.001)	North produces more
Wales vs Northern England	Yes (p < 0.001)	North produces more
Wales vs Southern England	No (p ≈ 0.67)	Similar — both less windy

Two clusters: the northern regions (Scotland, Northern England) have significantly higher capacity factors than the southern regions (Southern England, Wales). Within each cluster, the regions don’t differ significantly.

The “p adj” means these p-values are already corrected for the fact that you’re making 6 comparisons simultaneously. Without the correction, you’d have a 26% chance of a false positive; TukeyHSD keeps the family-wise error rate at 5%.

Quick check

With 4 groups, how many pairwise comparisons are there?

Extension: Visualise the TukeyHSD confidence intervals

This exercise is optional — for students who finish early.

R can plot TukeyHSD results directly — it shows the confidence interval for the mean difference for each pair. Intervals that cross zero mean no significant difference.

Solution

plot(tukey_result)

The plot shows 6 horizontal confidence intervals. Intervals that cross the dashed vertical line at zero indicate non-significant differences. You should see that Scotland–Northern England and Wales–Southern England cross zero (consistent with the p-values above), while the other four comparisons don’t.

This is a useful figure for a report — it shows both the size and the uncertainty of each pairwise difference at a glance.

Save your work

Copy the code you are most proud of into your week7.R file. Commit and push via GitHub Desktop. Write a commit message that describes what you did — not just “week 7”.