Week 7: Deepening Your Analysis

Effect sizes and what they mean

Introduction

In the content session, you learned ANOVA, post-hoc tests, and why effect sizes matter. Today you’ll put these together in a complete analysis, then apply them to your own project data.

Today’s goals:

  • Calculate and interpret η² (eta-squared) from ANOVA output
  • Calculate Cohen’s d for a pairwise comparison
  • Produce analysis that’s ready for your report
  • Apply these tools to your group project

Exercise 1: ANOVA workflow with effect sizes

Work through a complete analysis of the wind farm data, including effect sizes.

Step 1: Run the ANOVA and extract η² — the proportion of total variance explained by region.

η² = SSbetween / SStotal

You can read these values from the ANOVA summary table.

NoteHint 1

η² is just a ratio: the variation explained by the groups divided by the total variation. ss_total is the sum of both rows in the Sum Sq column.

NoteHint 2 
eta_sq <- ss_between / ss_total
TipSolution 
wind_aov <- aov(capacity_pct ~ region, data = wind)
aov_table <- summary(wind_aov)[[1]]
aov_table

ss_between <- aov_table["region", "Sum Sq"]
ss_total <- sum(aov_table[, "Sum Sq"])
eta_sq <- ss_between / ss_total
eta_sq

η² ≈ 0.31, meaning region explains about 31% of the variation in wind farm capacity factor. By conventional benchmarks (0.01 small, 0.06 medium, 0.14 large), this is a large effect.

This is the kind of number a policy-maker can use: “Where you put your wind farm matters a lot — geography accounts for nearly a third of the difference in output.”

Exercise 2: Cohen’s d for a pairwise comparison

Cohen’s d measures the size of the difference between two specific groups in standard deviation units. Calculate it for Scotland vs Southern England — the largest pairwise difference.

d = (mean₁ − mean₂) / pooled SD

where pooled SD = √[((n₁−1)s₁² + (n₂−1)s₂²) / (n₁ + n₂ − 2)]

NoteHint 1

Fill in the group names (scotland and south) and the pooled SD variable.

NoteHint 2 
cohens_d <- (mean(scotland) - mean(south)) / pooled_sd
TipSolution 
scotland <- wind$capacity_pct[wind$region == "Scotland"]
south <- wind$capacity_pct[wind$region == "Southern England"]

n1 <- length(scotland)
n2 <- length(south)
pooled_sd <- sqrt(
  ((n1 - 1) * sd(scotland)^2 + (n2 - 1) * sd(south)^2) /
    (n1 + n2 - 2)
)

cohens_d <- (mean(scotland) - mean(south)) / pooled_sd
cohens_d

d ≈ 1.5 — a very large effect. Scottish onshore wind farms produce about 1.5 standard deviations more electricity per unit of capacity than Southern English ones.

In practical terms: the mean difference is about 6 percentage points of capacity factor (30% vs 24%). For a 50 MW wind farm operating over a year, that’s roughly 26 GWh vs 21 GWh — a meaningful difference for energy planning.

Extension: η² from scratch

This exercise is optional — for students who finish early.

Calculate η² manually from the group means, group sizes, and overall mean. This helps you understand what the ANOVA is actually computing.

SSbetween = Σ nᵢ (x̄ᵢ − x̄)²

SStotal = Σ (xᵢⱼ − x̄)²

TipSolution 
grand_mean <- mean(wind$capacity_pct)

ss_between_manual <- 0
for (r in unique(wind$region)) {
  group_vals <- wind$capacity_pct[wind$region == r]
  ss_between_manual <- ss_between_manual + 
    length(group_vals) * (mean(group_vals) - grand_mean)^2
}

ss_total_manual <- sum((wind$capacity_pct - grand_mean)^2)

eta_sq_manual <- ss_between_manual / ss_total_manual
cat("Manual calculation: eta-squared =", round(eta_sq_manual, 3), "\n")

You should get the same η² ≈ 0.31 as from the ANOVA table. This confirms that η² is simply the fraction of total variation that comes from differences between groups.

Now: your project data

The rest of this session is for your group. Today’s goals:

  1. Run your main statistical test — t-test (if 2 groups) or ANOVA (if 3+ groups)
  2. Check assumptions — histograms, Shapiro-Wilk, transform if needed
  3. Calculate an effect size — Cohen’s d (for t-test) or η² (for ANOVA)
  4. Write a results paragraph — suitable for your report

If you’re using ANOVA and it’s significant, run TukeyHSD() to find which pairs differ.

Questions to answer as you work:

  • Is your result significant? At what level?
  • Is the effect big enough to matter? (Is that a big number?)
  • What’s the confidence interval? Does it include values that would change your conclusion?
  • What assumptions did you check? Were any violated?

Save your work

Commit your analysis code and your results paragraph to your group repo via a pull request. If you’ve started your individual report, commit that to your personal repo too.