Week 5: Designing Your Investigation

Controls, confounders, and sampling

Welcome to Phase 2

In Weeks 1–4, you learned to question, explore, analyse, and communicate — all using the biomass mini-project. Now you’ll design your own investigation. Same skills, your question.

This page accompanies the content session. The exercises below are discussion-based — work with your neighbours.

Exercise 1: Critique HolmesCo’s Site Investigation

Read the HolmesCo ground investigation report below, then answer the questions with your partner.

WarningHolmesCo Ground Investigation Report

See the full report: HolmesCo Site Investigation

Summary: HolmesCo was hired to assess whether a proposed wind farm site in County Durham has suitable ground conditions. They drilled 3 boreholes — all in the valley bottom where access was easy — and concluded that “the bedrock is competent sandstone throughout the site.”

Discuss with your partner:

  1. Where are the boreholes? Where will the turbines actually go?
  2. Is 3 boreholes enough to characterise a whole site?
  3. What kind of sampling bias is this?
  4. What’s missing that would make this a proper investigation?
  • (a) No boreholes on the ridgetops where the turbines are actually planned. The boreholes are in the valley — convenient, but not representative.
  • (b) Three is a very small sample. You can’t characterise geological variability across a site from 3 points.
  • (c) Selection bias / convenience sampling. They sampled where it was easy to get a drill rig, not where the data were needed.
  • (d) No control site for comparison. What does “competent” mean without a reference? Compared to what?

A better investigation would place boreholes on the ridgetops (where the turbines go), use a stratified design across different geological units, and include enough boreholes to capture spatial variability.

Exercise 2: Spot the Confounder

For each scenario below, identify the most likely confounder and suggest how to address it.

Scenario A: > “Communities near wind farms report higher rates of headaches than > communities without wind farms.”

Scenario B: > “Countries with more solar panels have higher GDP per capita.”

Scenario C: > “HolmesCo found that deeper boreholes in County Durham have higher > temperatures. They conclude that drilling deeper always finds hotter > rock.”

A — Wind farms and headaches

Likely confounders: awareness/nocebo effect (people who know they’re near a wind farm report more symptoms), urban vs rural (different baseline health profiles), age demographics (older rural populations), reporting bias (motivated communities may report more).

Addressing it: compare communities that are similar in other respects (matched by demographics, rurality) but differ in wind farm proximity. Ideally, survey before and after a wind farm is built.

B — Solar panels and GDP

Likely confounders: latitude and governance (wealthy, well-governed countries at temperate latitudes both invest in solar and have high GDP), investment climate (both solar deployment and GDP growth respond to stable institutions), reverse causation (rich countries can afford solar, not solar making countries rich).

Addressing it: look at changes within countries over time (does solar growth precede GDP growth?), or control for GDP per capita and governance quality.

C — Borehole depth and temperature

Likely confounder: location. HolmesCo drilled deeper boreholes in the geothermal area (where they expected interesting results) and shallower boreholes elsewhere. The correlation between depth and temperature is confounded by where they chose to drill. Deeper is hotter (geothermal gradient), but the relationship they’re measuring is partly an artefact of their non-random drilling locations.

Addressing it: standardise depth across sites, or control for location in the analysis.

Integrative exercise: Design sketch

Browse the list of summative project topics for 5 minutes. Then, with your provisional group, pick a candidate topic and sketch answers to:

  1. What’s our question? (One sentence.)
  2. What data would we need?
  3. What’s our control/comparison?
  4. What confounders should we worry about?

Each group shares one sentence with the class.

Key points

  • You can’t know if something is big unless you measure something else. Controls are how you isolate what matters.
  • A confounder is correlated with both your treatment and your outcome. If you ignore it, your result might be an artefact.
  • Your sampling strategy is a decision you must justify. “We sampled these because they were there” is honest but weak.
  • Everything HolmesCo does wrong this week, you should do right in your project.