Session Plans: Weeks 5–10
Phase 2 — Summative Group Projects
Weeks 5–10: Session-by-Session Plans
Phase 2: summative group projects. Groups choose a sustainable energy question, design an investigation, analyse data, and each write an individual 4-page policy report.
These plans follow the generic templates in Session templates.md. Each session is 110 minutes. Timings are guidelines, not scripts.
Conventions: - 🎓 = instructor-led segment - ✏️ = student activity - 💻 = live coding or WebR exercise - 💬 = discussion or think-pair-share - 📋 = logistics or admin
Recurring character: HolmesCo
Several exercises use scenarios from HolmesCo, a fictional geological consultancy with an optimistic attitude to evidence and a flexible relationship with rigour. (Named in fond tribute to Arthur Holmes of Durham — who deserves better.) HolmesCo’s reports provide a consistent foil for students to evaluate: their claims are always technically sourced but frequently undermined by poor experimental design, base rate neglect, multiple testing, or overconfident extrapolation. Students should develop the reflex of asking “what has HolmesCo not thought of?”
Week 5 Content Session: “Designing your investigation”
Aims: Teach the principles of experimental design: controls, confounding, replication, and sampling strategy. Students should leave able to critique a bad design and sketch a good one. This is the conceptual foundation for their summative project.
Session plan
| Time | Type | Activity |
|---|---|---|
| 0–5 | 📋 | Welcome to Phase 2. Recap the arc: in Weeks 1–4 you learned to question, explore, analyse, and communicate using a scaffolded project. Now you design your own investigation. Same skills, your question. Briefly outline the summative assessment: individual 4-page policy report (30%), based on group project work. |
| 5–25 | 🎓 | Concept block 1: What makes a good experiment? The logic of comparison: you can’t know if something is big unless you measure something else. Controls: what you hold constant so you can isolate the variable of interest. Treatment vs control groups. Geoscience angle: in field sciences we rarely get to run true experiments — we observe natural variation and try to isolate factors. This makes design more important, not less. Example: comparing wind farm output across sites. What should you control for? (Wind speed, turbine model, terrain, grid curtailment.) What can’t you control? (Weather, maintenance schedules.) |
| 25–40 | 💬✏️ | Exercise 1: Critique HolmesCo’s site investigation. Hand out a one-page summary of a HolmesCo ground investigation report. The scenario: HolmesCo has been hired to assess whether a proposed wind farm site in County Durham has suitable ground conditions.They drilled 3 boreholes — all in the valley where access was easy — and concluded that the bedrock is competent sandstone throughout the site. Students work in pairs to identify the problems: (a) no boreholes on the ridgetops whereturbines are actually planned;; (b) three is a very small sample; (c) selection bias — they sampled where it was convenient, not where it was representative; (d) no control site for comparison. Discuss: what would a better investigation look like? |
| 40–60 | 🎓 | Concept block 2: Confounding variables. A confounding variable is correlated with both the treatment and the outcome, so you can’t tell which caused the effect. Classic example: ice cream sales and drowning rates (both caused by hot weather). Energy example: countries with more solar panels also have higher GDP — does solar power cause wealth? (No: both correlate with latitude, governance, and investment climate.) Geological example: deeper boreholes tend to find higher temperatures — but depth also correlates with location (HolmesCo drilled deeper in the geothermal area, shallower elsewhere). How to handle confounders: randomisation (when possible), stratification, matching, and — when all else fails — acknowledging them honestly. |
| 60–75 | 💬✏️ | Exercise 2: Spot the confounder. Three short scenarios (energy/geoscience themed). For each, students identify the likely confounder and suggest how to address it. Example: “HolmesCo reports that communities near wind farms have higher rates of reported headaches than communities without wind farms.” (Confounder candidates: rural vs urban, age demographics, awareness/nocebo effect, reporting bias.) |
| ☕ Possible 5-minute break here | ||
| 75–95 | 🎓 | Concept block 3: Sampling strategies. How to choose your sample. Random sampling: every unit has equal probability — the gold standard but often impractical in geoscience. Stratified sampling: divide the population into subgroups, sample within each. (e.g., sample boreholes from each geological formation, not just the accessible one.) Systematic sampling: every nth unit (e.g., regular grid of monitoring stations). Convenience sampling: whatever’s easiest (HolmesCo’s default). When is each appropriate? Key message: your sampling strategy is a decision you must justify in your report. “We sampled these because they were there” is honest but weak. |
| 95–105 | ✏️💬 | Integrative exercise: Design sketch. Show the list of available summative project topics (curated sustainable energy questions with data sources). Students browse for 5 minutes, then form provisional groups (3–4 people). Each group picks a candidate topic and spends 5 minutes sketching: what’s our question? What data would we need? What’s our control/comparison? What confounders should we worry about? Groups share one sentence each. |
| 105–110 | 📋 | Wrap-up. Application session: you’ll finalise your group, choose your topic, and write a one-page research plan. You’ll also set up your group’s GitHub repo. |
Notes
- HolmesCo’s site investigation should be written as a plausible (if brief) consultancy report — complete with a logo, “Prepared for” header, and a confident conclusion. The humour comes from the gap between the confidence and the evidence. Keep it affectionate.
- The topic list for summative projects needs to be curated in advance: each topic should have accessible public data, a genuine policy dimension, and enough complexity for a 4-page report. 8–12 options is about right for groups of 3–4 from a cohort of 100+.
- Group formation: allow self-selection but provide a mechanism for students without a group (e.g., sign-up sheet for “free agents”). Aim for groups of 3–4.
Week 5 Application Session: “Your question, your plan”
Aims: Groups finalise their topic and write a structured one-page research plan. Set up collaborative Git workflow.
Session plan
| Time | Type | Activity |
|---|---|---|
| 0–10 | 📋 | Setup. Finalise groups. Any students without a group join one now (instructor facilitates). Distribute the full topic list with data source descriptions.By the end of today: a one-page research plan committed to your group repo.. |
| 10–25 | 🎓💻 | Git collaboration intro. Instructor demonstrates on screen: (1) Creating a shared repo in the course GitHub Organization (or show the pre-created one via GitHub Classroom). (2) Each group member clones it. (3) Creating a branch: “I’m going to work on the data exploration, so I’ll create a branch called explore-data.” (4) Making a change, committing, pushing the branch, opening a Pull Request on GitHub. (5) A teammate reviews and merges. Emphasise: branches keep your work separate until you’re ready to combine. PRs let your teammates review before merging. We’ll use this workflow for the rest of the module. |
| 25–40 | ✏️📋 | Groups set up repos. Each group: clones their repo, each member creates a branch, makes a trivial commit (e.g., adds their name to a team.md file), pushes, and opens a PR. The group merges all PRs. This is a dry run of the workflow they’ll use for real work. Demonstrators circulate. |
| 40–80 | ✏️💬 | Research plan drafting. Groups work together to complete a structured research plan template (provided as a markdown file in the repo): (1) Research question: What are you investigating? (One sentence.) (2) Hypothesis: What do you expect to find, and why? (3) Data sources: Where will your data come from? What variables? What time period? (4) Comparison/control: What are you comparing to? How will you isolate the effect of interest? (5) Potential confounders: What alternative explanations should you worry about? (6) Analysis plan: What statistical tools do you anticipate using? (descriptive stats, t-tests, ANOVA, regression — link to what they’ve learned and what’s coming). (7) Division of labour: Who’s doing what? Instructor and demonstrators circulate, challenge groups on their designs (“What’s your control?” “How will you handle this confounder?” “Is your sample representative?”). |
| 80–95 | 💬 | Plan presentations. Each group gives a 60-second pitch: question, hypothesis, data source, comparison. Class and instructor give brief feedback. Focus on: is the question answerable with available data? Is there a clear comparison? |
| 95–105 | 📋 | Commit and wrap-up. Groups commit their research plan via a PR (one member writes, others review and merge). Preview Week 6: “Next week you’ll learn the formal tools for testing hypotheses — and you’ll start applying them to your own data.” |
| 105–110 | 📋 | Buffer. |
Notes
- The research plan is a checkpoint, not an exam. It should be completed during the session; the instructor provides formative feedback verbally. Groups who need to revise can update it later.
- Git collaboration will be messy. Expect merge conflicts, confusion about branches, and at least one group who pushes to main by accident. This is fine — it’s the learning. Brief “what to do when Git complains” guidance should be available (one page, in the repo or on the course site).
- Topics should be finalised by the end of this session. Groups who can’t decide should be gently pushed — indecision costs more than a suboptimal choice.
Week 6 Content Session: “Testing hypotheses”
Aims: Formalise hypothesis testing: t-tests, p-values, assumptions, and interpretation. Then show the limits of the frequentist approach via the base rate fallacy and Bayesian conditional thinking. Students should leave knowing how to run and interpret a t-test and knowing why a significant p-value doesn’t necessarily mean what they think it means.
Session plan
| Time | Type | Activity |
|---|---|---|
| 0–5 | 📋 | Recap. Week 3 introduced the logic of hypothesis testing: null vs alternative, Type I and Type II errors, “how much evidence is enough?” This week: the formal machinery. We’ll learn how to run a t-test in R — and then we’ll learn why it can mislead you. |
| 5–30 | 🎓💻 | Concept block 1: The t-test. What it does: compares two group means and asks whether the difference is larger than you’d expect by chance. Walk through the logic: (1) assume no difference (null hypothesis); (2) calculate how far apart the observed means are, relative to the variability in the data (the t-statistic); (3) ask: if there were really no difference, how often would we see a t-statistic this extreme? That probability is the p-value. Live demo in R. Use a prepared dataset — could be emissions data from two types of power plant, or geological measurements from two formations. Show: t.test(x ~ group, data = df). Read the output: t-statistic, degrees of freedom, p-value, confidence interval. Interpret: “The mean difference is X, with a 95% CI of [Y, Z]. The p-value is 0.03, meaning we’d see a difference this large about 3% of the time if there were truly no difference.” |
| 30–40 | 💬✏️ | Exercise 1: Interpret a t-test. Give students R output from a t-test (on screen, not run live). Ask them to write one sentence interpreting the result for a policy audience. Share and compare. Common mistakes to watch for: “the p-value means there’s a 3% chance the null is true” (wrong — this is exactly the confusion we’ll address in block 3). |
| 40–60 | 🎓 | Concept block 2: Assumptions and when tests go wrong. Every statistical test rests on assumptions. For the t-test: (1) Independence: observations in one group don’t affect the other. (2) Normality: the data within each group are approximately normally distributed (or the sample is large enough for CLT to help). (3) Equal variance (for Student’s t; Welch’s t relaxes this). How to check: histograms, QQ plots, Levene’s test (briefly). What happens when assumptions are violated? The test can give misleading p-values. Geological example: HolmesCo measured soil permeability at 10 sites in each of two formations. The data are log-normally distributed (as permeability always is). They ran a t-test on the raw values and got p = 0.04. But on log-transformed values, p = 0.23. The “significant” result was an artefact of skewness. |
| 60–70 | ✏️💻 | Exercise 2: Check your assumptions. Students open WebR. Given a dataset, they: (1) make histograms of each group, (2) decide whether to transform, (3) run t.test() before and after transformation, (4) compare. Brief discussion: did the conclusion change? |
| ☕ Possible 5-minute break here | ||
| 70–95 | 🎓💬 | Concept block 3: “But what does the p-value actually mean?” — Conditional probability and the base rate fallacy. This is the conceptual centrepiece. Frame: “I’m going to show you why a statistically significant result can still be wrong most of the time.” Warm-up: Monty Hall. “Some of you simulated the Monty Hall problem in first-year Python — you wrote code that showed switching wins 2/3 of the time. But most people’s gut says it’s 50/50. The reason your gut is wrong is that it ignores the information the host gives you by opening a door — it confuses P(win) with P(win given what you now know). That confusion between unconditional and conditional probability is exactly what this block is about — and it has serious consequences for how we interpret statistical tests.” The medical test. Start with the classic: a disease affects 1 in 1,000 people. A test is 99% accurate (99% sensitivity, 99% specificity). You test positive. What’s the probability you have the disease? Poll the class. Most will say 99%. Work through the arithmetic: out of 100,000 people, 100 have the disease (99 test positive), 99,900 don’t have it (999 test positive by mistake). Total positives: 99 + 999 = 1,098. Of those, only 99 actually have the disease. P(disease given positive) = 99/1,098 ≈ 9%, not 99%. Let this land. Then ask: why is it so low? Because the disease is rare. The base rate matters enormously. |
| The geological version: HolmesCo strikes gold. “HolmesCo is prospecting for gold in County Durham. They’ve developed a geochemical assay that detects gold-bearing rock with 95% accuracy: if gold is present, the assay says ‘positive’ 95% of the time; if gold is absent, the assay says ‘negative’ 95% of the time. They test 10,000 samples from across the county. 50 samples test positive. HolmesCo’s press release: ‘Major gold discovery in County Durham — 50 confirmed hits!’” Work through the numbers. Suppose only 5 in 10,000 samples are actually gold-bearing (a generous base rate for the Northern Pennines). Expected true positives: 5 × 0.95 ≈ 5. Expected false positives: 9,995 × 0.05 ≈ 500. So roughly 505 positives, of which only 5 are real. HolmesCo’s 50 positives? Almost certainly all false. The press release is nonsense. | ||
| Connection to p-values. This is exactly the problem with p-values in research. A p-value tells you P(data | H₀) — the probability of seeing this result if the null is true. But what you actually want is P(H₀ | data) — the probability that the null is true given your result. These are not the same thing, and the difference depends on the base rate — how likely H₀ was to be false before you looked at the data. If you’re testing an implausible hypothesis (rare disease, gold in Durham), a “significant” p-value is probably a false positive. If you’re testing something already well-supported by prior evidence, a significant result is much more convincing. This is, informally, the Bayesian insight: your prior beliefs matter. You don’t need to teach Bayes’ theorem formally — the intuition is what counts. | ||
| 💬 | Brief discussion. “When HolmesCo finds a significant result, what’s the first question you should ask?” (How plausible was the hypothesis before the test?) “When you run a t-test in your project, what’s your prior expectation? Is the effect you’re looking for plausible or surprising?” | |
| 95–105 | ✏️ | Integrative exercise: Plan your test. Groups briefly discuss: for your project, what comparison will you test? What’s your null hypothesis? What’s your prior expectation — is the effect plausible or a long shot? What assumptions should you check? Write 3–4 sentences. This feeds directly into the application session. |
| 105–110 | 📋 | Wrap-up. “In the application session, you’ll run your first real tests on your project data. Remember: a significant p-value is the start of the conversation, not the end.” |
Notes
- The base rate segment is the most important 25 minutes of the course. Take your time. Let the numbers sink in. The moment when students realise that HolmesCo’s 50 “confirmed hits” are almost certainly all false positives is the Week 6 equivalent of the traitor reveal.
- The medical test example is the intuition-builder; the HolmesCo example is the one that sticks (because it’s funny and geological). Use both.
- Do not introduce Bayes’ theorem as a formula. The goal is conditional thinking — the habit of asking “how plausible was this before I tested it?” Students who want the formal framework can explore it independently; for this course, the intuition is sufficient.
- The connection to p-values should be made explicit but not belaboured. The key sentence: “A p-value tells you how surprising the data are if the null is true; it does not tell you how likely the null is given the data. The difference depends on the base rate.”
- This also sets up Week 7’s multiple comparisons problem: if you run 20 tests, your base rate for any individual hypothesis being true is low, so false positives accumulate.
Week 6 Application Session: “First tests”
Aims: Groups apply hypothesis testing to their project data. Run and interpret t-tests in R. Check assumptions. Begin building the analytical core of their reports.
Session plan
| Time | Type | Activity |
|---|---|---|
| 0–10 | 📋 | Setup. Pull latest changes from group repo. Recap: you have a research plan and a hypothesis. Today you test it. |
| 10–30 | 🎓💻 | Guided walkthrough: t-test workflow in R. Instructor demonstrates a complete workflow on example data (not a group’s project data — something neutral). (1) Load data. (2) Visualise: histograms of each group, boxplots side by side. (3) Check assumptions: shapiro.test() for normality, visual check via QQ plot. (4) Run the test: t.test(y ~ group, data = df). (5) Interpret: read the output, extract the confidence interval, state the conclusion in plain English. (6) If assumptions violated: log-transform and re-test, or use wilcox.test() as a non-parametric alternative. Emphasise: always visualise before you test. |
| 30–80 | ✏️💻 | Group work: Apply to your data. Groups work on their project data. Instructor and demonstrators circulate. Suggested workflow: (1) Load your data. Make sure it’s clean. (2) Define your groups/comparison. (3) Visualise. What does it look like? (4) Check assumptions. (5) Run the t-test. (6) Interpret. (7) Write up a paragraph of results. Groups who finish early: try a second comparison, or explore subgroup analyses. Groups who are stuck on data cleaning: demonstrators prioritise getting them to a testable state. |
| 80–95 | 💬 | Discussion and synthesis. Ask 2–3 groups to share their result. For each: what was the hypothesis? What was the p-value? What was the effect size? Do you believe the result? Why or why not? Instructor models the conditional-thinking habit: “How plausible was this result before you tested? Does the p-value change your mind a little or a lot?” |
| 95–105 | 📋 | Commit and wrap-up. Commit analysis code and written results to group repo via PR. “Next week: what if you have more than two groups to compare?” |
| 105–110 | 📋 | Buffer. |
Notes
- Many groups will spend much of this session on data cleaning and wrangling, not on the t-test itself. This is realistic — and worth saying aloud: “In real research, 80% of the time is data preparation.”
- If a group’s data don’t suit a t-test (e.g., they have more than two groups, or their variable is categorical), help them identify the right approach and preview Week 7. Don’t force a t-test where it doesn’t fit.
- Encourage groups to commit intermediate work (even rough code) rather than waiting for a polished result. The commit history is part of the learning.
Week 7 Content Session: “Comparing groups”
Aims: Extend hypothesis testing to multiple groups (ANOVA). Introduce the multiple comparisons problem and why it matters. Teach effect sizes as a complement to p-values. Thread: building on Week 6’s base rate insight, show that testing many hypotheses inflates the false positive rate.
Session plan
| Time | Type | Activity |
|---|---|---|
| 0–5 | 📋 | Recap. Last week: t-tests for two groups; the base rate fallacy (HolmesCo’s gold). This week: what happens when you have more than two groups — and more than one test. |
| 5–25 | 🎓 | Concept block 1: Why not just do lots of t-tests? If you have three groups (e.g., energy output from wind farms in three regions), you could run three pairwise t-tests (A vs B, A vs C, B vs C). What’s wrong with that? Each test has a 5% false positive rate. Three tests: the probability of at least one false positive is 1 − 0.95³ ≈ 14%. Ten tests: 40%. Twenty tests: 64%.HolmesCo’s mineral survey. “HolmesCo tested soil samples from 20 sites across Northern England for enrichment in 20 critical minerals — lithium, cobalt, rare earths, and so on. They found statistically significant enrichment (p < 0.05) for three minerals at one site. Press release: ‘Critical mineral discovery in Northern England!’” Ask the class: if there’s truly nothing there, how many significant results would you expect by chance? (20 minerals × 20 sites = 400 tests × 0.05 = 20 expected false positives. Three hits is actually fewer than expected by chance.)) This is the multiple comparisons problem. |
| 25–35 | 💬 | Exercise 1: How many tests? Quick poll / discussion. “In your project, how many comparisons are you planning to make?” Students count. Then: “If you run all of them at α = 0.05, what’s your family-wise false positive rate?” Work through the formula briefly. The point: if you’re testing more than one thing, you need to account for it. |
| 35–55 | 🎓💻 | Concept block 2: ANOVA. The solution for comparing multiple groups simultaneously. One-way ANOVA: tests whether any group differs from the others, without specifying which. Walk through the logic: (1) total variation in the data = variation between groups + variation within groups; (2) if the between-group variation is large relative to within-group, at least one group is different; (3) the F-statistic captures this ratio; (4) the p-value tells you how surprising this F is under the null. Live demo in R: aov(output ~ region, data = wind_df) |> summary(). Read the output: F-statistic, p-value. If significant: post-hoc tests to find out which groups differ. TukeyHSD() with correction for multiple comparisons. Brief mention: Bonferroni correction (simple, conservative), Tukey’s HSD (better for pairwise comparisons after ANOVA). |
| 55–70 | ✏️💻 | Exercise 2: Run an ANOVA. Students in WebR. Provided dataset: energy output from wind farms in four regions (or similar). (1) Visualise with boxplots. (2) Run aov(). (3) Is it significant? (4) Run TukeyHSD(). Which pairs differ? (5) Write a one-sentence conclusion. |
| ☕ Possible 5-minute break here | ||
| 70–90 | 🎓 | Concept block 3: Effect sizes — “significant” is not “important.” A p-value tells you whether an effect is detectable; it doesn’t tell you whether it’s big enough to matter. With a large enough sample, trivially small differences become “significant.” Example: HolmesCo measures the temperature in two boreholes, 1,000 readings each. Mean difference: 0.02°C. p-value: 0.001. “Highly significant!” But is 0.02°C meaningful for any practical purpose? Effect size measures: Cohen’s d (standardised mean difference): d = 0.2 (small), 0.5 (medium), 0.8 (large). Show how to calculate and report it. R: effectsize::cohens_d(y ~ group, data = df) or manual calculation. For ANOVA: η² (eta-squared) — proportion of variance explained. Key message: always report both the p-value and an effect size or confidence interval. A policy-maker needs to know “how big?” not just “is it real?” Link to “Is that a big number?” from Week 2 — same habit, formalised. |
| 90–105 | ✏️💬 | Integrative exercise: HolmesCo’s report card. Hand out a short HolmesCo “technical summary” that reports three hypothesis tests. One is significant with a large effect; one is significant with a tiny effect; one is non-significant but has a confidence interval that nearly excludes zero. For each, students write: “Should we act on this? Why or why not?” Discuss. |
| 105–110 | 📋 | Wrap-up. “In the application session, apply these tools to your project. Remember: significance is the start of the conversation. Effect size tells you whether it matters.” |
Notes
- Not all groups will need ANOVA. Groups with only two-group comparisons should deepen their t-test interpretation (effect sizes, confidence intervals, assumption checks). The application session should accommodate both tracks.
- HolmesCo’s mineral survey is a memorable example because the “three significant minerals” actually underperform chance. This subverts the expectation that significance = discovery and makes the multiple comparisons problem viscerally clear. The critical minerals framing adds an extra lesson: policy urgency can make dodgy results feel more plausible.
- The effect size segment connects back to Week 2’s “Is that a big number?” This should be made explicit — it’s the statistical formalisation of the same instinct.
Week 7 Application Session: “Deepening your analysis”
Aims: Groups continue analysis. Those who need ANOVA apply it; others refine their t-test analyses with effect sizes, confidence intervals, and assumption checks. All groups should be producing analysis that could appear in their reports.
Session plan
| Time | Type | Activity |
|---|---|---|
| 0–10 | 📋 | Setup. Pull latest changes. Today is flexible: work on whatever your project needs most. Two tracks available — ANOVA (for groups with 3+ categories) and deepening t-test analyses (for groups with two-group comparisons). Both tracks include effect sizes and assumption checking. |
| 10–25 | 🎓💻 | Brief guided example. Instructor works through one complete analysis on example data: (1) ANOVA → significant → TukeyHSD → report effect sizes. OR (2) t-test → check assumptions → log-transform → re-test → report Cohen’s d and CI. Keep it short — students need maximum work time today. |
| 25–80 | ✏️💻 | Group work. Groups continue their project analysis. Suggested goals for today: (1) Run and interpret your main statistical test (t-test or ANOVA). (2) Check assumptions (histograms, QQ plots, transformation if needed). (3) Calculate and report an effect size. (4) Write a “Results” paragraph suitable for your report. (5) If time: run a secondary analysis or explore a subgroup. Instructor and demonstrators circulate. Priority: help groups who are stuck; challenge groups who are coasting (“What’s your effect size? What’s the confidence interval? Would you make a policy recommendation based on this?”). |
| 80–95 | 💬 | Cross-group check-in. Each group gives a 30-second update: what they tested, what they found, and what they’re worried about. Instructor provides brief feedback. Highlight common issues across groups (e.g., “Several of you have skewed data — remember to check and transform”). |
| 95–105 | 📋 | Commit and wrap-up. Commit analysis code and results. “Next week: what is a model, and what can go wrong when you build one?” |
| 105–110 | 📋 | Buffer. |
Notes
- This session should feel like a supervised lab, not a structured exercise. Groups have different needs and should work at their own pace. The instructor’s role is coaching, not lecturing.
- Groups who are behind on data cleaning should be given focused help. By the end of this session, every group should have run at least one statistical test.
- Groups who are ahead can start producing figures for their reports or exploring additional questions in their data.
Week 8 Content Session: “Models and their limits”
Aims: What is a model? Linear regression in R. Model assumptions and diagnostics. Then the critical message: models are simplifications, and their limits — overfitting, extrapolation failure, and blind spots — are as important as their predictions.
Session plan
| Time | Type | Activity |
|---|---|---|
| 0–5 | 📋 | Recap. You can now test hypotheses, compare groups, and quantify effect sizes. This week: models. A model is a simplified story about how the world works. The question is always: how much of the story does it capture, and what does it leave out? |
| 5–25 | 🎓💻 | Concept block 1: What is a model? And: linear regression. A model is a deliberate simplification that captures some features of reality and ignores others. Every equation in science is a model. Every statistical test you’ve run contains an implicit model (the t-test models the data as two normal distributions with equal variance). Linear regression makes the model explicit: y = β₀ + β₁x + ε. “y increases by β₁ for every one-unit increase in x, plus noise.” Live demo in R: lm(output ~ windspeed, data = turbine_df) |> summary(). Read the output: coefficients, R², p-value for each term. Plot: ggplot(turbine_df, aes(windspeed, output)) + geom_point() + geom_smooth(method = "lm"). Interpret: “Wind speed explains about 60% of the variation in output. For each 1 m/s increase in wind speed, output increases by ~X kWh.” |
| 25–40 | ✏️💻 | Exercise 1: Fit and interpret. Students in WebR. Provided dataset: solar panel output vs temperature (counterintuitive — output decreases at high temperatures for crystalline silicon panels). (1) Make a scatter plot. (2) Fit lm(). (3) Interpret the slope and R². (4) Plot the fit. Is the relationship linear? Does the model capture the pattern? |
| 40–60 | 🎓 | Concept block 2: Assumptions and diagnostics. Linear regression assumes: (1) Linearity — the relationship is actually linear. (2) Independence of residuals. (3) Homoscedasticity — constant variance of residuals. (4) Normality of residuals (for inference, not prediction). How to check: plot(model) gives four diagnostic plots. Walk through each. Most important: residuals vs fitted (linearity + homoscedasticity) and QQ plot (normality). What to do when assumptions are violated: transform, add terms, or use a different model. |
| 60–75 | ✏️💬 | Exercise 2: HolmesCo’s regression. HolmesCo has fitted a linear model to predict groundwater level from monthly rainfall. The R² is 0.45, the p-value is very small, and HolmesCo concludes: “Rainfall is the dominant control on groundwater levels.” Show the diagnostic plots: there’s a clear seasonal pattern in the residuals (the relationship isn’t just linear — there’s a time-lag and seasonal cycle the model isn’t capturing). Students identify the problem. Discussion: what does R² = 0.45 actually mean? (55% of the variation is unexplained. “Dominant” is a stretch.) |
| ☕ Possible 5-minute break here | ||
| 75–100 | 🎓💬 | Concept block 3: The three failures of models. (1) Overfitting: a model that fits the training data perfectly but fails on new data. It’s memorised the noise. Demo: fit a polynomial of degree 15 to 20 data points — it goes through every point but oscillates wildly between them. Compare to the straight line that fits worse but predicts better. Lesson: more complex ≠ more useful. (2) Extrapolation failure: a model that works within the observed range but breaks down outside it. Example: UK solar capacity has grown roughly exponentially since 2010. Extrapolate to 2050 and you’d predict solar capacity exceeding the UK’s entire electricity grid. What’s the model not capturing? (Planning constraints, grid integration limits, diminishing returns on suitable sites.) Another: HolmesCo extrapolated a linear decline in a quarry’s groundwater level and predicted the aquifer would be empty by 2028. They didn’t account for seasonal recharge or the fact that the decline was caused by a temporary pumping phase. (3) Blind spots: things the model structurally cannot represent. A single-turbine power curve model has no term for wake interference from neighbouring turbines, so it overpredicts a wind farm’s output by 10–20%. A carbon accounting model that counts biomass combustion as zero structurally cannot see the emissions it’s ignoring. Lesson: always ask “what is my model not capturing?” |
| 100–105 | ✏️ | Quick reflection. Students write one sentence: “In my project, the biggest thing my analysis might be missing is…” |
| 105–110 | 📋 | Wrap-up. “In the application session, you’ll fit models to your project data and deliberately try to break them.” |
Notes
- The overfitting demo (polynomial degree 15) is visually memorable and should be done live in R. Use
geom_smooth(method = "lm", formula = y ~ poly(x, 15))vsy ~ xon the same plot. - The three failures are the intellectual core of this session and should feel like a progression: overfitting (too much faith in your data), extrapolation (too much faith in your model), blind spots (too much faith in your framework). Each is a different form of hubris.
- HolmesCo appears twice this week (regression + extrapolation). Both scenarios should feel plausible — the kind of thing a real consultant might do under time pressure.
- The “What is my model not capturing?” question should become a reflex, like “Is that a big number?” from Week 2.
Week 8 Application Session: “Fitting and breaking models”
Aims: Groups apply regression to their project data where appropriate. All groups attempt an extrapolation exercise to experience model failure first-hand. Review and refine earlier analyses.
Session plan
| Time | Type | Activity |
|---|---|---|
| 0–10 | 📋 | Setup. Pull latest changes. Today has two parts: (1) apply regression to your project if it’s relevant; (2) a structured extrapolation exercise that everyone does. |
| 10–25 | 🎓💻 | Brief guided example. Instructor demonstrates fitting lm() to project-relevant data, checking diagnostics, and interpreting results. Keep it brief — students have seen this in the content session. |
| 25–55 | ✏️💻 | Group work: Regression on your data. Groups who have a regression question work on it. Groups whose projects are better suited to group comparisons (t-test/ANOVA) should instead refine their existing analyses: add effect sizes, improve figures, check and document assumptions. All groups: write up a “Methods and Results” section draft. |
| 55–80 | ✏️💻 | Extrapolation exercise (all groups). Provided dataset: UK renewable energy capacity (wind + solar) from 2010–2024. (1) Fit a linear model to the trend. Plot the fit. (2) Extrapolate to 2040. What does the model predict? Is this plausible? (3) Now fit an exponential model (lm(log(capacity) ~ year)). Extrapolate to 2040. Better or worse? (4) What’s the model not capturing? (Students should identify: planning constraints, grid limits, saturation effects, policy changes.) (5) Extension: look up actual 2025 data. How does it compare to both extrapolations? Discussion: both models “fit” the historical data well. Neither predicts the future correctly. What does this tell you about modelling? |
| 80–95 | 💬 | Discussion. Share extrapolation results. Key message: a model’s quality in-sample tells you nothing about its quality out-of-sample. “What is my model not capturing?” is the question that separates a good analyst from HolmesCo. Groups briefly share: “In our project, the biggest thing our analysis might be missing is…” |
| 95–105 | 📋 | Commit and wrap-up. Commit all work. “Your analysis is now nearly complete. Over the next two weeks, you’ll refine it, write it up, and have it reviewed.” |
| 105–110 | 📋 | Buffer. |
Notes
- The extrapolation exercise is designed to be humbling. Both the linear and exponential models produce absurd 2040 predictions, but for different reasons. The discussion should surface the idea that all models are wrong, but some are useful — and you need to know where the usefulness ends.
- Groups at this stage should have substantive analysis to show. Instructor should check on each group’s progress and flag any who are falling behind.
Week 9 Content Session: “AI, reproducibility, and integrity”
Aims: Teach students to use AI tools effectively in research. Understand what AI does well and badly. Connect to reproducibility and scientific integrity. This session is more discursive than technical — more debate and reflection, less coding.
Session plan
| Time | Type | Activity |
|---|---|---|
| 0–5 | 📋 | Recap. Your projects are nearly done. You’ve designed investigations, run tests, fitted models, and identified limitations. This week: two forces that shape modern research — AI and the pressure to produce results. |
| 5–25 | 🎓💬 | Concept block 1: What AI does well, and what it does badly. AI is good at: generating first-draft code, summarising text, suggesting structure, spotting syntax errors, explaining error messages. AI is bad at: knowing whether its answer is correct, understanding your specific data, making judgement calls about methodology, citing sources reliably, distinguishing a meaningful result from a spurious one. Live demo (if feasible): give an AI assistant the HolmesCo gold assay scenario from Week 6. Ask it to interpret the results. See what it does with the base rate problem. (It will likely get it wrong, or give a generically correct answer without applying it to the specific numbers.) Discussion: the AI is fluent but not thoughtful. |
| 25–40 | 💬 | Exercise 1: AI audit. “Think about a time in this module when you used AI (or were tempted to). What did it help with? Where did you have to override it?” Open discussion or think-pair-share. Collect themes on the board. Common patterns: AI is useful for debugging code and generating plot templates; it’s unreliable for interpreting results and choosing methods. |
| 40–60 | 🎓 | Concept block 2: Reproducibility. The replication crisis: many published findings don’t hold up when others try to reproduce them. Why? (p-hacking, underpowered studies, selective reporting, data not shared.) Connection to this course: your commit history is your reproducibility record. If someone cloned your repo and ran your code, would they get the same results? Practical reproducibility habits: (1) Script everything (no “I typed this in the console”). (2) Document your decisions (why you chose this test, why you excluded these data points). (3) Record your software versions. (4) Share your data and code (within ethical limits). Connection to the policy report: a policy-maker should be able to trace your conclusion back to your data. |
| 60–75 | 💬✏️ | Exercise 2: Reproducibility check. Students open their project repo. Can they re-run their analysis from scratch (fresh R session, run the script, get the same results)? Spend 10 minutes trying. Identify anything that breaks: hardcoded file paths, missing libraries, steps done in the console but not saved. Fix what you can. |
| ☕ Possible 5-minute break here | ||
| 75–95 | 🎓💬 | Concept block 3: Scientific integrity. Brief and frank. The spectrum from honest mistakes to deliberate fraud: (a) Innocent errors — rounding, wrong column, misread output. (Your peer reviewer catches these.) (b) Questionable research practices — p-hacking, HARKing (Hypothesising After Results are Known), selective reporting. (Often unintentional. This is what the base rate fallacy and multiple comparisons sessions were about.) (c) Fabrication and falsification — making up or altering data. (Rare, career-ending, and easier to detect than people think — e.g., Benford’s law, duplicated images, statistically impossible results.) Where does AI-generated analysis sit? If AI writes your analysis and you don’t check it, you are responsible for the errors. If AI generates data or figures that don’t reflect reality, that’s fabrication — even if unintentional. The HolmesCo connection: HolmesCo doesn’t fabricate data, but their corner-cutting on design, analysis, and interpretation produces conclusions that are functionally no better than fabrication. The lesson: integrity isn’t just “don’t lie.” It’s “do the work properly.” |
| 95–105 | 💬 | Open discussion / Q&A. This is a looser session. Students may have questions about their reports, about AI use policies, about what counts as plagiarism when AI is involved. Address what comes up. |
| 105–110 | 📋 | Wrap-up. “In the application session, you’ll use AI to critique your own work — and then decide how much to trust it.” |
Notes
- This session should feel different in tone: more conversational, fewer exercises, more room for debate. It’s the closest the module comes to a seminar.
- The live AI demo is high-risk, high-reward. If it works (AI gets the base rate problem wrong), it’s a powerful illustration. Have a backup screenshot in case the AI gives a correct answer on the day.
- The integrity discussion should be honest and non-preachy. Students know what plagiarism is. The more useful message is about unintentional bad practice — the kind of thing HolmesCo does — where you believe your own results because you didn’t check carefully enough.
Week 9 Application Session: “AI as critic”
Aims: Students use AI to critique their own draft analyses. Evaluate where AI is helpful and where it misleads. Continue refining analyses and writing reports.
Session plan
| Time | Type | Activity |
|---|---|---|
| 0–10 | 📋 | Setup. Today’s exercise: use an AI assistant to review your project analysis. Your job is not to accept its feedback uncritically — it’s to evaluate each suggestion. |
| 10–30 | 🎓💻 | Guided example: AI critique. Instructor demonstrates: paste a section of analysis (code + written interpretation) into an AI tool. Read the AI’s feedback. For each point, ask: is this valid? Does it apply to our specific data? Is it suggesting something we’ve already addressed? Show a case where the AI gives genuinely useful feedback (e.g., “you haven’t checked the normality assumption”) and a case where it’s wrong or generic (e.g., “consider a larger sample size” when the sample is the full population). |
| 30–70 | ✏️ | Individual/group work: AI-assisted review. Students feed their analysis to an AI and evaluate the response. For each piece of AI feedback, they record: (a) what the AI suggested, (b) whether they agree, (c) what they did about it. This record goes into a “reflections” document in their repo. Students also continue refining their analyses and writing their reports. |
| 70–85 | 💬 | Discussion: What did the AI get right and wrong? Collect examples. Themes to draw out: AI is good at spotting mechanical issues (missing labels, unchecked assumptions) but poor at evaluating whether your analysis actually answers your question. AI rarely asks “Is that a big number?” or “What is this model not capturing?” — the habits this course has taught you. |
| 85–105 | ✏️ | Writing time. Protected time for report writing. Students should be working on their individual reports (not just group analysis). Instructor circulates for questions about structure, content, and interpretation. |
| 105–110 | 📋 | Wrap-up and preview. “Next week is the last week. You’ll exchange drafts, review each other’s work one final time, and prepare to submit.” |
Notes
- AI tools available: students can use whatever they have access to (ChatGPT, Copilot, Posit Assistant, etc.). The exercise is tool-agnostic.
- The reflections document is a useful artefact: it demonstrates that students engaged critically with AI rather than just accepting its output. Consider requiring it as a supplementary submission alongside the report.
- Some students may not want to use AI, and that’s fine. They can spend this session on report writing, or critique a classmate’s analysis instead (a human version of the same exercise).
Week 10 Content Session: “Bringing it together”
Aims: How to structure a compelling report. Peer review in science. Common pitfalls in writing up results. This session is the last content delivery; it should leave students confident about what a good report looks like.
Session plan
| Time | Type | Activity |
|---|---|---|
| 0–5 | 📋 | Recap. You’ve designed an investigation, collected and analysed data, checked assumptions, fitted models, and identified limitations. Now: write it up so someone else can understand it and act on it. |
| 5–25 | 🎓 | Concept block 1: Report structure. Walk through the expected structure of the 4-page policy report: (1) Summary (half page): the main finding, the key evidence, the recommendation. A reader who stops here should still know the answer. (2) Introduction (half page): the question, why it matters, what you investigated. (3) Methods (half to one page): data sources, analytical approach, what tests and why. Enough detail that someone could reproduce your analysis. (4) Results (one to one and a half pages): figures, test results with effect sizes and confidence intervals, plain-language interpretation. (5) Discussion and conclusions (one page): what the results mean, caveats and limitations, policy recommendation. Show examples from the Phase 1 exemplars (faithful AI report). Point out structural features: summary states the conclusion up front; figures are captioned with “so what”; limitations are honest. |
| 25–40 | ✏️💬 | Exercise 1: Summary swap. Students bring a draft summary paragraph (or write one now). Swap with a neighbour (not from the same project group). Each reader answers: (a) what is the main finding? (b) do I trust it? (c) what would I do with this information? Discuss: if the reader can’t answer these questions from the summary alone, the summary needs work. |
| 40–60 | 🎓 | Concept block 2: Peer review in science. How peer review works: submit → editor assigns reviewers → anonymous (usually) critique → revise and resubmit. Why it matters: it’s how science self-corrects. Its weaknesses: slow, biased toward established researchers and ideas, doesn’t catch fraud well. What makes a good review: specific, constructive, focused on the argument and evidence (not just grammar). Connection to this course: your GitHub Issues review in Week 4 was peer review. This week you’ll do it again, on work that counts. What you learned from Week 4’s traitor exercise should make you a better reviewer now. |
| 60–75 | 💬✏️ | Exercise 2: Common pitfalls. Show 5–6 excerpts from fictional reports (could be HolmesCo’s greatest hits). Each has a common writing problem: (1) Conclusion not supported by the data shown. (2) Figure with no caption or reference in the text. (3) p-value reported without effect size or CI. (4) “The results prove that…” (overstatement). (5) Methods too vague to reproduce. (6) No limitations section. Students identify the problem and suggest a fix. Quick-fire, 2 minutes each. |
| ☕ Possible 5-minute break here | ||
| 75–95 | 🎓 | Concept block 3: Telling the story honestly. The difference between advocacy and analysis. Both the faithful and traitor exemplars made policy recommendations — but one built its case honestly and the other manipulated the reader. Recap the key habits from the course: Compared to what? (Week 2.) What assumptions are we making? (Week 3.) How plausible was this before we tested? (Week 6.) What is the model not capturing? (Week 8.) A good report uses these questions reflexively. A HolmesCo report ignores them. Your goal: be the analyst whose work a policy-maker can trust. |
| 95–105 | 📋 | Peer review briefing. Explain the process for the application session: each student will be assigned another student’s repo (from a different project group). They will read the draft report and file GitHub Issues using the review template (updated from Week 4). Emphasise: this is summative-quality work. Be honest, specific, and constructive. |
| 105–110 | 📋 | Wrap-up. “Application session: bring a complete draft. You’ll review and be reviewed. Then: final revisions and submission.” |
Notes
- This session is largely a writing workshop in lecture format. The energy should come from the examples and exercises, not from new technical content.
- HolmesCo’s “greatest hits” in exercise 2 provide continuity and humour in the final week. Each pitfall should be recognisable from earlier sessions — this is a chance for students to see how far they’ve come.
- The honest storytelling segment should tie together the full arc of the course. It’s worth naming the key habits explicitly and showing that each one came from a specific week.
Week 10 Application Session: “Review, revise, submit”
Aims: Students submit draft reports, conduct peer review via GitHub Issues, and make final revisions. This mirrors the Week 4 application session but with higher stakes and (hopefully) more sophistication.
Session plan
| Time | Type | Activity |
|---|---|---|
| 0–10 | 📋 | Orientation. Three phases: finalise draft (20 min), peer review (40 min), revise (30 min). Every student should have a near-complete draft committed to their personal repo. If not: the first 20 minutes are your last chance. |
| 10–30 | ✏️ | Final draft preparation. Students make last edits to their reports. Commit and push by minute 30. Hard deadline: whatever is committed at minute 30 is what gets reviewed. |
| 30–35 | 📋 | Peer review setup. Announce review assignments (pre-prepared: each student reviews one student from a different project group). Students navigate to their assigned repo. |
| 35–75 | ✏️ | Peer review via GitHub Issues. Each student reads their assigned report and files 3–4 GitHub Issues with structured comments. Updated review template: |
## Peer Review: [Author's name]
### Summary and clarity
- [ ] The summary states a clear finding and recommendation
- [ ] The argument flows logically from question → method → result → conclusion
- [ ] A non-specialist policy-maker could follow the report
### Evidence and analysis
- [ ] Statistical tests are appropriate for the data and question
- [ ] Assumptions are checked and documented
- [ ] Effect sizes or confidence intervals are reported (not just p-values)
- [ ] Figures are well-chosen, clearly labelled, and referenced in the text
- [ ] Numbers have context ("Is that a big number?")
### Uncertainty and limitations
- [ ] The report acknowledges what the analysis cannot tell us
- [ ] Assumptions and potential confounders are discussed
- [ ] The conclusion matches the strength of the evidence (no overstatement)
### Reproducibility
- [ ] The methods are described clearly enough to reproduce
- [ ] Code is committed and runs
### One thing done well:
### One thing to improve:| Time | Type | Activity |
|---|---|---|
| 75–90 | 💬 | Debrief. Brief whole-class discussion. What did you notice in the reports you reviewed? Any recurring issues? What was the best thing you saw? Instructor highlights themes. Compare to Week 4: are reviews more sophisticated now? (They should be — students now have vocabulary for assumptions, effect sizes, confounders, and base rates.) |
| 90–105 | ✏️ | Revision time. Students read their reviews and make final revisions. Commit updated report. |
| 105–110 | 📋 | Course wrap-up. Brief reflection: where you started (Week 1, “is biomass carbon-neutral?”) and where you are now. You can design an investigation, test a hypothesis, build and critique a model, spot misleading claims, and write for a policy audience. These are the skills you’ll use in your dissertations and careers. Thank the students. Remind them of the submission deadline. |
Notes
- Peer review assignments should cross project groups so reviewers evaluate unfamiliar work. This is harder and more valuable than reviewing a project you already understand.
- The review template is more demanding than Week 4’s. It now includes effect sizes, assumption checking, reproducibility, and explicit checks against overstatement — reflecting everything taught in Weeks 5–9.
- After the session: instructor (and demonstrators) review all reports and Issues. Brief written feedback via a closing comment on each Issue. Students can revise until the submission deadline (which should be at least a few days after Week 10).
- Consider whether the AI reflections document from Week 9 should be submitted alongside the report. It adds minimal burden and provides evidence of critical AI engagement.
Cross-cutting threads: Weeks 5–10
HolmesCo’s arc
HolmesCo appears in most content sessions as a consistent example of how not to do things. Their failures map to the course’s learning outcomes:
| Week | HolmesCo scenario | What’s wrong | Skill tested |
|---|---|---|---|
| 5 | Site investigation with 3 valley boreholes — none on ridgetops where turbines are planned | Selection bias, convenience sampling, tiny sample | Experimental design |
| 6 | “Gold discovered in County Durham!” | Base rate neglect; P(data|H₀) ≠ P(H₀|data) | Conditional thinking |
| 7 | 3 “critical mineral” hits from 400 tests | Multiple comparisons; fewer hits than expected by chance | Multiple testing correction |
| 8 | Linear extrapolation of groundwater decline | Extrapolation failure; missing recharge term | Model limits |
| 9 | (implied) AI-written EIA | Uncritical use of AI; lack of domain checks | AI literacy |
| 10 | Greatest hits: pitfall excerpts | Summary of common errors | Report writing |
By Week 10, “What would HolmesCo do?” is a shorthand for every bad practice the course teaches students to avoid.
Key refrains
These questions should become automatic by Week 10:
| Refrain | Introduced | Reinforced |
|---|---|---|
| “Is that a big number?” | Week 2 | Weeks 5, 7, 10 |
| “Compared to what?” | Week 2 | Weeks 4, 5, 7 |
| “What assumptions are we making?” | Week 3 | Weeks 6, 8, 10 |
| “How plausible was this before we tested?” | Week 6 | Weeks 7, 9 |
| “What is the model not capturing?” | Week 8 | Weeks 9, 10 |
| “What would HolmesCo do?” | Week 5 | Weeks 6, 7, 8, 10 |
Git progression (Phase 2)
| Week | Git activity |
|---|---|
| 5 | Create group repo; each member: branch → commit → PR → merge |
| 6 | Commit analysis code via PR to group repo |
| 7 | Continue group commits; begin individual report in personal repo |
| 8 | Commit model code; cross-reference group and personal repos |
| 9 | Commit AI reflections document; continue individual report |
| 10 | Final draft committed to personal repo; peer review via Issues |
Progressive skill building (Phase 2)
| Skill | Week 5 | Week 6 | Week 7 | Week 8 | Week 9 | Week 10 |
|---|---|---|---|---|---|---|
| Statistics | Design principles | t-test; Bayesian thinking | ANOVA; effect sizes | Regression; diagnostics | — | — |
| R | — | t.test(), shapiro.test() |
aov(), TukeyHSD(), cohens_d() |
lm(), plot(model), poly() |
AI tools | — |
| Communication | Research plan | Write results paragraph | Report effect sizes | “What is the model not capturing?” | AI as critic; reproducibility | Full report; peer review |
| Critical thinking | Critique design; spot confounders | Base rate fallacy | Multiple comparisons | Overfitting; extrapolation; blind spots | AI limitations; integrity | Synthesise all habits |
| Git | Group repos; branches; PRs | Group commits | Begin personal report repo | Cross-reference repos | Commit reflections | Final submission; peer review via Issues |