AI as critic

Research Methods — Week 9 Application

Today’s plan

Goals

Use AI to critique your own work — then evaluate the critique.

Phase	Time	What
Guided example	20 min	See AI review in action
AI-assisted review	40 min	Feed your analysis, evaluate feedback
Discussion	15 min	What did it get right/wrong?
Writing time	20 min	Protected time for your report

• [10 min] Setup. Today’s exercise: your job is not to accept AI feedback uncritically — it’s to evaluate each suggestion.

Questions?

Submit questions anonymously:

PollEv.com/geol

text geol to 07480 781235

• Open the Poll Everywhere Q&A activity before the session and leave it active throughout.
• Questions appear in your Poll Everywhere presenter view; upvoted questions rise to the top.
• Address during transitions or at the end of concept blocks, not mid-explanation.
• Reset (archive) the activity between sessions so questions don’t carry over from the previous week.

Guided example

🎓💻 AI critique demo

Live demo

🖥️ Pasting an analysis into an AI tool

For each piece of AI feedback, ask:

1. Is this valid?
2. Does it apply to our specific data?
3. Is it something we’ve already addressed?

• [20 min] Demonstrate with a section of analysis. Show a case where AI gives genuinely useful feedback (“you haven’t checked normality”) and a case where it’s generic or wrong (“consider a larger sample” when you have the full population).

Two examples

AI got it right

“You haven’t checked whether the residuals are normally distributed.”

✓ Valid, actionable, easy to fix.

AI got it wrong

“Consider collecting more data to increase statistical power.”

✗ Our dataset is the full population, not a sample.

AI-assisted review

✏️ Your analysis

The exercise

1. Feed your analysis (code + interpretation) to an AI
2. For each piece of feedback, record:
   • What the AI suggested
   • Whether you agree
   • What you did about it
3. Save this record as a reflections document in your repo

• [40 min] Students work individually or in groups. The reflections document demonstrates critical AI engagement.
• Some students may prefer not to use AI — that’s fine. They can spend this time on report writing, or critique a classmate’s analysis instead.

What to look for

AI is good at spotting mechanical issues:

• Missing labels, unchecked assumptions, formatting

AI rarely asks the hard questions:

“Is that a big number?”

“What is this model not capturing?”

Those are the habits you have learned.

Discussion

💬 What did the AI get right and wrong?

Share examples

What was the best feedback AI gave you?

What was the worst?

• [15 min] Collect examples. Themes: AI spots mechanical issues but can’t evaluate whether your analysis answers your question.

Writing time

✏️ Report writing

Protected time

The remaining time is for report writing.

You should be working on your individual report — not just group analysis.

I’ll circulate for questions about structure, content, and interpretation.

• [20 min] Students write. Instructor circulates. This is valuable — protected writing time in class is rare and appreciated.

Wrap-up

Before you leave

• AI reflections document committed to your repo
• Report draft in progress

Next week is the last week.

You’ll exchange drafts, review each other’s work, and submit.

• The reflections document could be required as a supplementary submission. Consider requiring it — it’s low-burden and demonstrates critical engagement.