External Validity, Effect Size & Implementation

Retrieval Warm-Up

Without notes, write one idea you already know that might connect to External Validity, Effect Size & Implementation. Then write one question the lesson should help you answer.

Essential Question

When does a research finding travel successfully into the real world?

Learning Objectives

Students will be able to:

• define external validity and generalizability;
• distinguish statistical detectability from practical importance;
• explain effect size conceptually;
• identify implementation barriers such as cost, access, fidelity, burden, and adoption;
• compare efficacy with real-world implementation outcomes;
• justify when a pilot test is more responsible than immediate full adoption.

generalizability

Vocabulary in Context

generalizability · effect size · practical significance · implementation · fidelity · adoption · feasibility · scalability · pilot study · replication

Substantial Core Reading

A study can be well controlled and still have limited generalizability. External validity asks whether the result applies to other people, places, tasks, and times. Students learn that generalization is an evidence question rather than an assumption.

Effect size concerns how large a difference or relationship is, while practical importance asks whether that magnitude matters for the real-world decision. Implementation adds another layer: an intervention that works under research conditions may perform differently when time, training, cost, access, staffing, compliance, culture, or competing priorities change.

Worked Example / Synthesis

A tightly controlled memory intervention improves delayed test performance by a modest amount in one laboratory sample.

Students consider whether a school should:

• adopt it everywhere immediately;
• reject it;
• run a measured pilot;
• or seek more evidence.

The best response depends on effect size, cost, risk, population similarity, feasibility, and available alternatives.

Psychological Scientist's Desk

Claim: “If a study has a very small p-value, the intervention must be important.”

Students distinguish statistical evidence from effect magnitude and practical decision value without requiring advanced inferential statistics.

Data Lens

Students compare three fictional interventions showing different combinations of effect size, cost, burden, and retention. They rank decisions and explain why the biggest effect is not automatically the best implementation choice.

Claims Laboratory

Evaluate:

• “Statistically significant means important.”
• “A replicated lab effect will automatically work in a school.”
• “If implementation fails, the scientific finding was false.”
• “The cheapest intervention is best.”

Guided Investigation

Build a Claim → Measure → Evidence → Alternative Explanation → Limitation → Conclusion map for External Validity, Effect Size & Implementation.

Extended Inquiry from the Locked Plan

Independent Assignment

Write a 450–650 word recommendation for a fictional school deciding whether to pilot an evidence-supported intervention. Use effect size, generalizability, feasibility, and ethics.

Research Notebook Checkpoint

For the semester project, state:

• the population actually represented;
• the setting represented;
• the strongest justified generalization;
• one population or setting that remains uncertain;
• whether a pilot would be appropriate before broad application.

Academic Journal Reflection

Why can a scientifically credible result still be a poor basis for a large real-world decision?

5-question formative check

Review status