Problem 1
Assuming the null model is true, results at least this incompatible with the null would occur about 3% of the time; it is not the probability the null is true.
Faculty solution guide
Logic of Significance Tests
Accept equivalent statistically correct reasoning. Require context, design, assumptions, uncertainty, and limits.
Problem 2
A common error is to report a calculation without identifying the population, design, assumptions, uncertainty, context, or limits on causal and generalizing claims.
Problem 3
A complete response identifies population, cases, variables, parameter or estimand, study design, and conditions for Logic of Significance Tests.
Problem 4
Repair the earliest invalid assumption, display, calculation, or inference; then rerun the interpretation and state how the correction changes the claim.
Problem 5
A strong application defines context, data source, design, analysis plan, uncertainty, and limits on generalization or causation.
Problem 6
Use at least two of: independent recomputation, alternative graph or model, residual/diagnostic check, resampling, sensitivity analysis, subgroup analysis, simulation, or comparison with a known benchmark.