← Module 5: Scientific Investigations
Inquiry Question 1: What are the steps and considerations necessary to plan and conduct a scientific investigation?
Communicate scientific understanding using suitable language and terminology, including the role of peer review and replication in confirming scientific findings
A focused answer to the HSC Investigating Science Module 5 dot point on peer review and replication. Covers what peer review does, why it matters, the reproducibility crisis, and worked HSC past exam questions on confirming scientific findings.
Have a quick question? Jump to the Q&A page
What this dot point is asking
NESA wants you to explain peer review and replication as the two main mechanisms by which scientific claims become established knowledge, and to recognise that science is self-correcting through these processes. This dot point underpins almost every "evaluate" question in Modules 6, 7 and 8.
The answer
A finding by one researcher is provisional. Scientific knowledge emerges only when claims survive critical scrutiny. Two formal processes do this work.
Peer review
Before publication in a reputable scientific journal, a manuscript is sent to two to four independent experts in the field (the "peers"). They evaluate the work and recommend acceptance, revision or rejection.
What reviewers check.
- Originality and significance. Does the work advance the field?
- Methodology. Are the variables, controls, sample size and statistical methods appropriate?
- Data quality. Are the data plausible and well-presented?
- Conclusions. Do the conclusions follow from the data without overreach?
- Replicability. Are methods described in enough detail for others to repeat the work?
- Ethics. Have appropriate approvals been obtained?
Strengths. Peer review filters out obvious errors, methodological flaws and overstated claims before they enter the scientific record. It is the central credibility-signal in modern science.
Limitations. Peer review is imperfect. Reviewers can miss fraud (Wakefield), share field-wide biases, or be too slow for fast-moving science. It does not detect data fabrication unless the data are visibly impossible.
Replication
Independent researchers repeat the original experiment, ideally in different labs with different operators and equipment, to test whether the result holds.
Types of replication.
- Direct replication. Same protocol, same conditions, different team.
- Conceptual replication. Same hypothesis tested with different methods.
- Systematic review and meta-analysis. Combining results from multiple studies to estimate the true effect.
What replication contributes. It distinguishes robust findings from one-off results due to chance, undisclosed bias or methodological flaws.
The 2010s reproducibility crisis
In 2015 the Open Science Collaboration attempted to replicate 100 published psychology studies. Only 36 per cent produced effects in the same direction at statistical significance. Similar concerns affect biomedical research: Amgen replicated only 6 of 53 "landmark" cancer studies in 2012, and Bayer replicated only 25 per cent of preclinical findings in 2011.
Causes. Publication bias (only positive results published), p-hacking (running analyses until something looks significant), small sample sizes, undisclosed conflicts of interest, and outright fraud.
Reforms.
- Pre-registration. Researchers register hypotheses, methods and analyses before collecting data.
- Open data. Raw data made publicly available for re-analysis.
- Registered reports. Journals accept papers based on the methodology before results are known.
- Higher statistical standards. Larger sample sizes, more stringent significance thresholds.
Australian context
The National Health and Medical Research Council (NHMRC) funds replication studies and enforces research integrity through the Australian Code for the Responsible Conduct of Research. The Australian Research Council (ARC) similarly enforces standards for non-medical research. Both require open data where appropriate.
Self-correcting nature of science
Science differs from belief systems in that claims are provisional and can be withdrawn. Retraction Watch maintains a public database of retracted papers. The Wakefield MMR paper, the South Korean stem-cell papers by Hwang Woo-suk and the social-priming literature in psychology are landmark retractions that show the system at work, even if slowly.
Past exam questions, worked
Real questions from past NESA papers on this dot point, with our answer explainer.
2024 HSC5 marksExplain the role of peer review and replication in establishing scientific knowledge.Show worked answer →
A 5-mark answer needs both processes defined, what each contributes, and how they work together.
- Peer review
- Before a scientific paper is published in a reputable journal, the manuscript is sent to two to four independent experts in the field. Reviewers evaluate the methodology, statistical analysis, conclusions and whether the work advances the field. They can recommend acceptance, revision or rejection.
- What it contributes
- Filters out methodological errors, identifies overstated conclusions, catches statistical mistakes, ensures methods are described in enough detail for others to follow. Peer review is the first checkpoint that distinguishes peer-reviewed science from anecdote and self-published claims.
- Replication
- Independent researchers repeat the original experiment, ideally in different labs with different equipment, to test whether the result holds. Successful replication strengthens the evidence; failed replication undermines the original claim.
- What it contributes
- Replication tests whether findings are robust to different operators, samples and instruments. A single positive result is provisional; multiple independent replications establish a result as scientific knowledge.
- Working together
- Peer review filters at the gate of publication; replication tests claims after publication. Both are necessary. The 2010s reproducibility crisis (where many psychology and biology papers failed to replicate) showed that peer review alone is insufficient.
Markers reward both definitions, distinct functions and the recognition that scientific knowledge requires both.
2022 HSC3 marksWhy is the failure to replicate a published finding significant for science?Show worked answer →
A 3-mark answer needs the significance of replication failure, an example, and the consequence.
- Significance
- Science is provisional and self-correcting. A single published finding is suggestive but not established knowledge. Independent replication tests whether the result is robust to different researchers, samples, instruments and assumptions. Failed replication signals that the original claim may be a false positive, due to chance, methodology flaws, p-hacking, undisclosed bias or fraud.
- Example
- Andrew Wakefield's 1998 Lancet paper linking the MMR vaccine to autism failed to replicate in 14 subsequent studies. The paper was retracted in 2010 after revelations of conflict of interest and data fabrication. The retraction is the canonical case of replication discrediting a false claim.
- Consequence
- Replication failure prompts retraction of the original paper, re-examination of related findings and updating of the consensus. The 2010s reproducibility crisis in psychology, where only 36 per cent of 100 studies replicated, has led to pre-registration, open data and stricter statistical standards.
Markers reward the function of replication, a named example and the practical consequence for science.
Related dot points
- Evaluate scientific investigations and findings in terms of reliability, validity, accuracy and precision of data
A focused answer to the HSC Investigating Science Module 5 dot point on reliability, validity, accuracy and precision. The four concepts every Investigating Science student must distinguish, with worked HSC past exam questions.
- Plan, source and acknowledge primary and secondary data appropriate to the investigation
A focused answer to the HSC Investigating Science Module 5 dot point on primary and secondary data. Covers the distinction, sourcing and acknowledging secondary data, evaluating source quality, and worked HSC past exam questions.
- Investigate a case where a scientific claim has been retracted, including the role of media in disseminating discredited claims
A focused answer to the HSC Investigating Science Module 7 dot point on Andrew Wakefield's 1998 paper. Covers the original claim, the methodological flaws, the conflict of interest, the retraction and its lasting impact on vaccination rates, and worked HSC past exam questions.