Skip to content

Science's Quality Control Pipeline

Run the Peer Review Pipeline Fullscreen

About This MicroSim

This interactive pipeline diagram demystifies the often-invisible process of how scientific knowledge is validated. Using a vis-network flowchart, it shows the journey of a research finding from initial research question through study design, data collection, analysis, manuscript preparation, journal submission, peer review, publication, replication attempts, and ultimately scientific consensus. Branching paths at the peer review stage show three possible outcomes: accept, revise and resubmit, or reject.

Each node is clickable and reveals a description panel explaining what happens at that stage, common pitfalls, and how long it typically takes. A confidence tracker bar at the top shows how the reliability of a scientific claim changes at each stage -- low at the hypothesis stage, medium at publication, high after successful replication, and very high at consensus. A failed replication path leads to reassessment, illustrating that science is self-correcting.

The color scheme reinforces the narrative: early research stages appear in light blue, the peer review bottleneck in yellow (caution), successful publication in green, and failed paths in red. This visualization helps students understand why peer-reviewed research carries more weight than preliminary findings and why replication is essential.

How to Use

  1. Start at the leftmost node ("Research Question") and click on it to read what happens at this stage.
  2. Follow the pipeline from left to right, clicking each node to learn about that stage of the scientific process.
  3. Watch the confidence tracker bar at the top change as you move through the pipeline stages.
  4. At the peer review stage, explore all three branches: accept, revise and resubmit, and reject.
  5. Follow the path past publication to replication attempts, and notice the branch between successful replication (leading to consensus) and failed replication (leading to reassessment).
  6. Note the time estimates at each stage to understand why building scientific consensus takes years.

Iframe Embed Code

You can add this MicroSim to any web page by adding this to your HTML:

1
2
3
4
<iframe src="https://dmccreary.github.io/ecology/sims/peer-review-pipeline/main.html"
        height="600px"
        width="100%"
        scrolling="no"></iframe>

Lesson Plan

Grade Level

9-12 (High School Science / AP Environmental Science)

Duration

40 minutes

Learning Objectives

  • Describe the steps from initial research through peer review, publication, and replication to scientific consensus
  • Explain why peer review and replication are essential for building reliable scientific knowledge
  • Evaluate the confidence level of scientific claims based on where they are in the pipeline

Prerequisites

  • Basic understanding of the scientific method (hypothesis, experiment, analysis)
  • Familiarity with the concept of scientific publications and journals
  • Introduction to evaluating sources of information

Standards Alignment

  • NGSS Science and Engineering Practice 8: Obtaining, evaluating, and communicating information
  • AP Environmental Science: Topic 10.1 -- Introduction to Sustainability; Science Practices

Activities

  1. Source Sorting Warm-Up (5 min): Give students five claims about ecology (e.g., "Coffee cures cancer" from a blog, "Biodiversity loss accelerates extinction" from a peer-reviewed journal). Ask: Which do you trust more and why? Discuss what makes some sources more reliable.

  2. Pipeline Walk-Through (15 min): As a class, click through every node in the pipeline. At each stage, discuss: What could go wrong here? How does this step increase our confidence in the findings? Students create a timeline showing each stage and its typical duration.

  3. Peer Review Role Play (10 min): Divide students into groups of three: author, reviewer 1, and reviewer 2. Give them a simplified one-paragraph "manuscript abstract" about an ecological finding. Reviewers write two strengths and two weaknesses. Author responds to the reviews. Discuss: Was this process useful? How might it catch errors?

  4. Claim Evaluation (10 min): Present three ecological claims at different pipeline stages (preliminary study, single published paper, replicated consensus). Students rank them by confidence level and justify their rankings using the pipeline model.

Assessment Questions

  1. Explain why a single published study, even in a prestigious journal, does not constitute scientific consensus.
  2. A news headline reads "Scientists discover that deforestation reverses climate change." This is based on one unpublished preprint. Using the pipeline model, evaluate the confidence level of this claim.
  3. Describe how the peer review process serves as quality control and identify two limitations of peer review.

References

  1. National Academies of Sciences (2019). Reproducibility and Replicability in Science. The National Academies Press.
  2. Kelly, C.D. (2019). Rate and success of study replication in ecology and evolution. PeerJ, 7, e7654.
  3. Oreskes, N. (2019). Why Trust Science? Princeton University Press.