Verification and Reproducibility¶
Summary¶
This chapter covers verification and validation as habits: independent checks, primary source checks, cross-model checks, test-driven research, and result replication. It addresses overconfidence and sycophancy in model output and builds audit trails and reproducibility packages. After this chapter you can decide whether an AI-assisted result meets the standard of your field.
Concepts Covered¶
This chapter covers the following 16 concepts from the learning graph:
| Concept | Concept Impact Score |
|---|---|
| Verification | 56 |
| Validation | 1 |
| Independent Check | 8 |
| Primary Source Check | 1 |
| Cross Model Check | 1 |
| Test Driven Research | 1 |
| Result Replication | 1 |
| Uncertainty Awareness | 5 |
| Overconfidence | 1 |
| Sycophancy | 3 |
| Output Bias | 3 |
| Audit Trail | 10 |
| Reproducibility Package | 1 |
| Peer Verification | 1 |
| Acceptance Review | 1 |
| Error Tracing | 1 |
Prerequisites¶
This chapter builds on concepts from:
- Chapter 1: Research Workflows and the Researcher as Orchestrator
- Chapter 2: How Language Models Work
- Chapter 3: Choosing Models and Writing Clear Prompts
- Chapter 4: Context Engineering and Advanced Prompting
- Chapter 6: Knowledge Bases, Personal Knowledge Graphs, and the Anatomy of an Agent Skill
- Chapter 10: Orchestrating Teams of Planning and Execution Agents
- Chapter 11: AI-Assisted Literature Search and Synthesis
- Chapter 13: Simulation, Testing, and Reproducible Code
Round Up the Evidence
Every earlier chapter had a check in it, and this chapter is where the checks become a standard you can state. By the end you will be able to say, for a result that an agent helped produce, exactly what would convince a skeptical colleague. Trust, but verify!
Verification and Validation¶
Two words that people often use interchangeably name different questions, and engineers already know the difference. Verification asks whether the work was done right: whether the code implements the formula, whether the computation matches the specification, whether the numbers in the paper are the numbers the script produced. Validation asks whether the right thing was done: whether the specification itself matches reality, whether the simulated circuit behaves as the fabricated one does, whether the result answers the research question. A design can be fully verified against a specification that was wrong, and it will be wrong in exactly the way the specification was.
Both matter for AI-assisted work, and they fail differently. Verification failures are the typical agent failures of earlier chapters: a wrong formula, a changed number, an invented citation. Validation failures are quieter. An agent will faithfully implement the question as you stated it, and if the question was framed badly, every verified result answers the wrong thing. This is one reason that Chapter 14 put so much weight on framing, and why a skeptical reader's first question is often "is this the right comparison?"
Verified Against What?
Notice that a verification always has an object: the specification, the standard, the primary source. When someone says "it was verified", ask against what. If the reference is another output of the same process, nothing independent has happened.
Kinds of Checks¶
An independent check is a check whose failure causes are different from those of the thing being checked. That condition is what gives a check its power. Running the same script twice checks nothing, because the same bug appears twice. Computing the same quantity by a different method, with a different tool, from different data, or by a different person has some chance of disagreeing when something is wrong. Four kinds of independent check are especially useful with agents.
A primary source check is going to the original source to confirm a claim: the paper itself and not a summary of it, the datasheet and not a model's recollection of its limits, the standard and not a paraphrase of its requirement. It is the answer to the fabricated and misattributed claims of Chapters 2 and 11. It takes minutes when the agent supplied the location of every claim.
A cross-model check asks a different model, or a fresh session of the same one with none of the earlier context, to answer the same question or review the same output independently, and compares the answers. It is cheap and a good screen. Its limits are real: models share much of their training data and many of their blind spots, so agreement is weak evidence and disagreement is a useful flag. Use it to decide what to look at, and not to decide what is true.
A known-answer test, from Chapter 13, runs a method on an input whose answer you know independently. And a result replication, taken up below, reruns or repeats the work from the beginning.
A worked example, with illustrative numbers. An agent's script reports an ENOB of 9.4 bits for capture-07. A known-answer test (the ideal quantizer record) shows that the script returns 10.0 for a known 10-bit record, which verifies the implementation. An independent method, a least-squares sine fit of the kind described in the ADC test standard IEEE 1241, gives 9.3 bits on the same record, which is within the expected difference between the methods. A primary source check confirms from the lab notebook that the input frequency was 499 kHz and not the 99 kHz in the script's default. Each check could have failed, and each of them fails differently from the others.
| Check | What it guards against | Cost | Weakness |
|---|---|---|---|
| Known-answer test | Implementation errors | Low once the test exists | Only covers the input you tried |
| Independent method | Method-specific errors and shared assumptions | Medium | The two methods may share an input error |
| Primary source check | Misquotation, fabrication, outdated facts | Low per claim | Only as good as the source |
| Cross-model check | Gross errors, a screening aid | Very low | Shared blind spots |
| Replication | Non-reproducible results | Medium to high | Does not catch a wrong question |
Table 1. Kinds of verification checks and what each one buys.
Diagram: Verification Method Matcher¶
Verification Method Matcher
Type: microsim
sim-id: verification-method-matcher
Library: p5.js
Status: Built
Bloom Level: Evaluate
Bloom Verb: recommend
Learning Objective: The learner will recommend which of five verification methods best fits each of eight AI-assisted claims, with at least 7 of 8 matching the answer in Content.
Prerequisites: known-answer test, independent method, primary source check, cross-model check, and replication, all defined in the section "Kinds of Checks" above.
Evidence of Mastery: For each of eight claims the learner commits one of five methods. A method is correct when it equals the answer in Content. Mastery is at least 7 of 8 correct on the first attempt. Reading the table of methods is exploration and is not evidence.
Misconceptions: (1) Asking the same model again is an independent check. (2) A cross-model check proves a claim. (3) A single check method suits every claim.
Instructional Rationale: Evaluate-level work needs the learner to choose a method by what it guards against. The claims differ in what could be wrong with them, so a single favorite method cannot score well.
Content:
All claims are invented for illustration and labeled "illustrative". Method options: Known-answer test, Independent method, Primary source check, Cross-model check, Replication.
| # | Claim or situation | Correct method | Why (shown as feedback) |
|---|---|---|---|
| 1 | A script reports ENOB = 9.4 for capture-07, and you have never validated the script. | Known-answer test | Run it on a record whose ENOB you know, to check the implementation before trusting any real result. |
| 2 | A model's summary says Paper P reports 9.2 bits of ENOB at 1 MS/s. | Primary source check | The claim is about what a paper says, and only the paper can settle it. |
| 3 | The FFT-based ENOB is 9.4 and you want another estimate that does not share the FFT's assumptions. | Independent method | A different method, such as a sine fit, can disagree for different reasons. |
| 4 | An agent labeled 300 abstracts relevant or not, and you want a cheap screen for likely mislabels before sampling by hand. | Cross-model check | A second model's disagreement points to items worth reading, at very low cost. |
| 5 | A colleague wants to reuse your figure, and you must confirm it regenerates from raw data on a clean machine. | Replication | The question is whether the result can be reproduced from the raw inputs and code. |
| 6 | A model states "the standard requires jitter below 10 ps in section 4.3". | Primary source check | A requirement is quoted from a document, and the document must be read. |
| 7 | You doubt a simulation result because a model-written netlist might misstate a component value. | Independent method | A hand calculation of the expected value is a different route to the same quantity. |
| 8 | A result must hold for a different batch of devices before you publish a claim. | Replication | The claim is that the result recurs, which only another run can show. |
Provenance: The five methods are from the section "Kinds of Checks" of this chapter. The claims are invented for illustration.
Rules: Each claim has exactly one correct method, chosen by what the claim could be wrong about: an implementation, a method, a quotation, a screening question, or recurrence.
Learner Activity:
- The learner reads claim 1.
- The learner chooses one of the five methods and submits.
- The sim shows whether it was correct and the "Why" sentence.
- After claim 8, the sim shows the count correct and groups the claims by method.
Feedback: Eight claims, one attempt each, in a fixed order. Correct: "Correct:
Starting State: Claim 1 is shown with five methods and none chosen. The question on screen is "What could be wrong with this claim, and which check would reveal it?"
Chapter Anchors: The chapter states that an independent check has different failure causes from the thing checked, that cross-model agreement is weak evidence while disagreement is a useful flag, and that a primary source check is made against the original document.
MicroSim: Verification Method Matcher. Open it fullscreen.
Working Test-First and Replicating¶
Test-driven research is the practice, borrowed from software, of deciding in advance what result would count as success or failure and writing it down before the experiment or analysis is run. It extends the acceptance criteria of Chapter 4 and the go criteria of Chapter 14 to results. The reason is a human one, made worse by agents: once a result is visible, it is easy to find a reading that makes it acceptable, and an agent asked to produce "a good result" will help. A criterion written first cannot be adjusted to fit.
Result replication is obtaining the result again. There are two levels, and both are worth distinguishing. Computational replication reruns your own analysis from the raw data in a clean environment, to confirm that the result is a property of the code and data and not of a leftover file or an unrecorded setting. Independent replication has another person, another instrument, or another dataset produce the result from the description, which also tests the description. The first is cheap and should be routine. The second is what ultimately moves a field, and a reproducibility package, described below, is what makes it possible.
Behaviors of Models That Undermine Checks¶
Three tendencies of language models weaken verification when they are not recognized. They are properties of the models, not of any one product.
Uncertainty awareness is the discipline of stating how sure a result is, and asking for it. A model's fluent register carries no information about its confidence, so its self-reported confidence should not be treated as calibrated. What serves is explicit uncertainty in the work: confidence intervals from repeated measurements, a range and not a point, a list of the assumptions on which a number depends, and the sample size behind every statistic.
Overconfidence is the model's tendency to state conclusions with a certainty that its evidence does not support, including stating a specific figure where "unknown" would be correct. The countermeasures are those of Chapter 3: make "unknown" a legal output, require evidence with every claim, and ask for the strongest reason the answer could be wrong.
Sycophancy is the tendency to agree with the user, to tell them what they appear to want to hear, and to change an answer under pushback even when no new evidence has been given. It is particularly harmful in research, because researchers are invested in their results. A worked example: asked "I measured an ENOB of 9.5 bits, which looks great, doesn't it?", a model is likely to agree and elaborate. Asked "Here are the measurement conditions and the ENOB value. What are the three most likely reasons this number could be wrong?", the same model will produce something useful. The difference is in the framing. The practical countermeasures are to ask neutral questions that do not reveal your preferred answer, to ask for the case against, to flip the premise and see whether the answer flips with it, and to treat a change of answer under pushback as a warning unless the pushback contained evidence.
Output bias is a systematic tilt in what a model produces. It can favor the most popular method over a better suited but less written-about one, the literature in the most widely represented language and venues, recent and well-publicized work over older foundations, and the framing the prompt implies. It is hard to see from one answer. Counter it by asking for alternatives explicitly, by checking which literature a survey draws on, and by running the same task with different phrasings and noting what changes.
Flip the Premise
To test whether an answer reflects the evidence or your framing, ask the question again with the opposite premise: "I suspect this result is wrong. What supports that?" If the model agrees with both framings, it is following you, and neither answer is evidence.
Diagram: Sycophancy Spotter¶
Sycophancy Spotter
Type: microsim
sim-id: sycophancy-spotter
Library: p5.js
Status: Built
Bloom Level: Analyze
Bloom Verb: distinguish
Learning Objective: The learner will distinguish, among six exchanges between a researcher and a model, the replies that are sycophantic from those that are calibrated, with at least 5 of 6 classified correctly.
Prerequisites: sycophancy, overconfidence, uncertainty awareness, and the countermeasures, all defined in the section "Behaviors of Models That Undermine Checks" above.
Evidence of Mastery: For each of six exchanges the learner commits "sycophantic" or "calibrated". A classification is correct when it equals the answer in Content. Mastery is at least 5 of 6 correct on the first attempt. Reading the definitions is exploration and is not evidence.
Misconceptions: (1) A model that changes its answer is being responsive. (2) A polite and agreeable reply is a good reply. (3) A model that declines to confirm is being unhelpful.
Instructional Rationale: Analyze-level work needs the learner to separate an update that follows from evidence from agreement that follows from social pressure. Both appear in several tones, so tone alone is not a cue.
Content:
All exchanges are invented for illustration and labeled "illustrative".
| # | Exchange | Correct classification | Why (shown as feedback) |
|---|---|---|---|
| 1 | Researcher: "I think this 10-bit ADC has an ENOB of 11 bits." Model: "You're right, 11 bits is achievable here." | Sycophantic | It agrees with an impossible claim for a nominal 10-bit converter without any evidence. |
| 2 | Researcher: "Are you sure? I think the notch is at 60 Hz." Model (earlier answer, from pasted text: 50 Hz): "You may be right, I apologize. The notch is at 60 Hz." | Sycophantic | It reverses an evidenced answer when no new evidence was given. |
| 3 | Researcher: "The datasheet text I pasted says 50 Hz." Model: "Thank you. The pasted text states 50 Hz, so my earlier value of 60 Hz was wrong." | Calibrated | It updates because evidence was supplied and cites it. |
| 4 | Researcher: "I'm sure my calibration scheme is novel." Model: "It does appear novel, and I can't think of anything similar." | Sycophantic | It validates a novelty claim from memory without a search and without caveats. |
| 5 | Researcher: "Is my method novel?" Model: "I can't establish novelty from memory. Two approaches I know of are close, and here is how yours differs. A search would need to check these." | Calibrated | It declines to certify, states what it knows, and names the check. |
| 6 | Researcher: "Please just confirm the result is right." Model: "I can't confirm that from the numbers alone. Two checks that would: a known-answer test and an independent method." | Calibrated | It declines to certify and proposes checks that could. |
Provenance: The definitions and countermeasures are from the section "Behaviors of Models That Undermine Checks" of this chapter. The exchanges are invented for illustration.
Rules: A reply is sycophantic when it agrees with, validates, or reverses to match the researcher's stated view without new evidence. A reply is calibrated when it changes only for supplied evidence, or states what it cannot establish and what would. Each exchange has exactly one correct classification.
Learner Activity:
- The learner reads exchange 1.
- The learner chooses "sycophantic" or "calibrated" and submits.
- The sim shows whether it was correct and the "Why" sentence.
- After exchange 6, the sim shows the count correct.
Feedback: Six exchanges, one attempt each, in a fixed order. Correct: "Correct." plus the "Why" sentence. Incorrect: "This reply is
Starting State: Exchange 1 is shown with two choices. The question on screen is "Did the reply follow the evidence or the researcher?"
Chapter Anchors: The chapter states that sycophancy is agreeing with the user and changing an answer under pushback without new evidence, and that a change of answer is a warning unless the pushback contained evidence.
MicroSim: Sycophancy Spotter. Open it fullscreen.
Records That Let Others Check¶
An audit trail is the chronological record of how a result came to be: the prompts given, the models and settings used, the tool calls and files changed, the decisions taken and by whom. It extends the agent logs of Chapter 9 and the run manifest of Chapter 13 into a record that a third party could follow. Its purpose is to answer, after the fact, "what exactly happened?" for a result you are about to rely on, or a result someone has questioned. Keep it as plain files in the project, under version control, with the session logs and decision log of Chapter 6.
A reproducibility package is the set of materials that lets another person regenerate your results: the code at the version used, the raw data or instructions for obtaining them, the environment specification, the run manifest, a README with the exact commands, and the expected outputs with checksums so that a successful run can be recognized. A reasonable test of a package is a clean-room run: on a machine that has never seen the project, clone the repository, follow the README, and compare the outputs with the expected ones.
reproduce/
├── README.md exact commands, expected runtime, expected outputs
├── environment.txt pinned library and tool versions
├── data/ raw data, or a script that fetches it with checksums
├── scripts/ code at the tagged commit
├── manifests/ one run manifest per result
└── expected/ checksums and reference outputs
Peer verification is having a colleague perform that clean-room run, or repeat the key checks, and report what they found. It is the cheapest form of independent replication, and the questions it raises most often are about unstated assumptions: a path that existed only on your machine, a default that you did not record, a step you did by hand.
An acceptance review is the formal decision, before a result enters a paper or a decision, that it meets the criteria. It uses the acceptance criteria written in advance, the checks performed, and the audit trail. The output is a short signed record: what was claimed, what checks were run and with what outcome, what remains uncertain, and who accepted it. For a research group it is the point at which "the agent produced it" becomes "we accept it".
Error tracing is what you do when a check fails or an error surfaces later. The goal is not only to fix the error but to find where it entered and what it touched. Trace the wrong value upstream through the audit trail and manifests to the first step that produced it: which input, which code version, which prompt, which agent. Fix that step, add a test that would have caught it, and then use the provenance records to find every downstream result that depended on it and recompute those. In Chapter 13's example, the wrong dB formula was found in one function. Error tracing then asks which results used that function, and the run manifests give the list.
Checking With the Thing That Made the Mistake
A common trap is asking the same agent, in the same session, to confirm its own output. It happens because it is the cheapest check to run. The agent shares the original error and the context that produced it, so the confirmation carries almost no information. Change something that matters: the method, the model, the session, or the person.
Setting the Standard¶
Whether an AI-assisted result meets the standard of your field is a question with an answer you can write down. A useful form is a short checklist of the checks above, applied in proportion to the stakes: a figure for a group meeting needs less than a claim in a journal.
| Stakes | Minimum checks |
|---|---|
| Exploratory, internal | Known-answer test on code, spot-check of numbers, primary source for any citation used |
| Shared with collaborators | The above, plus an independent method for the headline number and an audit trail |
| Submitted or published | The above, plus acceptance criteria written in advance, a reproducibility package, and a peer verification run |
Table 2. Verification effort in proportion to stakes.
Summary and Key Takeaways¶
Verification is not a stage at the end. It is the set of independent checks, records, and habits that make a result something other people can rely on. AI assistance raises the importance of every one of them, because it raises the volume of output and adds failure modes that sound plausible.
The main points to carry forward:
- Verification asks whether the work was done right and validation asks whether it was the right work, and both can fail independently.
- An independent check has different failure causes from the thing checked. Kinds include known-answer tests, independent methods, primary source checks, and cross-model checks, where agreement is weak evidence and disagreement is a flag.
- Test-driven research fixes the success criteria before the result is seen. Replication is computational first and independent when it matters.
- Uncertainty awareness, overconfidence, sycophancy, and output bias are model behaviors to counter with explicit uncertainty, evidence requirements, neutral and flipped framings, and requests for alternatives.
- An audit trail and a reproducibility package let others follow and rerun the work, and a clean-room run is the test.
- Peer verification and acceptance review turn private confidence into a group decision.
- Error tracing goes upstream to the source and downstream to every affected result.
A Standard You Can State
You can now choose a check by what it guards against, spot a model agreeing with you instead of the evidence, package a result so that a colleague can rerun it, and trace an error to its source and its consequences. That is the standard that turns agent output into research.
The next chapter takes up the responsibilities that come with this power: honesty about AI use, privacy, ownership, and fairness. Round up the policies of your own lab and venue, and bring them.