Skip to content

References: Brainstorming and Problem Framing

  1. Brainstorming - Wikipedia - Covers Alex Osborn's group idea-generation method, its rules on deferring judgment and seeking quantity, and variations. Supports the chapter's separation of divergent generation from convergent evaluation.

  2. Hypothesis - Wikipedia - Explains what a scientific hypothesis is, working hypotheses, falsifiability, and statistical testing. Directly supports the chapter's rule that a hypothesis must name a change, effect size, and conditions that a result could contradict.

  3. Design of experiments - Wikipedia - Presents independent and dependent variables, randomization, blocking, replication, and a worked weighing example. Supports the chapter's treatment of baselines, controls, and fixing the support criterion before data are seen.

  4. The Craft of Research (4th Edition) - Wayne C. Booth, Gregory G. Colomb, Joseph M. Williams - University of Chicago Press - Moves a researcher from topic to question to significance using the "I am studying X because I want to find out Y in order to help my reader understand Z" sentence template, a concrete device for problem framing.

  5. How to Solve It (2nd Edition) - George Polya - Princeton University Press - Gives a four-phase problem-solving method and a question list ("Have you seen it before?", "Could you solve a related problem?") that made analogy and restating the problem teachable heuristics, clarifying analogical thinking.

  6. Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers - arXiv (Si, Yang, Hashimoto) - Blind study in which expert reviewers rated LLM-generated research ideas as more novel but slightly less feasible than expert ideas. Evidence for why ranking, feasibility, and novelty checks matter.

  7. Towards Understanding Sycophancy in Language Models - arXiv (Sharma et al., Anthropic) - Shows that AI assistants tend to agree with users, partly because human preference data rewards agreement. Explains why the chapter demands explicit devil's advocate prompts rather than neutral requests for opinion.

  8. Tree of Thoughts: Deliberate Problem Solving with Large Language Models - arXiv (Yao et al.) - Introduces a framework where a model explores several reasoning paths, evaluates them, and backtracks. Illustrates the generate-then-evaluate structure behind idea generation and ranking, with benchmark results.

  9. You and Your Research - Richard W. Hamming (transcript) - Bell Labs mathematician's talk on choosing important problems, asking what the important problems in your field are, and why. A practitioner's perspective on problem framing and project selection.

  10. Prompt engineering overview - Anthropic Documentation - Official guide to defining success criteria, writing prompts, and iterating, with links to best practices and an interactive tutorial. Supports the structured idea-generation prompts and explicit output formats used in this chapter.