Token Prediction Analysis
Which factor best predicts how many tokens are needed to generate a chapter? This interactive MicroSim compares three potential predictors: number of concepts, word count, and number of MicroSims.
Regression Comparison Results
| Predictor | R² | Interpretation |
|---|---|---|
| Concepts | 1.6% | Very weak predictor |
| Words | 0.3% | Essentially no relationship |
| MicroSims | 5.6% | Weak but best of the three |
Key Finding
None of these variables are good predictors of token usage.
Even the best predictor (MicroSims) explains only 5.6% of the variance. This means ~94% of the variation in token usage is explained by other factors not captured in these metrics.
What This Tells Us
The analysis reveals that token consumption during chapter generation is driven by factors beyond simple metrics:
- Context loading overhead - The skill loads course descriptions, learning graphs, and reference files regardless of chapter size
- Agent reasoning complexity - Some topics require more "thinking" even with fewer concepts
- Example generation - Worked examples vary in complexity independent of concept count
- Parallel vs sequential execution - Agent behavior differs based on execution mode
Interesting Observations
Looking at specific outliers:
- Chapter 9 (Probability Fundamentals): 69,200 tokens for 19 concepts and 5,591 words - highest token usage
- Chapter 17 (Inference for Means): Only 25,000 tokens for 18 concepts and 7,028 words - very efficient
- Chapter 10 (Conditional Probability): 55,700 tokens for just 5 concepts - highest tokens-per-concept ratio
Features
- Interactive toggle - Switch between Concepts, Words, and MicroSims views
- Side-by-side R² comparison - See all three R² values at once
- Dynamic regression line - Updates with each predictor selection
- Rich tooltips - Hover over points to see all chapter metrics
Data Source
Data extracted from chapter generation logs (/logs/ch-*.md) for chapters 6-19, excluding chapters with estimated values.