Skip to content

Probability Distribution Explorer

Copy this iframe to your website:

1
<iframe src="https://dmccreary.github.io/signal-processing/sims/distribution-explorer/main.html" width="100%" height="642px" scrolling="no"></iframe>

Run MicroSim in Fullscreen

Description

This MicroSim draws random samples from the distributions used most often in signal processing and compares them with the theory. The drop-down list selects uniform \(U(a, b)\), Gaussian \(N(\mu, \sigma^2)\), exponential \(\mathrm{Exp}(\lambda)\), binomial \(B(n, p)\) (Bernoulli when \(n = 1\)), Poisson \(\mathrm{Pois}(\lambda)\), or the central-limit mode, in which every sample is the sum \(S = U_1 + \dots + U_n\) of \(n\) independent \(U(0, 1)\) values. The two parameter sliders change with the distribution.

The plot shows the theoretical PDF (or the PMF of a discrete distribution) in maroon and a blue histogram of the first \(N\) samples, with \(N\) from 10 to 10000. Continuous histograms are density-scaled with bin width \(h = 3.49\,\sigma N^{-1/3}\) (Scott's rule). Show CDF switches to the cumulative distribution \(F(x) = P(X \le x)\) and the empirical CDF of the samples. The gold band marks \(\mu \pm k\sigma\) (slider Band k) and is labelled with the probability it holds,

\[P(\mu - k\sigma \le X \le \mu + k\sigma) = F(\mu + k\sigma) - F(\mu - k\sigma).\]

Gaussian overlay adds, in green, the Gaussian with the same mean and variance. The table compares the theoretical mean, variance and standard deviation with the sample mean \(\bar{x}\), the sample variance \(s^2 = \frac{1}{N-1}\sum_i (x_i - \bar{x})^2\) and \(s\), and the band probability with the fraction of samples in the band. It also shows the standard error of the mean \(\sigma/\sqrt{N}\).

All samples come from a seeded generator (Mulberry32 uniforms, the Box–Muller transform for Gaussians, the inverse CDF for exponentials), so the page always opens with the same samples (seed 1). New samples moves to the next seed. Raising \(N\) keeps the first samples and adds new ones.

This simulation is used in Chapter 1: Mathematical Foundations.

Lesson Plan

Learning Objectives

After using this MicroSim, students will be able to:

  1. Explain the difference between a PDF or PMF and a CDF, and compute an interval probability as \(F(b) - F(a)\)
  2. Calculate the mean, variance and standard deviation of the uniform, Gaussian, exponential, binomial and Poisson distributions from their parameters
  3. Apply the 68–95–99.7 rule to Gaussian signals and compare it with the band probabilities of non-Gaussian distributions
  4. Describe how the sample mean and variance approach the true values as \(N\) grows, using the standard error \(\sigma/\sqrt{N}\)
  5. Explain the central limit theorem by observing that a sum of \(n\) uniform variables approaches a Gaussian with mean \(n/2\) and variance \(n/12\)

Target Audience

Undergraduate students in a first signal processing, signals and systems, or engineering probability course. Plan 30–40 minutes for the four activities.

Prerequisites

  • Integrals and sums as areas
  • The definitions of mean, variance and standard deviation (Chapter 1)
  • Reading a histogram

Activities

All sample values below are what the MicroSim shows for seed 1, the samples drawn when the page loads. After New samples the sample column changes; the theory column does not.

Activity 1: The 68–95–99.7 rule (8 min). Keep the Gaussian with \(\mu = 0.0\) and \(\sigma = 1.00\) and set \(N = 10000\). Step Band k through 1, 2 and 3. The theoretical probabilities are 0.683, 0.954 and 0.997, and the fractions of samples in the band are 0.684, 0.956 and 0.998. Now set \(\mu = 1.0\) and \(\sigma = 2.00\): the band at \(k = 1\) moves to \([-1, 3]\), but its probability is still 0.683 (and 0.954 at \(k = 2\)), because the probability depends only on \(k\). Check Show CDF with \(k = 1\): \(F(\mu - \sigma) = 0.159\) and \(F(\mu + \sigma) = 0.841\), so \(\Delta F = 0.683\), the same area.

Activity 2: Sample statistics converge (8 min). Return to \(\mu = 0.0\), \(\sigma = 1.00\), \(k = 1\), and step \(N\):

\(N\) Sample mean Sample variance Fraction in \(\mu \pm \sigma\) Standard error \(\sigma/\sqrt{N}\)
10 −0.285 1.473 0.500 0.3162
100 0.210 0.980 0.690 0.1000
1000 −0.029 0.968 0.689 0.0316
10000 0.016 0.998 0.684 0.0100

The error of the sample mean shrinks roughly like \(\sigma/\sqrt{N}\). At \(N = 10\) press New samples a few times and watch how far the sample mean and the histogram move from one seed to the next.

Activity 3: Same mean, different shape (10 min). Use \(N = 10000\).

  1. Exponential, \(\lambda = 1.00\): mean and standard deviation are both 1.000 (sample 0.995 and 0.985). The band holds 0.865, 0.950 and 0.982 for \(k\) = 1, 2 and 3, not 0.683, 0.954 and 0.997: the exponential is skewed, with a long right tail.
  2. Set \(\lambda = 0.50\): the theoretical mean becomes 2.000 and the variance 4.000 (\(1/\lambda\) and \(1/\lambda^2\)). The sample mean doubles exactly to 1.990, because the same uniform numbers are rescaled, and the band probability stays 0.865.
  3. Uniform, \(a = -2.0\), \(b = 2.0\): variance \((b - a)^2/12 = 1.333\) (sample 1.327). The band holds only 0.577 at \(k = 1\) and 1.000 at \(k = 2\), because \(\mu \pm 2\sigma = \pm 2.309\) covers the whole support.
  4. Poisson, \(\lambda = 4.0\): mean and variance are both 4.000 (sample 3.992 and 4.007). The band \([2, 6]\) holds 0.798 of the probability.
  5. Binomial, \(n = 1\), \(p = 0.30\) (Bernoulli): mean \(p = 0.300\) and variance \(p(1 - p) = 0.210\) (sample 0.297 and 0.209).

Activity 4: The central limit theorem (10 min). Choose CLT: sum of n uniforms, check Gaussian overlay, and set \(N = 10000\), \(k = 1\). Step Terms n and record the read-outs:

\(n\) \(\mathrm{Var}(S) = n/12\) \(P(\mu \pm \sigma)\), theory Sample fraction Largest gap between \(F\) and \(\Phi\)
1 0.083 0.577 0.584 0.0572
2 0.167 0.650 0.659 0.0164
3 0.250 0.667 0.670 0.0099
6 0.500 0.674 0.676 0.0048
12 1.000 0.679 0.681 0.0023
30 2.500 0.681 0.684 0.0009

The histogram is flat for \(n = 1\), triangular for \(n = 2\) and already bell-shaped for \(n = 3\). The band probability climbs toward the Gaussian 0.683, and from \(n = 3\) on the largest gap between the exact CDF and the Gaussian CDF roughly halves each time \(n\) doubles (0.0099 at \(n = 3\), 0.0048 at \(n = 6\), 0.0023 at \(n = 12\)). With \(n = 12\) and \(k = 2\) the band holds 0.955, against 0.954 for a Gaussian.

Assessment

  1. Thermal noise is Gaussian with \(\mu = 0\) and \(\sigma = 2\) mV. What fraction of noise samples lies between −4 and +4 mV? Answer: ±4 mV is \(\mu \pm 2\sigma\), so 0.954 (95.4%); the MicroSim shows 0.954 for any \(\sigma\) when \(k = 2\).
  2. Waiting times are exponential with a mean of 2 s. What are \(\lambda\), the variance and the probability that a wait lies within one standard deviation of the mean? Answer: \(\lambda = 0.5\) per second, variance \(1/\lambda^2 = 4\) s², \(\sigma = 2\) s; the band \([0, 4]\) s holds \(1 - e^{-2} = 0.865\).
  3. For a Poisson variable with \(\lambda = 4\), the band \(\mu \pm \sigma\) holds 0.798 instead of 0.683. Give two reasons. Answer: the band is the closed interval \([2, 6]\) and \(X\) takes only integer values, so both end points count in full (\(P(X = 2) = 0.147\), \(P(X = 6) = 0.104\)); and the Poisson distribution is not Gaussian (it is discrete and skewed).
  4. \(S\) is the sum of 12 independent \(U(0, 1)\) values. Give \(E[S]\) and \(\mathrm{Var}(S)\), and explain why \(S - 6\) is a quick approximate standard Gaussian. Answer: \(E[S] = 12 \times 0.5 = 6\) and \(\mathrm{Var}(S) = 12 \times \tfrac{1}{12} = 1\), so \(S - 6\) has mean 0 and variance 1, and by the central limit theorem its CDF is within 0.0023 of \(\Phi\).
  5. With \(N = 100\) the sample mean of a \(N(0, 1)\) source was 0.210. Is the generator biased? Answer: not necessarily. The standard error is \(\sigma/\sqrt{N} = 0.100\), so 0.210 is 2.1 standard errors from 0, unusual but possible; at \(N = 1000\) and \(N = 10000\) the sample means are −0.029 and 0.016, within 1 and 1.6 standard errors.

Common Misconceptions

"68% of the values always lie within one standard deviation of the mean." That holds only for the Gaussian. The MicroSim shows 0.577 for the uniform distribution, 0.865 for the exponential and 0.798 for Poisson with \(\lambda = 4\). The sum of \(n\) uniforms moves toward 0.683 only as \(n\) grows, as the central limit theorem predicts.

"The sample mean and variance are the mean and variance of the distribution." They are estimates. With \(N = 10\) Gaussian samples (\(\mu = 0\), \(\sigma = 1\)) the sample mean is −0.285 and the sample variance 1.473, and New samples changes both. Only as \(N\) grows do they settle near the true values.

References

  1. Probability distribution - Wikipedia - PDFs, PMFs and CDFs of discrete and continuous random variables
  2. Normal distribution - Wikipedia - the Gaussian density, its moments and why it models thermal noise
  3. 68–95–99.7 rule - Wikipedia - the probabilities within one, two and three standard deviations of a Gaussian mean
  4. Central limit theorem - Wikipedia - why sums of independent random variables approach a Gaussian
  5. Irwin–Hall distribution - Wikipedia - the exact distribution of a sum of \(n\) uniform variables used in the CLT mode
  6. Poisson distribution - Wikipedia - counts of random events, with mean equal to variance
  7. Box–Muller transform - Wikipedia - how the sketch turns uniform random numbers into Gaussian ones
  8. The Scientist and Engineer's Guide to Digital Signal Processing - Steven W. Smith - free online textbook; its chapter on statistics, probability and noise covers histograms, the PDF and the Gaussian