Skip to content

title: 2-D Convolution and Image Filtering description: Slide a 3×3 or 5×5 kernel over a small grayscale image and see every multiply-add: blur, sharpen, Sobel and Laplacian kernels, padding, flipping and noise. image: /sims/image-convolution/image-convolution.png og:image: /sims/image-convolution/image-convolution.png twitter:image: /sims/image-convolution/image-convolution.png social: cards: false quality_score: 100

2-D Convolution and Image Filtering

Copy this iframe to your website:

1
<iframe src="https://dmccreary.github.io/signal-processing/sims/image-convolution/main.html" width="100%" height="674px" scrolling="no"></iframe>

Run MicroSim in Fullscreen

Description

This MicroSim extends the 1-D Convolution Visualizer to images. The input \(x[m, n]\) is a 32 × 32 grayscale image built in code (0 = black, 255 = white): a bright square, a disc, a single bright pixel (an impulse), a one-pixel line and a horizontal ramp. Row \(m\) counts down and column \(n\) counts right. A kernel \(h[i, j]\), with \(i\) and \(j\) from \(-R\) to \(R\), produces

\[y[m, n] = \frac{1}{D}\sum_{i=-R}^{R}\sum_{j=-R}^{R} h[i, j]\,x[m - i, n - j].\]

The panels show the input with a 12 × 12 crop outlined, a zoom on the crop with the gold kernel window, the crop's output filling in as the window slides, and the full output. Below them, the rotated kernel, the pixels under the window and their products are laid out cell by cell, with the sum, the divisor \(D\) and \(y[m, n]\).

  • Kernel chooses identity, box blur 3×3 or 5×5, Gaussian, sharpen, Sobel x or y, Laplacian, emboss, or a custom 3 × 3 kernel typed into the grid (decimals or fractions such as 1/9). Editing a preset switches to Custom.
  • Play and Step move the window; so do clicks on any pixel and the arrow keys.
  • Edge padding repeats the border pixels instead of padding with zeros.
  • Normalise sets \(D\) to the sum of the weights (when that sum is not 0).
  • Flip kernel off computes the cross-correlation \(\sum h[i, j]\,x[m + i, n + j]\), as CNN layers do.
  • Add noise adds fixed Gaussian noise (\(\sigma = 12\)) and reports the noise level in and out.
  • Output display maps \(y\) to gray: clipped to 0–255, signed (0 = mid-gray) or \(|y|\); auto picks signed when the weights sum to 0.

This MicroSim is used in Chapter 15: Signal Processing Applications and AI Integration.

Lesson Plan

Learning Objectives

After using this MicroSim, students will be able to:

  1. Calculate the output of a 3 × 3 kernel at one pixel by rotating the kernel, multiplying it with the neighbourhood and adding the products
  2. Predict the output of a kernel on a flat region from the sum of its weights, and explain why blur kernels are normalised
  3. Distinguish convolution from cross-correlation and identify the kernels for which the difference changes the result
  4. Compare how smoothing and derivative kernels respond to edges, thin lines, ramps and noise, using \(\sigma_{out} \approx \sigma_{in}\sqrt{\sum h^2}\)
  5. Evaluate the artifacts that zero padding and edge padding create at the image border

Target Audience

Undergraduate students in a digital signal processing or image processing course (junior or senior year). Plan 25 minutes for the activities below.

Prerequisites

  • 1-D discrete convolution and the impulse response (Chapter 4)
  • A grayscale image as a 2-D array of numbers
  • Summation notation

Activities

Activity 1: One multiply-add by hand (6 min). Choose Box blur 3×3 and keep the default pixel (7, 13), just right of the square. The window holds 200 in its left column and 40 in the other six cells, so \(\Sigma = 3 \cdot 200 + 6 \cdot 40 = 840\), \(D = 9\) and \(y[7, 13] = 93.33\) (gray 93). Choose Gaussian 3×3: \(\Sigma = 4 \cdot 200 + 8 \cdot 40 + 4 \cdot 40 = 1280\) and \(y = 1280/16 = 80\). Go back to the box blur and uncheck Normalise: \(y[7, 13] = 840\), the flat background becomes \(9 \cdot 40 = 360\), almost everything turns white, and the status line reports \(y\) from 16 to 2160. The sum of the weights is the gain on flat regions.

Activity 2: Why convolution flips the kernel (7 min). Return to Sobel x with Flip kernel checked. The grid shows \(h\) with −1, −2, −1 on the left, but the arithmetic panel uses it rotated by 180°, so \(y[7, 13] = 4 \cdot 200 - 4 \cdot 40 = 640\). Uncheck Flip kernel: \(y = -640\). Check it again and click (7, 18), just right of the impulse at (7, 17), which is \(255 - 40 = 215\) above the background: \(y = 215 \cdot 2 = 430\). Click the eight other pixels around the impulse: the outputs are −215, 0, 215 / −430, 0, 430 / −215, 0, 215, which is \(215\,h\) exactly as typed in the grid. Without the flip the pattern is reversed. Convolving an impulse returns the kernel itself, which is why \(h\) is called the impulse response.

Activity 3: Edges, lines and ramps (6 min). Choose Sobel y and click (16, 10), (17, 10) and (18, 10), above, on and below the thin line: \(y\) = −640, 0 and 640. A line gives two responses of opposite sign. Choose Sobel x: \(y[17, 10] = 0\): along a horizontal line Sobel x responds only near the line's ends. Click the ramp at (26, 15), where the pixels to the left and right are 112 and 128: Sobel x gives \(4 \cdot 112 - 4 \cdot 128 = -64\), while Laplacian gives 0, because the second difference of a straight ramp is zero.

Activity 4: Borders and noise (6 min). Choose Box blur 3×3 and click the top-left pixel (0, 0). Five cells of the window lie outside the image (red dashed outlines) and read 0, so \(\Sigma = 160\) and \(y = 17.78\); the output gets a dark frame. Check Edge padding: those cells read 40 and \(y = 40\). With Sobel x and zero padding the largest output, 960, is not an edge at all but the right border of the ramp, \(4 \cdot 240\); edge padding lowers the maximum to 640. Finally uncheck Edge padding and check Add noise. The status line reports \(\sigma\) = 11.7 at the input and, at the output, 11.7 (identity), 3.8 (box 3×3), 4.2 (Gaussian), 2.3 (box 5×5), 38.7 (Sobel x) and 53.8 (Laplacian). For white noise the prediction \(\sigma_{in}\sqrt{\sum h^2}\) gives 3.9, 4.4, 2.3, 40.5 and 52.3: smoothing kernels suppress noise and derivative kernels amplify it.

Assessment

  1. With the Gaussian 3×3 kernel, pixel (7, 12) has 200 in the left and middle columns of its window and 40 in the right column. What is \(y[7, 12]\)? Answer: \(\Sigma = 4 \cdot 200 + 8 \cdot 200 + 4 \cdot 40 = 2560\), so \(y = 2560/16 = 160\), as the sim shows.
  2. At the left edge of the square, pixel (7, 3) has 40 | 200 | 200 in every row of its window. What does Sobel x give with convolution, and with correlation? Answer: convolution uses the rotated kernel, \(4 \cdot 40 - 4 \cdot 200 = -640\); correlation gives \(+640\).
  3. Why is the Laplacian zero everywhere inside the ramp, while Sobel x is not? Answer: on \(x = 8n\) the second differences \(x[n-1] - 2x[n] + x[n+1]\) vanish in both directions, but the first difference does not: Sobel x gives \(4(x[n-1] - x[n+1]) = -64\).
  4. A video is a sequence of frames. How many multiplications per second does a 3 × 3 kernel need on 1920 × 1080 video at 30 frames per second, and how many if the kernel is separable? Answer: \(1920 \cdot 1080 \cdot 9 \cdot 30 = 559{,}872{,}000 \approx 5.6 \times 10^8\). A separable kernel such as the Gaussian, \([1\;2\;1]^T[1\;2\;1]\), needs 6 per pixel: \(3.7 \times 10^8\).
  5. CNN layers compute cross-correlation, not convolution. Does that matter? Answer: not for learned kernels, since learning \(h\) for correlation is the same as learning the rotated \(h\) for convolution. It matters when a fixed kernel such as Sobel x is copied between conventions: the output changes sign.

Common Misconceptions

"Convolution slides the kernel as it is written." True convolution rotates the kernel by 180° first. For the symmetric kernels (box, Gaussian, sharpen, Laplacian) the panel header reads "(= h)" and nothing changes, but for Sobel x the value at (7, 13) changes from 640 to −640 when Flip kernel is unchecked.

"A black output pixel means no response." Set Output display to clip to 0–255 with Sobel x: the left edge of the square (\(y = -640\)) and the flat background (\(y = 0\)) are both black. The signed display shows them as gray 43 and gray 128, and the right edge (\(y = 640\)) as gray 213.

References

  1. Kernel (image processing) - Wikipedia - common 3 × 3 kernels, the convolution sum, edge handling and normalisation
  2. Sobel operator - Wikipedia - the Sobel kernels and the sign conventions that follow from flipping the kernel
  3. Gaussian blur - Wikipedia - Gaussian smoothing and why the kernel is separable
  4. Discrete Laplace operator - Wikipedia - the 5-point Laplacian kernel as a second difference
  5. Cross-correlation - Wikipedia - how correlation differs from convolution by the reflection of one signal
  6. Convolutional neural network - Wikipedia - convolutional layers, which apply learned kernels by cross-correlation
  7. Chapter 24: Linear Image Processing - Steven W. Smith, The Scientist and Engineer's Guide to Digital Signal Processing - free chapter on 2-D convolution, separability and image filters