Audio Signal Processing, Sound Classification, and Speech I/O¶

Summary¶

This chapter covers real-time sound recognition on the Raspberry Pi 5. It explains how microphones (including microphone arrays and USB microphones) capture analog sound as a digital audio signal, and how the Fast Fourier Transform converts that signal into a frequency spectrum and spectrogram for analysis, along with sampling rate, audio buffering, and noise-floor/signal-to-noise considerations. Building on that signal-processing foundation, it covers sound and keyword classification, wake word detection, and voice commands using audio feature extraction techniques such as mel-frequency cepstrum coefficients. The chapter then covers speech-to-text and text-to-speech, general sound event and environmental sound classification, and the I2S audio interface and DAC/amplifier/speaker hardware used for audio output, closing with the privacy considerations of always-listening microphones and the tradeoffs between on-device and cloud audio processing. Students finishing this chapter will be able to set up a Pi 5 with a microphone to run a pretrained real-time sound or keyword recognition demo.

Concepts Covered¶

This chapter covers the following 34 concepts from the learning graph:

Microphone Array
USB Microphone
Audio Sampling Rate
Analog Audio Signal
Digital Audio Signal
Audio Buffer
Fast Fourier Transform
Frequency Spectrum
Spectrogram
Sound Classification
Keyword Spotting
Wake Word Detection
Voice Command
Audio Feature Extraction
Mel Frequency Cepstrum
Noise Floor
Signal To Noise Ratio
Audio Preprocessing Pipeline
Text To Speech
Speech To Text
Audio Model Inference
Sound Event Detection
Environmental Sound Class
Alert Sound Trigger
Audio Latency
I2S Audio Interface
PCM5102 DAC
Speaker Output
Amplifier Module
Audio Confidence Score
Multi Modal Recognition
Audio Privacy Considerations
On Device Processing
Cloud Versus Edge Processing

Prerequisites¶

This chapter builds on concepts from:

Time to Listen In

Berry waving welcome Chapter 17 taught a Pi 5 and AI HAT+ to see — bounding boxes, confidence scores, real-time inference on camera frames. This chapter teaches the same board to listen. Same accelerator, same inference vocabulary, one brand-new signal: sound. Let's build something!

Every microphone in this chapter's projects does the same basic job a distance sensor did back in Chapter 6: turning a physical quantity into a number a program can read. The difference is speed and volume. A photoresistor might report a light level a few times a second; a microphone reports tens of thousands of numbers every second, each one a tiny snapshot of air pressure. Turning that flood of numbers into "wake word detected" or "smoke alarm heard" is the whole story of this chapter — a sound-specific version of the sense-think-act cycle from Chapter 1, and a close cousin of the image-recognition pipeline from Chapter 17.

From Sound Wave to Digital Signal¶

A sound wave is a rapid pattern of air pressure pushing against a microphone's diaphragm, thousands of times each second. The diaphragm's motion produces an analog audio signal: a voltage that rises and falls continuously, with no jumps or gaps, tracing the exact shape of the pressure wave outside. That's the same kind of continuous signal a photoresistor produced back in Chapter 6, just changing far faster — a light level might drift over a full second, while a mid-pitch musical note completes hundreds of full cycles in that same second.

A microcontroller or computer cannot store a continuously varying voltage; it can only store numbers. Converting the analog signal into a digital audio signal — a sequence of discrete numeric values, each one a snapshot of the voltage at a specific instant — is the same analog-to-digital conversion introduced for the photoresistor in Chapter 6, just running far more often. How often it runs is set by the audio sampling rate: the number of snapshots captured per second, measured in hertz (Hz). A typical speech-recognition pipeline on the Pi 5 samples at 16,000 Hz (16 kHz); music applications often use 44,100 Hz or 48,000 Hz.

Sampling rate matters because of a hard rule from signal processing: to faithfully capture a sound at a given frequency, the sampling rate must be at least twice that frequency, written \( f_s \geq 2f_{max} \). Sample too slowly and higher-pitched sounds get digitized as the wrong pitch entirely, an effect called aliasing — which is why a wake-word model trained on 16 kHz audio performs poorly if it's accidentally fed audio sampled at only 8 kHz. Because a program cannot usefully react to one sample at a time, samples are collected into an audio buffer: a short block of consecutive samples, typically representing 20 to 100 milliseconds of sound, that a program reads and processes together as a single chunk before moving on to the next block.

Berry's Key Insight

Berry thinking Sampling rate is a budget, not a suggestion. Every doubling buys twice the pitch range but also doubles the data a Pi 5 has to chew through every second. Most speech projects in this book run happily at 16 kHz — no need to record at 48 kHz "just in case" when a program only needs to hear a spoken word.

Before looking at the interactive version of this idea, it helps to actually watch a sampling rate that's too low distort a waveform, rather than just reading about it.

Diagram: Audio Sampling Rate and Aliasing Explorer¶

Run the Audio Sampling Rate and Aliasing Explorer MicroSim fullscreen

Audio Sampling Rate and Aliasing Explorer (MicroSim)

Type: microsim sim-id: audio-sampling-aliasing-explorer
Library: p5.js
Status: Specified

Learning objective: Students will apply (Bloom L3: Apply) the relationship between tone frequency and sampling rate to predict when a digitized signal will show aliasing distortion.

Canvas: 700x450px default, responsive — recompute plot width from windowWidth in windowResized(), keeping a minimum readable height of 350px on narrow screens.

Layout: two stacked waveform plots. The top plot draws a smooth continuous sine wave representing the analog audio signal at the current tone frequency. The bottom plot draws the same time window but shows only the discrete sample points (small filled circles) taken at the current sampling rate, connected with straight line segments to show what a reconstructed digital signal actually looks like. A text readout above the plots shows the current tone frequency, sampling rate, and the computed Nyquist minimum (\( 2 \times \) tone frequency).

Controls: a createSlider() labeled "Tone Frequency (Hz)" (range 100-2000, default 440); a createSlider() labeled "Sampling Rate (Hz)" (range 200-4000, default 1000); a createButton() labeled "Play Tone" that toggles playback of the current tone using p5.sound's p5.Oscillator, muting automatically after 2 seconds.

Interaction: whenever the sampling rate slider drops below twice the tone frequency, the reconstructed waveform in the bottom plot visibly warps into a different, lower apparent frequency (aliasing), and a warning banner appears reading "Aliasing! Sampling rate is below the Nyquist minimum of [2x tone frequency] Hz for this tone." The banner disappears automatically once the sampling rate slider is moved back above that minimum. Clicking any sample point in the bottom plot shows a small tooltip with that sample's exact time and amplitude value.

Implementation: p5.js. Compute the continuous sine wave analytically each frame; compute sample points by evaluating the same sine function only at intervals of 1 / samplingRate. Redraw both plots fully each frame from current slider values so the display always reflects live parameter changes. Parent the canvas to the enclosing <div> and recompute plot dimensions inside windowResized().

Microphone Hardware and Signal Quality¶

The physical microphone captures that analog signal in the first place, and the projects in this book use two different kinds. A USB microphone is a self-contained microphone with its own built-in analog-to-digital converter, connecting to the Pi 5 over USB and appearing to Linux as a ready-to-use digital audio source — the simplest option to wire up, since there's no separate ADC or driver configuration to manage. A microphone array is two or more microphone elements mounted together on a single board at a known spacing, so software can compare tiny timing differences between the elements to estimate which direction a sound came from — a capability a single USB microphone cannot offer, since direction requires at least two listening points to triangulate from.

Every microphone, and every room it sits in, also picks up sound nobody wants: the hum of a computer fan, traffic outside, the electrical hiss inherent in any analog circuit. The noise floor is the baseline level of that unwanted background sound, measured when nothing meaningful is happening — the quietest a recording ever actually gets in a given environment. The signal-to-noise ratio (SNR) compares how much louder the sound a project cares about is compared to that noise floor, usually expressed in decibels. A high SNR means a wake word stands out clearly above the background hiss; a low SNR means a classifier has to work much harder to tell speech apart from noise, and often makes more mistakes doing it.

Now that both microphone types and both signal-quality terms are defined, the table below sorts them by what matters most when choosing hardware for a project.

Feature	USB Microphone	Microphone Array
Number of elements	One	Two or more
Direction sensing	No	Yes, via timing differences between elements
Setup complexity	Plug-and-play over USB	Slightly more driver and configuration work
Typical use in this book	Single-speaker wake word or voice command demos	Noisier rooms, or projects that need to sense direction

Berry's Tip

Berry sharing a tip A five-dollar move that beats a fancier microphone every time: move the mic closer to the sound source and away from fans and speakers. Cutting the distance in half roughly doubles how loud your signal is relative to the room's noise floor — free signal-to-noise ratio, no code required.

Seeing Sound: The Frequency Domain¶

A raw digital audio signal — sometimes called a waveform — shows loudness changing over time, but it hides an important detail: which pitches are present at any given moment. The Fast Fourier Transform (FFT) is an algorithm that converts a block of time-based audio samples into a frequency spectrum: a list of how much energy the signal contains at each frequency, from low pitches to high pitches, for that one block. Where the waveform answers "how loud, right now," the frequency spectrum answers "which pitches, right now" — and most sound classification depends far more on the second question, since a dog bark and a doorbell chime can be equally loud while occupying completely different pitch ranges.

Running the FFT on one buffer produces one frequency spectrum, but sound changes over time, so most audio pipelines run the FFT on a new buffer many times per second and stack the results side by side. That stack of spectra, arranged with time along one axis, frequency along the other, and color or brightness showing energy, is called a spectrogram — visually, it looks like a heat map of a sound's pitch content changing moment to moment, and it's the representation most sound- and speech-classification models are actually trained to recognize, rather than the raw waveform.

Berry's Key Insight

Berry thinking A spectrogram is why a trained model can tell a smoke alarm apart from a slammed door even though both are loud, sudden sounds. Loudness alone can't tell them apart — but their frequency spectrums look nothing alike. The FFT is what makes that hidden shape visible to a program.

Before exploring the interactive spectrogram below, remember its two axes: the vertical axis is frequency, taken straight from the frequency spectrum just defined, and the horizontal axis is time, with each vertical slice representing one FFT result.

Diagram: FFT Spectrogram Explorer¶

Run the FFT Spectrogram Explorer MicroSim fullscreen

FFT Spectrogram Explorer (interactive diagram)

Type: interactive-diagram sim-id: fft-spectrogram-explorer
Library: p5.js
Template: https://github.com/dmccreary/signal-processing/tree/main/docs/sims/fft-sound-file
Status: Specified

Learning objective: Students will analyze (Bloom L4: Analyze) how a waveform, a frequency spectrum, and a spectrogram represent the same sound in three different ways, and identify how sound type changes each representation.

Canvas: 700x520px, responsive — stack the three panels vertically on screens narrower than 500px and recompute panel heights on windowResized().

Layout: three stacked panels sharing the same time axis. Top panel: the raw waveform (amplitude vs. time) of the currently selected clip. Middle panel: a live bar chart of the frequency spectrum for the current playback position, one bar per frequency bin, using p5.sound's FFT.analyze(). Bottom panel: a scrolling spectrogram built by plotting each new frequency-spectrum snapshot as a new vertical color column, brightest colors indicating the most energy at that frequency.

Controls: a createSelect() dropdown labeled "Sound Clip" with four short preloaded audio options — "Pure Tone," "Dog Bark," "Speech Sample," "White Noise" — each under 3 seconds; a createButton() labeled "Play / Pause."

Interaction: hovering anywhere over the spectrogram panel draws a vertical highlight line at that time position and simultaneously redraws the frequency-spectrum bar chart above it to match that exact moment, letting a student directly compare a spectrogram column to its source spectrum. Clicking any bar in the frequency-spectrum panel shows a tooltip with that bar's exact frequency range in Hz and its magnitude value.

Implementation: p5.js with the p5.sound library's FFT object for real-time analysis during playback, and a pre-computed spectrogram buffer (array of frequency-spectrum snapshots) built once when a clip is selected so hovering can look up any past moment instantly rather than only the live playback position.

From Spectrogram to Features: Preparing Audio for a Model¶

A full spectrogram is detailed, but it's also large — feeding every raw frequency bin into a model wastes computation on pitch differences too fine for a wake-word model to actually need. Audio feature extraction is the process of condensing a spectrogram down into a smaller set of numbers that still captures what matters for classification, discarding detail that doesn't help distinguish one sound from another. The most common feature-extraction technique for speech and sound classification produces Mel-frequency cepstral coefficients (MFCCs): a set of values computed by reshaping the frequency spectrum onto the mel scale — a frequency scale that stretches and compresses pitch ranges to match how the human ear actually perceives loudness differences at low versus high pitches — and then compressing the result into a short list of numbers, typically 13 to 40 values per audio buffer.

Sampling, buffering, running the FFT, and extracting MFCCs never happen as isolated steps in a real project — they run back-to-back, automatically, on every new chunk of audio. The audio preprocessing pipeline is that full chain: microphone capture, buffering, FFT, and feature extraction, packaged so a new set of model-ready numbers comes out the other end every time a new audio buffer comes in. It plays the same role for sound that the camera-to-model pipeline from Chapter 17 plays for images: turning raw sensor data into the exact numeric shape a trained model expects.

Before looking at code, it helps to name the two parameters that show up in almost every Python audio-capture script: the sampling rate in hertz, set to match what the model was trained on, and the buffer size in samples, which sets how much audio is processed at once and therefore how quickly the pipeline can react to a new sound.

import sounddevice as sd
import numpy as np

SAMPLE_RATE = 16000   # samples per second, matches the wake-word model
BUFFER_SIZE = 512     # samples per buffer (about 32 ms at 16 kHz)

def process_audio(buffer, frames, time_info, status):
    # buffer: a NumPy array holding BUFFER_SIZE audio samples
    spectrum = np.fft.rfft(buffer[:, 0])   # Fast Fourier Transform
    magnitude = np.abs(spectrum)           # energy present at each frequency
    # magnitude now feeds into MFCC extraction and, from there, the model

stream = sd.InputStream(
    samplerate=SAMPLE_RATE,
    blocksize=BUFFER_SIZE,
    channels=1,
    callback=process_audio,
)

The sd.InputStream object keeps calling process_audio automatically, once per buffer, for as long as the stream stays open — the Python equivalent of the sense-think-act cycle running continuously in the background, without a manually written loop.

Diagram: Audio Preprocessing Pipeline Flow¶

Run the Audio Preprocessing Pipeline Flow MicroSim fullscreen

Audio Preprocessing Pipeline Flow (workflow diagram)

Type: workflow-diagram sim-id: audio-preprocessing-pipeline-flow
Library: p5.js
Template: https://github.com/dmccreary/linear-algebra/tree/main/docs/sims/ml-pipeline
Status: Specified

Learning objective: Students will analyze (Bloom L4: Analyze) the stages of an audio preprocessing pipeline from microphone capture to model inference, and identify the data format that exists at each stage boundary.

Canvas: 700x340px, responsive — stack boxes into two rows of three on screens narrower than 560px instead of one row of six.

Layout: six connected boxes drawn left to right with directional arrows between them: "Microphone Capture" → "Audio Buffer" → "FFT" → "Feature Extraction (MFCC)" → "Model Inference" → "Confidence Score / Decision." Each box uses a distinct accent color from the book's palette (raspberry #C2185B, indigo #3F51B5, circuit green #2E7D32, copper gold #D4AF37, cycling through the set).

Controls: none beyond click interaction; a small createButton() labeled "Close Info" dismisses any open infobox.

Interaction: clicking any box opens an infobox beneath the diagram showing that stage's one-sentence definition (matching the surrounding chapter prose) plus the data format at that point in the pipeline — e.g., clicking "Audio Buffer" shows "512 raw sample values"; clicking "Feature Extraction (MFCC)" shows "13-40 MFCC values." Hovering an arrow between two boxes shows a tooltip with that stage's typical time cost, pulled from the latency table later in the chapter (e.g., hovering the arrow into "FFT" shows "Milliseconds, grows with buffer size"). Only one infobox is open at a time.

Implementation: p5.js, boxes and arrows laid out from a small array of stage objects {name, dataFormat, timeCost, color} with x-position computed as a fraction of width so the layout reflows on windowResized(). Hit-testing via rectangular bounds around each box.

Running the Model: Inference, Confidence, and Latency¶

Once MFCCs (or another feature set) come out of the preprocessing pipeline, they're handed to a trained model exactly the way Chapter 16 described for images: audio model inference is running a pretrained model forward on a new set of audio features to produce a prediction, without any further training taking place. The model's output typically includes an audio confidence score — a number, usually between 0 and 1, expressing how certain the model is that its prediction is correct, the audio equivalent of the confidence score Chapter 16 introduced for object detection. A wake-word model might report a confidence score of 0.92 for "heard the wake word" on one buffer and 0.04 on the next, and a program typically only acts once that score clears a chosen threshold.

Every stage of the pipeline — capture, buffering, FFT, feature extraction, inference — takes a small amount of time, and their sum is the audio latency: the delay between a sound actually happening in the room and the program producing a result. Low latency matters most for wake-word and alert-sound projects, where even a half-second delay can feel sluggish or miss a fast-moving safety event; latency matters far less for something like periodically logging environmental sound levels once a minute.

Now that every stage has been defined, the table below organizes them by their typical cost to a project's overall audio latency.

Pipeline Stage	What It Does	Typical Time Cost
Buffering	Collect enough samples for one analysis window	Fixed by buffer size, tens of milliseconds
FFT	Convert the buffer into a frequency spectrum	A few milliseconds, growing with buffer size
Feature extraction (MFCC)	Reduce the spectrum to a compact feature set	A few milliseconds
Model inference	Run the trained model on the extracted features	A few milliseconds, faster on the AI HAT+ accelerator

Classifying Sound: Events, Environments, and Alerts¶

With inference and confidence in place, it's worth naming the different jobs a sound-recognition model can actually be trained to do, since "recognize sound" really covers several distinct tasks. Sound classification is the general task of assigning a single label to a clip of audio from a fixed set of possible categories — "dog bark," "car horn," "silence." Sound event detection goes a step further: instead of labeling an entire clip at once, it identifies exactly when a specific sound starts and stops within a longer, continuous audio stream, which matters for anything that needs to react to a sound the moment it happens rather than after the fact.

The categories a sound classifier chooses between are its environmental sound classes — labels describing everyday sounds in a space, like "footsteps," "running water," "smoke alarm," or "glass breaking," as opposed to speech or music. When one of those classes represents something a program should genuinely respond to — a smoke alarm, a doorbell, glass breaking — crossing the model's confidence threshold for that class can be wired up as an alert sound trigger: code that runs a specific action, like lighting a NeoPixel or sending a notification, the moment that class is detected with enough confidence.

The list below ties each of those four terms to a concrete example, reinforcing the definitions just given.

Sound classification example: deciding whether a short clip is speech, music, or background noise.
Sound event detection example: marking the exact moment a doorbell chime starts, somewhere inside a ten-minute recording.
Environmental sound class examples: "glass breaking," "dog barking," "smoke alarm," "running water."
Alert sound trigger example: lighting a red NeoPixel and logging a timestamp the instant "smoke alarm" clears an 0.85 confidence threshold.

Keyword Spotting, Wake Words, and Voice Commands¶

Speech gets its own specialized branch of sound classification, built around a handful of related tasks. Keyword spotting is the task of continuously listening to an audio stream and detecting whether one specific word or short phrase from a small, fixed vocabulary was just spoken, without transcribing everything else that was said. Wake word detection is the most common real-world use of keyword spotting: a lightweight model runs constantly, listening only for one particular phrase — "Hey Pi," for example — and stays otherwise silent about everything else it hears, deliberately ignoring all other speech until that one phrase triggers it.

Once a wake word has triggered the system, a voice command is a short, structured spoken instruction the program is designed to recognize and act on — "turn on the light," "stop," "what's the temperature" — typically handled by a second, larger model or process that only runs after the wake word fires, since running full command recognition continuously would waste far more processing power than most classroom projects can spare.

Berry's Tip

Berry sharing a tip Set a wake-word confidence threshold too low and your robot answers to the TV. Set it too high and it ignores you calling its name from across the room. There's no universally "correct" number — test it in the room the project will actually live in, and nudge the threshold until it's berry close to right.

Diagram: Wake Word Confidence Threshold Simulator¶

Run the Wake Word Confidence Threshold Simulator MicroSim fullscreen

Wake Word Confidence Threshold Simulator (MicroSim)

Type: microsim sim-id: wake-word-confidence-threshold-simulator
Library: p5.js
Status: Specified

Learning objective: Students will apply (Bloom L3: Apply) a chosen confidence threshold to a stream of simulated wake-word detection scores, classifying each event as a true trigger, a false accept, or a missed detection.

Canvas: 700x450px default, responsive — recompute the strip-chart width from windowWidth in windowResized().

Layout: a horizontally scrolling strip chart plotting a simulated audio confidence score (0 to 1 on the vertical axis) against time (horizontal axis) as simulated audio events stream past. A horizontal threshold line is drawn across the chart at the current slider value. A running tally box in the top-right corner shows live counts of "True Triggers," "False Accepts," and "Missed Detections."

Controls: a createSlider() labeled "Confidence Threshold" (range 0-1, step 0.01, default 0.5); a createButton() labeled "Generate New Test Run" that reseeds the simulated event stream; a createButton() labeled "Run / Pause."

Interaction: the underlying simulation generates a mix of two event types with different score distributions — true wake-word events (scores clustered high, mean around 0.8) and background noise events (scores clustered low, mean around 0.2), with some overlap so the threshold choice has real consequences. Each event crossing the threshold line lights up green if it was truly a wake-word event (true trigger) or red if it was actually background noise (false accept); a true wake-word event that stays under the threshold is marked with a small orange flag (missed detection). Clicking any past event on the strip chart reopens an infobox naming whether it was truly a wake-word or background event and showing its exact score, so a student can audit individual decisions rather than only the tally.

Implementation: p5.js, an array of event objects {time, trueLabel, score} generated from two different random distributions (randomGaussian() centered differently for wake-word versus background events). Redraw the full visible window each frame from the event array and current threshold value so moving the slider immediately re-colors past events and updates the tally.

Speech I/O: Turning Speech into Text and Text into Speech¶

Voice commands only cover a small, fixed vocabulary, but some projects need to handle open-ended spoken language instead. Speech-to-text (STT) is the process of converting a recording of spoken language into written text, using a model trained on a much larger and more general vocabulary than a keyword spotter — the same underlying skill that powers automatic captions. Text-to-speech (TTS) runs the process in reverse: converting written text into a synthesized spoken audio signal, letting a Pi 5 project talk back instead of only displaying text on an OLED screen the way the robots in Chapter 8 did.

Worth Knowing

Berry with a general note Text-to-speech isn't just a fun project feature — it's assistive technology that screen readers rely on every day for people who are blind or low-vision. Building a TTS project in this chapter means practicing the same technology behind tools people depend on daily.

Chapter 17's camera pipeline and this chapter's microphone pipeline don't have to run separately. Multi-modal recognition combines predictions from more than one type of sensor input — vision and audio, for example — to make a single, more reliable decision than either sensor could make alone. A doorbell project might combine "camera sees a person" from the Chapter 17 pipeline with "microphone hears a knock" from this chapter's sound event detection, only triggering a notification when both fire together. That combination cuts down on false alerts a single sensor would produce on its own: a delivery truck driving past sounds a little like a knock but isn't accompanied by a person standing at the door, and a person walking past isn't accompanied by a knock at all.

From Digital Signal to Sound: Audio Output Hardware¶

Text-to-speech and alert sounds both end the same way: a digital audio signal has to become real, physical sound. Getting there requires dedicated hardware, connected in a specific order. The I2S audio interface ("Inter-IC Sound") is a digital communication protocol, similar in spirit to the I2C and SPI buses used for the displays and sensors in earlier chapters, purpose-built for streaming digital audio between chips at a steady, precise rate. The Pi 5's GPIO header can carry I2S signals out to dedicated audio hardware instead of relying only on the board's built-in, lower-quality audio output.

A PCM5102 DAC ("digital-to-analog converter") is a small, inexpensive chip that receives the I2S digital audio stream and converts it back into an analog voltage — the reverse of the analog-to-digital conversion that happened back at the microphone. That analog signal is usually too weak to move a speaker cone on its own, so an amplifier module boosts the analog signal's power before it reaches the final stage: speaker output, the physical speaker whose cone vibrates to recreate the original sound wave, completing the round trip from analog signal, to digital signal, and back to analog again.

You've Got This!

Berry encouraging you Hear a faint buzz or hum through the speaker the first time you wire up a DAC and amplifier? That's not a bad chip — it's almost always a grounding issue or a power supply shared with something noisy. Root cause analysis from Chapter 1 applies here exactly the same as it did to a circuit: isolate one wire at a time until the hum disappears.

Before looking at the signal chain as a diagram, notice that each hop between these four stages changes the signal's form — digital data becomes a weak analog voltage, then a boosted analog voltage, then physical sound — which is exactly what the next diagram lets you click through.

Diagram: Audio Output Signal Chain¶

Run the Audio Output Signal Chain MicroSim fullscreen

Audio Output Signal Chain (graph data model)

Type: graph-data-model sim-id: audio-output-signal-chain
Library: vis-network
Status: Specified

Learning objective: Students will understand (Bloom L2: Understand) the order and role of each hardware stage that converts a digital audio signal into physical sound, and the form the signal takes at each stage.

Canvas: 700x350px, responsive — vis-network's built-in autoResize option combined with a window.addEventListener('resize', ...) call to network.redraw().

Layout: a left-to-right hierarchical vis-network graph with five nodes: "Raspberry Pi 5 (I2S source)," "I2S Audio Interface," "PCM5102 DAC," "Amplifier Module," "Speaker Output," connected by directional edges. Each edge is labeled with the signal type at that hop: "digital I2S data" (Pi 5 to interface), "digital I2S data" (interface to DAC), "analog voltage, weak" (DAC to amplifier), "analog voltage, boosted" (amplifier to speaker), "sound wave" (speaker to open air, drawn as a dashed edge to an unlabeled endpoint).

Controls: a single createButton()-equivalent HTML button labeled "Reset" that closes any open infobox (vis-network runs inside a plain <div>, so this can be a normal DOM button rather than a p5.js control).

Interaction: clicking any node opens an infobox below the diagram with that stage's one-sentence definition, matched to the surrounding chapter prose, plus one common troubleshooting tip (clicking "Amplifier Module" shows the grounding/hum tip from the text above). Hovering any edge shows its signal-type label as a tooltip, using vis-network's built-in title property on edges.

Implementation: vis-network with a hierarchical layout option set to left-to-right direction, fixed node positions disabled so the layout engine spaces nodes evenly. Use network.on("click", ...) to detect node clicks and network.on("hoverEdge", ...) for edge tooltips. Node data stored as a small array of objects with id, label, definition, and tip fields for easy infobox lookup.

The I2S DAC-and-amplifier chain above is the most capable way to make sound, but it's not the only one -- the poster below compares it against the passive and active piezo buzzers from Chapter 6 and a simple speaker driven directly by PWM.

Diagram: Sound Output Methods Compared¶

Run the Sound Output Methods Compared poster fullscreen

Sound Output Methods Compared (reused poster)

Type: interactive-infographic-poster
sim-id: sound-output
Status: Reused
Source: https://dmccreary.github.io/learning-micropython/posters/sound-output/
Source Repo: https://github.com/dmccreary/learning-micropython/tree/main/docs/posters/sound-output

Reused from the companion book Learning MicroPython's poster catalog. Places this chapter's I2S DAC-and-amplifier-and-speaker chain alongside the passive and active buzzers from Chapter 6 and a PWM-driven speaker, comparing wiring complexity and cost.

Privacy, On-Device Processing, and the Cloud-Versus-Edge Tradeoff¶

A microphone that's always listening raises a different kind of concern than a photoresistor or a distance sensor ever did, because sound can capture private conversations, not just a light level or a distance reading. Audio privacy considerations are the specific risks and responsibilities that come with any always-listening microphone project: what audio is recorded, whether it's stored or immediately discarded, who — or what software — can access it, and whether people nearby even know a microphone is active. These aren't abstract concerns. The same wake-word pipeline built in this chapter is a simplified version of the technology inside commercial smart speakers, and it deserves the same seriousness.

Berry's Serious Warning

Berry warning Take this one seriously: an always-listening microphone is a real privacy responsibility, not just a fun feature. Before running any wake-word or voice-command project around other people, tell them a microphone is active, keep audio processing local whenever the project allows it, and never record or store audio beyond what the project genuinely needs. This applies just as much to a classroom demo as it does to a finished product.

One design choice directly affects how much privacy risk a project carries. On-device processing means every step of the pipeline — capture, preprocessing, and inference — runs locally on the Pi 5 itself, with no audio ever leaving the device. The alternative is the broader cloud versus edge processing tradeoff: running inference locally on edge hardware like the Pi 5 and AI HAT+ (edge processing) versus streaming audio to a remote server for a larger, more powerful model to analyze (cloud processing). Cloud processing can offer higher accuracy from a bigger model, but it requires an internet connection, adds network latency, and means raw audio leaves the device — exactly the tradeoff Chapter 16 raised conceptually for AI accelerators, now applied specifically to a microphone.

The table below summarizes that tradeoff across the factors just discussed.

Factor	On-Device (Edge) Processing	Cloud Processing
Privacy	Audio never leaves the device	Audio is transmitted off-device
Internet required	No	Yes
Typical latency	Lower, no network round trip	Higher, network plus server time
Typical model size and accuracy	Smaller, optimized for the AI HAT+	Can be larger and more accurate
Works offline	Yes	No

Bringing It Together¶

This chapter turned a microphone into the same kind of sensor Chapter 1 first described, just running thousands of times faster: sense a physical quantity, think about what it means, act on the result. Every project from here forward — a wake-word robot, a smoke-alarm listener, a doorbell that only fires when it hears a knock and sees a person — leans on the same chain: analog signal, digitized samples, FFT, features, inference, confidence, and finally, a decision. Chapter 17's camera pipeline and this chapter's microphone pipeline together prove that the same sense-think-act pattern, running on the same AI HAT+ accelerator, scales cleanly across completely different senses.

You Unlocked a Superpower!

Berry celebrating That's berry impressive — you just gave a Raspberry Pi ears. From raw sound waves all the way to a wake word triggering a real action, you've got the full audio pipeline down, plus the judgment to run it responsibly. STEM is our superpower! See you in Chapter 19.

See Annotated References