Module 6

Digital audio

You don't need a DSP background, but you do need an accurate mental model of what a "buffer" is, because half of Spoke's plumbing moves them around.

Sound to numbers

Sound is air pressure varying over time. A microphone converts that to a varying voltage; an analog-to-digital converter measures that voltage many thousands of times per second. Each measurement is a sample.

Term	Meaning	Typical value
Sample rate	Measurements per second	48,000 Hz (Mac mics), 16,000 Hz (speech models)
Bit depth / format	Precision of each measurement	32-bit float on Apple platforms
Channels	Independent streams	1 (mono) for speech, 2 for stereo
Frame	One sample across all channels	—
Buffer	A chunk of consecutive frames	4,096 frames ≈ 85 ms at 48 kHz

Audio is processed in buffers rather than sample-by-sample purely for efficiency — a callback per sample at 48 kHz would be 48,000 function calls per second. In Spoke, bufferSize: 4096 means roughly 85 ms of audio arrives per callback. That's a real latency floor: no matter how fast everything downstream is, your first transcription cannot begin sooner than the first buffer.

Why 16 kHz for speech: the Nyquist theorem says you can represent frequencies up to half the sample rate. Human speech intelligibility lives almost entirely below 8 kHz, so 16 kHz captures everything that matters while carrying a third of the data. Speech models are trained at this rate, which is why conversion is mandatory rather than optional.

AVAudioEngine

AVAudioEngine is a graph of audio nodes — sources, effects, mixers, outputs — connected together. Spoke uses the simplest possible configuration: just the input node, with a tap on it.

let input = engine.inputNode
let format = input.outputFormat(forBus: 0)

input.installTap(onBus: 0, bufferSize: 4096, format: format) { buffer, time in
    // called ~12×/second on a real-time audio thread
}

engine.prepare()
try engine.start()

A tap is a non-destructive observer on a node — audio still flows normally, you just get a copy. prepare() pre-allocates resources so start() doesn't glitch.

Trap · silent capture failure

Audio capture fails quietly in more than one way, and none of them throw. A sample rate of zero means there's no usable input device or the node isn't configured. Separately, if microphone permission was denied, the engine typically starts fine and delivers buffers full of zeroes — perfectly valid silence, forever, with no error anywhere.

So check both things, and check them separately. Use AVCaptureDevice.authorizationStatus(for: .audio) for permission — never infer permission from the audio format. AudioCapture's sample-rate guard catches the missing-device case; DictationController handles permission explicitly before capture ever starts.

Trap · the real-time audio thread

The tap callback runs on a thread with hard deadlines. Miss one and the user hears a click. Inside that closure: never allocate memory, take locks, do file or network I/O, or touch UI. Spoke does the absolute minimum — computes an RMS value and hands the buffer off — then hops to other threads with Task.

Format conversion

The mic produces one format; the speech model wants another. AVAudioConverter bridges them, handling sample-rate conversion, channel mixing, and bit-depth changes.

let converter = AVAudioConverter(from: inputFormat, to: targetFormat)
converter?.primeMethod = .none   // don't clip the start of speech

Two subtleties that cause silent, confusing failures:

Output capacity must account for the sample-rate ratio. Converting 48 kHz to 16 kHz produces a third as many frames; going the other way produces three times as many. Allocate an output buffer that's too small and the converter quietly truncates your audio. BufferConverter computes ratio = targetRate / inputRate and adds headroom.

Rebuild the converter when the input format changes. If the user switches from the built-in mic to AirPods mid-session, the format changes underneath you. Caching a converter forever means it starts producing garbage. That's why BufferConverter tracks lastInputFormat.

Measuring level: RMS vs peak

The waveform in the overlay needs a single number per buffer representing loudness.

Peak is the largest absolute sample value. It's easy but jumpy — a single consonant spike sends it to full scale, and the meter looks like a strobe light.

RMS (root mean square) squares every sample, averages, then takes the square root. Squaring makes negatives positive, and the average smooths out spikes. It correlates much better with perceived loudness.

var sumOfSquares: Float = 0
for index in 0..<frameLength {
    let sample = samples[index]
    sumOfSquares += sample * sample
}
let rms = sqrt(sumOfSquares / Float(frameLength))

One more adjustment matters for the visuals. Speech RMS typically sits between 0.01 and 0.2 on a 0–1 scale, so a linear meter barely moves. Spoke multiplies by 6 and clamps, then applies a square root curve in the view. That's not cheating — human loudness perception is roughly logarithmic, so a curved mapping is genuinely more honest than a linear one.