#4709 Genetic Algorithms & Genetic Programming (GP)

#4709 Evolutionary Programming - Synth Patches

#4709 Describe a Python application to apply Genetic Algorithms or Programming to create synth patches

To automatically design synthesizer patches using evolutionary computing, you can build a system that searches the synth's parameter space to match a target sound or explore novel timbres.

Genetic Algorithms (GA) optimize fixed parameter vectors (e.g., knob values in a fixed synth architecture), while Genetic Programming (GP) can evolve entire synthesis topologies (e.g., modular signal trees).

Core Architecture

An evolutionary synth patch generator typically operates in a closed loop across four main stages:

[ Target Audio ] ──► ( Feature Extraction )
                             │
                             ▼
[ Population of Patches ] ──► [ Synth Engine (Render) ] ──► [ Loss / Fitness Eval ]
        ▲                                                          │
        └────────────── [ Selection & Mutation ] ◄─────────────────┘

1. Representation (Chromosome Encoding)

Genetic Algorithm (GA): A fixed-length float vector normalized between $0.0$ and $1.0$ mapping directly to synth parameters:
$\mathbf{x} = [\text{osc1\_type}, \text{cutoff}, \text{resonance}, \text{env\_attack}, \text{env\_decay}, \text{fx\_mix}, \dots]$
Genetic Programming (GP): An abstract syntax tree (AST) where leaf nodes are primitive generators (sine, saw, noise) and branch nodes are operations/processors (multiplication/AM, modulation/FM, low-pass filter, ADSR).

2. Sound Synthesis Backend

In-Python DSP: Libraries like scipy.signal, numpy, or pedalboard (Spotify) allow fast, vector-accelerated audio rendering.
External Synths / VSTs: dawdreamer or mido can host full VST3/AU plugins headlessly and render MIDI notes into raw audio buffers.

3. Fitness Function (Objective Metric)

To match a target audio file, extract acoustic features from both the target and candidate buffers:

Spectral Distance: Multi-resolution Short-Time Fourier Transform (STFT) error:
$\mathcal{L}_{\text{spectral}} = \frac{1}{T} \sum_{t} \left( \Vert{}\log \vert{}S_{\text{target}}\vert{} - \log \vert{}S_{\text{candidate}}\vert{} \Vert{}_1 + \Vert{}S_{\text{target}} - S_{\text{candidate}}\Vert{}_F \right)$
Perceptual Features: MFCCs (Mel-Frequency Cepstral Coefficients), spectral centroid, and spectral rolloff via librosa.
Novelty Search: If exploring new sounds rather than matching, fitness is scored by the distance in feature space from previous generations to encourage acoustic diversity.

Implementation Example (Python)

The following example uses DEAP (Distributed Evolutionary Algorithms in Python) and basic numpy synthesis to evolve a patch that matches a target frequency and filter profile:

import numpy as np
import librosa
from deap import base, creator, tools, algorithms

# 1. Toy Synthesizer Function
SAMPLE_RATE = 44100
DURATION = 1.0

def render_patch(params):
    freq, attack, decay, noise_mix = params
    t = np.linspace(0, DURATION, int(SAMPLE_RATE * DURATION), False)
    
    # Oscillator + Noise
    carrier = np.sin(2 * np.pi * (100 + freq * 800) * t)
    noise = np.random.normal(0, 0.2, len(t))
    signal = (1.0 - noise_mix) * carrier + (noise_mix * noise)
    
    # AD Envelope
    att_samples = int(max(0.01, attack) * 0.5 * SAMPLE_RATE)
    dec_samples = len(t) - att_samples
    env = np.concatenate([np.linspace(0, 1, att_samples), np.linspace(1, 0, dec_samples)])
    
    return signal * env

# Generate a synthetic "Target" sound
target_audio = render_patch([0.5, 0.2, 0.8, 0.1])
target_mfcc = librosa.feature.mfcc(y=target_audio, sr=SAMPLE_RATE, n_mfcc=13)

# 2. Genetic Algorithm Setup
creator.create("FitnessMin", base.Fitness, weights=(-1.0,))
creator.create("Individual", list, fitness=creator.FitnessMin)

toolbox = base.Toolbox()
toolbox.register("attr_float", np.random.uniform, 0.0, 1.0)
toolbox.register("individual", tools.initRepeat, creator.Individual, toolbox.attr_float, n=4)
toolbox.register("population", tools.initRepeat, list, toolbox.individual)

# 3. Fitness Evaluation
def evaluate(individual):
    candidate_audio = render_patch(individual)
    candidate_mfcc = librosa.feature.mfcc(y=candidate_audio, sr=SAMPLE_RATE, n_mfcc=13)
    loss = np.mean(np.abs(target_mfcc - candidate_mfcc))
    return (loss,)

toolbox.register("evaluate", evaluate)
toolbox.register("mate", tools.cxBlend, alpha=0.5)
toolbox.register("mutate", tools.mutGaussian, mu=0, sigma=0.1, indpb=0.2)
toolbox.register("select", tools.selTournament, tournsize=3)

# 4. Evolution Loop
population = toolbox.population(n=50)
algorithms.eaSimple(population, toolbox, cxpb=0.6, mutpb=0.3, ngen=40, verbose=True)

best = tools.selBest(population, 1)[0]
print(f"Evolved Patch: {best}, Fitness: {best.fitness.values[0]:.4f}")

Key Technical Challenges

Phase Invariance: Direct waveform mean-squared-error (MSE) fails because a slight phase shift destroys the loss even if the timbre matches. Always evaluate in the magnitude spectral or MFCC domain.
Non-Linear Parameter Spaces: Pitch and frequency controls are logarithmic, whereas toggles (e.g., waveform types) are discrete. Mapping requires hybrid mutation operators (e.g., bit-flips for discrete switches, Gaussian shifts for continuous knobs).
Render Bottleneck: Evaluating thousands of audio renders per generation is CPU-intensive. Use multiprocessing (multiprocessing.Pool or Ray) or GPU-accelerated DSP engines (like torch-audiomentations or JAX-based synthesis).

How does the fitness evaluation work?

Fitness evaluation works by measuring the perceptual and acoustic distance between the candidate sound rendered by the synth and the target reference sound.

A naive sample-by-sample subtraction ( $\Vert{}y_{\text{target}} - y_{\text{candidate}}\Vert{}^2$ ) fails completely in audio: two identical sounds with a tiny phase offset will sound identical to human ears, but have a massive time-domain error. Instead, fitness functions operate in the frequency, envelope, and perceptual feature domains.

The Evaluation Pipeline

[ Candidate Synth Parameters ] ──► [ Render Audio Buffer ]
                                              │
                                              ▼
 Target Audio Buffer ──────────────► [ Feature Extraction ] ◄── Candidate Audio Buffer
                                              │
                                              ├─► 1. Spectral Loss (STFT)
                                              ├─► 2. Perceptual Timbre (MFCCs)
                                              └─► 3. Temporal Envelope (RMS / Attack)
                                              │
                                              ▼
                                   [ Weighted Total Fitness ]

1. Multi-Resolution Spectral Loss

The core metric compares the magnitude spectrograms computed via Short-Time Fourier Transform (STFT) across multiple window sizes (e.g., $N = [512, 1024, 2048]$ samples). This captures both fine transient timing and overall frequency balance.

\mathcal{L}_{\text{spectral}} = \frac{1}{M} \sum_{m=1}^{M} \left( \mathcal{L}_{\text{mag}}^{(m)} + \mathcal{L}_{\text{log}}^{(m)} \right)

Linear Magnitude Loss ( $\mathcal{L}_{\text{mag}}$ ): Penalizes differences in loud, dominant harmonics using the Frobenius norm:
$\mathcal{L}_{\text{mag}} = \frac{\Vert{} \vert{}S_{\text{target}}\vert{} - \vert{}S_{\text{candidate}}\vert{} \Vert{}_F}{\Vert{} \vert{}S_{\text{target}}\vert{} \Vert{}_F}$
Log-Magnitude Loss ( $\mathcal{L}_{\text{log}}$ ): Penalizes differences in quiet harmonics, noise floors, and high frequencies:
$\mathcal{L}_{\text{log}} = \frac{1}{N} \Vert{} \log(\vert{}S_{\text{target}}\vert{} + \epsilon) - \log(\vert{}S_{\text{candidate}}\vert{} + \epsilon) \Vert{}_1$

2. Timbre & Brightness Descriptors

To guide the genetic search toward the correct instrument character before the exact harmonics align, scalar psychoacoustic features are compared:

Mel-Frequency Cepstral Coefficients (MFCCs): Compress the frequency spectrum onto the non-linear human auditory scale (the Mel scale). The Euclidean distance between candidate and target MFCC matrices captures overall vowel/timbre match.
Spectral Centroid: Represents the "brightness" or center of mass of the spectrum. If the target has a high cutoff and the candidate is muffled, this yields a large error gradient.
Spectral Rolloff & Flux: Measures how quickly high frequencies decay and how much the frequency spectrum changes over time.

3. Temporal Envelope Matching

A sound's dynamic character (plucked vs. pad vs. percussive) is evaluated in the time domain by smoothing the signal into its amplitude envelope:

\mathcal{L}_{\text{env}} = \frac{1}{T} \sum_{t=1}^T \vert{}\text{RMS}(y_{\text{target}}, t) - \text{RMS}(y_{\text{candidate}}, t)\vert{}

4. Composite Fitness Function (Code)

In Python, these metrics are normalized and combined into a single scalar minimization objective:

import numpy as np
import librosa

def compute_fitness(y_candidate, y_target, sr=44100):
    # 1. Multi-scale STFT loss
    stft_losses = []
    for n_fft in [512, 1024, 2048]:
        hop_length = n_fft // 4
        S_cand = np.abs(librosa.stft(y_candidate, n_fft=n_fft, hop_length=hop_length))
        S_targ = np.abs(librosa.stft(y_target, n_fft=n_fft, hop_length=hop_length))
        
        lin_loss = np.linalg.norm(S_targ - S_cand) / (np.linalg.norm(S_targ) + 1e-7)
        log_loss = np.mean(np.abs(np.log(S_targ + 1e-4) - np.log(S_cand + 1e-4)))
        stft_losses.append(lin_loss + log_loss)
    
    spectral_loss = np.mean(stft_losses)

    # 2. MFCC Timbre Loss
    mfcc_cand = librosa.feature.mfcc(y=y_candidate, sr=sr, n_mfcc=13)
    mfcc_targ = librosa.feature.mfcc(y=y_target, sr=sr, n_mfcc=13)
    mfcc_loss = np.mean(np.abs(mfcc_targ - mfcc_cand))

    # 3. RMS Envelope Loss
    rms_cand = librosa.feature.rms(y=y_candidate)
    rms_targ = librosa.feature.rms(y=y_target)
    env_loss = np.mean(np.abs(rms_targ - rms_cand))

    # Composite weighted fitness (lower is better)
    total_loss = (0.5 * spectral_loss) + (0.3 * mfcc_loss) + (0.2 * env_loss)
    return total_loss

From <https://gemini.google.com/app/a504a2e190d5b187> Google Gemini (3.7 Flash)

Search This Blog

Learning from ChatGPT