Hyperparameter Learning Tutorial

GA performance depends heavily on hyperparameters like mutation rate, crossover probability, and population size. This tutorial demonstrates online Bayesian learning of hyperparameters during evolution.

The Problem

Traditional approach: Set parameters once, hope they work.

Better approach: Learn optimal parameters from feedback during evolution.

Hyperparameter Control Methods

Fugue-evo supports several approaches (following Eiben et al.'s classification):

This tutorial focuses on Bayesian learning with conjugate priors.

Complete Example

//! Online Operator-Parameter Learning with a Thompson-Sampling Bandit
//!
//! This example demonstrates the *wired-in* hyperparameter learner: a
//! [`ThompsonSamplingTuner`] driven directly by [`SimpleGA::run_adaptive`].
//!
//! Each generation the GA Thompson-samples a per-gene mutation probability and a
//! whole-genome crossover probability from the tuner, applies those arm values to
//! its operators, and credits each arm with the observed parent-vs-offspring
//! improvement events. The arm's Beta draw is used *only* to pick the arm — it is
//! never returned as the parameter value itself (the bug the old learner had).
//!
//! Run with:
//! ```text
//! cargo run --example hyperparameter_learning
//! ```

use fugue_evo::prelude::*;
use rand::rngs::StdRng;
use rand::SeedableRng;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    println!("=== Thompson-Sampling Operator-Parameter Learning ===\n");

    let mut rng = StdRng::seed_from_u64(42);

    const DIM: usize = 20;
    let bounds = MultiBounds::symmetric(5.12, DIM);

    println!("Problem: {DIM}-D Rastrigin");
    println!("Learner: Thompson-sampling bandit over mutation- and crossover-probability arms\n");

    // Configure the bandit: candidate values ("arms") for each tunable parameter.
    // Each arm holds a Beta posterior over P(offspring improves on parents | arm).
    let config = ThompsonConfig {
        mutation_rate_arms: vec![0.01, 0.05, 0.1, 0.2, 0.4],
        crossover_prob_arms: vec![0.5, 0.7, 0.9],
        prior: BetaPosterior::uniform(),
        record_history: true,
    };

    println!("Mutation-rate arms:  {:?}", config.mutation_rate_arms);
    println!("Crossover-prob arms: {:?}\n", config.crossover_prob_arms);

    // Build a GA that consults the tuner every generation. `real_valued()` fixes
    // the genome/fitness types (no turbofish); we override the mutation operator
    // with a tunable Gaussian mutation whose per-gene probability the bandit sets.
    let mut ga = SimpleGABuilder::real_valued()
        .mutation(GaussianMutation::new(0.1))
        .population_size(100)
        .bounds(bounds)
        .fitness(Rastrigin::new(DIM))
        .max_generations(200)
        .adaptive_operators(config)
        .build()?;

    let result = ga.run_adaptive(&mut rng)?;

    // --- Posterior evolution ---------------------------------------------------
    let tuner = ga.tuner().expect("tuner is present after run_adaptive");
    let mr = tuner
        .parameter(PARAM_MUTATION_RATE)
        .expect("mutation-rate parameter");
    let arm_values = mr.values();

    println!("Posterior evolution — P(improvement) mean per mutation-rate arm:\n");
    print!("  {:>4}", "gen");
    for v in &arm_values {
        print!("   p={v:<5.2}");
    }
    println!("   selected");

    for snap in tuner.history().iter().step_by(20) {
        // Find this parameter's entry in the snapshot.
        if let Some((_, selected, means)) = snap
            .parameters
            .iter()
            .find(|(name, _, _)| name == PARAM_MUTATION_RATE)
        {
            print!("  {:>4}", snap.generation);
            for m in means {
                print!("   {m:>6.3}");
            }
            match selected {
                Some(v) => println!("   {v:.2}"),
                None => println!("   -"),
            }
        }
    }

    // --- Learned parameters ----------------------------------------------------
    println!("\n=== Learned operator parameters ===");
    for param in tuner.parameters() {
        let best = param.best_value();
        println!("\nParameter '{}':", param.name);
        for arm in param.arms() {
            let flag = if (arm.value - best).abs() < 1e-12 {
                " <-- best"
            } else {
                ""
            };
            println!(
                "  value {:<5.2}  P(improve)~{:.3}  pulls={}{}",
                arm.value,
                arm.posterior.mean(),
                arm.selections,
                flag
            );
        }
        println!("  => favored value: {best:.2}");
    }
    println!(
        "\nTotal improvement events fed back to the tuner: {}",
        tuner.total_observations()
    );

    println!("\n=== Result ===");
    println!("Best fitness (adaptive): {:.6}", result.best_fitness);

    // --- Comparison with fixed mutation rates ---------------------------------
    println!("\n--- Comparison with fixed mutation rates ---\n");
    for fixed_rate in [0.05, 0.1, 0.2, 0.5] {
        let best = run_with_fixed_rate(fixed_rate, DIM)?;
        println!("Fixed rate {fixed_rate:.2}: best = {best:.6}");
    }
    println!(
        "\nAdaptive (favored {:.2}): best = {:.6}",
        mr.best_value(),
        result.best_fitness
    );

    Ok(())
}

/// Run a non-adaptive GA with a fixed per-gene mutation probability for comparison.
fn run_with_fixed_rate(rate: f64, dim: usize) -> Result<f64, Box<dyn std::error::Error>> {
    let mut rng = StdRng::seed_from_u64(42); // Same seed for a fair comparison.
    let bounds = MultiBounds::symmetric(5.12, dim);

    let result = SimpleGABuilder::real_valued()
        .mutation(GaussianMutation::new(0.1).with_probability(rate))
        .population_size(100)
        .bounds(bounds)
        .fitness(Rastrigin::new(dim))
        .max_generations(200)
        .build()?
        .run(&mut rng)?;

    Ok(result.best_fitness)
}

Source: examples/hyperparameter_learning.rs

Running the Example

cargo run --example hyperparameter_learning

Key Components

Beta Posterior for Mutation Rate

// Prior: Beta(2, 2) centered around 0.5
let mut mutation_posterior = BetaPosterior::new(2.0, 2.0);

The Beta distribution is perfect for learning probabilities:

• Domain: [0, 1] (valid probability range)
• Conjugate to Bernoulli outcomes (success/failure)
• Prior parameters encode initial beliefs

Beta(2, 2):

• Mean = 0.5 (start uncertain)
• Moderate confidence (equivalent to 4 observations)

Observing Outcomes

// Check if mutation improved fitness
let improved = child_fitness > parent_fitness;

// Update posterior with observation
mutation_posterior.observe(improved);

Each observation updates the distribution:

• Success (improvement): Increases mean
• Failure: Decreases mean
• More observations → narrower distribution

Sampling Parameters

// Sample mutation rate from current posterior
current_mutation_rate = mutation_posterior.sample(&mut rng);

Thompson Sampling: Sample from posterior, use as parameter.

• Balances exploration (uncertainty) and exploitation (best estimate)
• Naturally adapts as confidence grows

Adaptation Interval

if gen % adaptation_interval == 0 {
    current_mutation_rate = mutation_posterior.sample(&mut rng);
}

Don't update every generation:

• Too frequent: Not enough signal
• Too rare: Slow adaptation
• Typical: Every 10-50 generations

Understanding the Output

Initial mutation rate (prior mean): 0.5000

Gen  20: Sampled mutation rate = 0.4123 (posterior mean = 0.3876)
Gen  40: Sampled mutation rate = 0.3456 (posterior mean = 0.3245)
...

=== Results ===
Learned hyperparameters:
  Final mutation rate (posterior mean): 0.2134
  95% credible interval: [0.1823, 0.2445]

Mutation statistics:
  Total mutations: 20000
  Successful mutations: 4268
  Observed success rate: 0.2134

The posterior converges toward the empirically optimal rate.

Credible Intervals

let ci = mutation_posterior.credible_interval(0.95);
println!("95% CI: [{:.4}, {:.4}]", ci.0, ci.1);

Unlike frequentist confidence intervals, Bayesian credible intervals have a direct interpretation: "95% probability the true value is in this range (given our data)."

Comparing with Fixed Rates

for fixed_rate in [0.05, 0.1, 0.2, 0.5] {
    let result = run_with_fixed_rate(fixed_rate)?;
    println!("Fixed rate {:.2}: Best = {:.6}", fixed_rate, result);
}

The learned rate often outperforms any single fixed rate because:

1. It adapts to the problem
2. It can change as evolution progresses
3. It handles different phases (exploration vs. exploitation)

Other Learnable Parameters

Crossover Probability

let mut crossover_posterior = BetaPosterior::new(2.0, 2.0);

// Observe: did crossover produce better offspring than parents?
let offspring_better = child_fitness > max(parent1_fitness, parent2_fitness);
crossover_posterior.observe(offspring_better);

Tournament Size

Use a categorical posterior for discrete choices:

let tournament_sizes = [2, 3, 5, 7];
let mut size_weights = vec![1.0; tournament_sizes.len()];

// Update weights based on selection quality
// ... observe which sizes produce better offspring

Multiple Parameters

Learn multiple parameters simultaneously:

struct AdaptiveGA {
    mutation_posterior: BetaPosterior,
    crossover_posterior: BetaPosterior,
    // ... other parameters
}

impl AdaptiveGA {
    fn adapt(&mut self, gen: usize, rng: &mut Rng) {
        if gen % interval == 0 {
            self.mutation_rate = self.mutation_posterior.sample(rng);
            self.crossover_prob = self.crossover_posterior.sample(rng);
        }
    }
}

Deterministic Schedules

For simpler adaptation, use time-based schedules:

use fugue_evo::hyperparameter::schedules::*;

// Linear decay: 0.5 → 0.05 over 500 generations
let schedule = LinearSchedule::new(0.5, 0.05, 500);
let rate = schedule.value_at(gen);

// Exponential decay
let schedule = ExponentialSchedule::new(0.5, 0.99, 500);

// Sigmoid decay
let schedule = SigmoidSchedule::new(0.5, 0.05, 500);

When to Use Which Method

Exercises

1. Prior sensitivity: Try Beta(1,1), Beta(5,5), Beta(10,1) priors
2. Learning speed: Vary adaptation interval (5, 20, 50, 100)
3. Multiple parameters: Learn both mutation and crossover rates

Next Steps

• Custom Operators - Create learnable custom operators
• Advanced Algorithms - Built-in adaptive algorithms