Cookbook: Common Patterns

This cookbook provides practical solutions to common problems and patterns when using NEAT-Python. Each recipe includes working code you can copy and adapt for your own projects.

How to: Set Specific Output Activation Functions

Problem: You need network outputs in a specific range (e.g., [-1, 1] for control problems).

Solution: Configure the activation function in your config file:

[DefaultGenome]
# For outputs in range [-1, 1]
activation_default = tanh
activation_mutate_rate = 0.0  # Don't change activation
activation_options = tanh     # Only allow tanh

Alternative: If you need to mix activation functions:

# Allow multiple options, set mutation rate > 0
activation_default = tanh
activation_mutate_rate = 0.2
activation_options = tanh sigmoid relu

Post-processing approach:

import math

# Get network output
output = net.activate(inputs)

# Transform to desired range
output_tanh = [math.tanh(x) for x in output]  # [-1, 1]
output_sigmoid = [1.0 / (1.0 + math.exp(-x)) for x in output]  # [0, 1]

Warning

Make sure your fitness function expects the same range as your activation function outputs!

See also: Overview of builtin activation functions for all available activation functions.

How to: Use Parallel Evaluation

Problem: Fitness evaluation is slow and you want to use multiple CPU cores.

Solution: Use ParallelEvaluator with a context manager for automatic cleanup:

import neat
import multiprocessing

def eval_genome(genome, config):
    """
    Evaluate a single genome.
    IMPORTANT: Must return fitness value (not set genome.fitness).
    """
    net = neat.nn.FeedForwardNetwork.create(genome, config)

    # Your fitness evaluation here
    fitness = 0.0
    for test_case in test_cases:
        output = net.activate(test_case.inputs)
        fitness += calculate_score(output, test_case.expected)

    return fitness  # Return, don't set genome.fitness

# Use context manager for proper cleanup
config = neat.Config(neat.DefaultGenome, neat.DefaultReproduction,
                     neat.DefaultSpeciesSet, neat.DefaultStagnation,
                     'config-file')

p = neat.Population(config)
p.add_reporter(neat.StdOutReporter(True))

with neat.ParallelEvaluator(multiprocessing.cpu_count(), eval_genome) as evaluator:
    winner = p.run(evaluator.evaluate, 300)
# Pool automatically cleaned up here

Common mistakes:

Warning

❌ Wrong: Setting genome.fitness in eval_genome

def eval_genome(genome, config):
    genome.fitness = 10.0  # Don't do this!

✅ Right: Returning fitness value

def eval_genome(genome, config):
    return 10.0  # Return the value

Warning

❌ Wrong: Forgetting to use context manager (memory leak)

evaluator = neat.ParallelEvaluator(4, eval_genome)
winner = p.run(evaluator.evaluate, 300)
# Pool never cleaned up!

✅ Right: Using with statement

with neat.ParallelEvaluator(4, eval_genome) as evaluator:
    winner = p.run(evaluator.evaluate, 300)

Performance tip: Start with multiprocessing.cpu_count() and adjust based on your workload.

See also: Module summaries for ParallelEvaluator API details.

How to: Use GPU-Accelerated Evaluation

Problem: CTRNN or Izhikevich spiking network evaluation is too slow, even with multiple CPU cores.

Solution: Use the GPU evaluators in neat.gpu to batch-evaluate the entire population on GPU. This requires CuPy: pip install 'neat-python[gpu]'

CTRNN example:

import math
from neat.gpu.evaluator import GPUCTRNNEvaluator

def input_fn(t, dt):
    """Return input signal at time t. Shape: [num_inputs]."""
    return [math.sin(2 * math.pi * t), math.cos(2 * math.pi * t)]

def fitness_fn(output_trajectory):
    """Compute fitness from output trajectory.

    Args:
        output_trajectory: numpy array of shape [num_steps, num_outputs]
    Returns:
        Scalar fitness value.
    """
    # Example: reward output that tracks a target
    return -float(np.mean((output_trajectory[:, 0] - target) ** 2))

evaluator = GPUCTRNNEvaluator(
    dt=0.01,       # integration timestep (seconds)
    t_max=1.0,     # total simulation time (seconds)
    input_fn=input_fn,
    fitness_fn=fitness_fn,
)
winner = population.run(evaluator.evaluate, n=300)

Izhikevich spiking network example:

from neat.gpu.evaluator import GPUIZNNEvaluator

def input_fn(t, dt):
    """Return input values at time t. Shape: [num_inputs]."""
    return [1.0, 0.5]

def fitness_fn(output_trajectory):
    """Compute fitness from spike train.

    Args:
        output_trajectory: numpy array of shape [num_steps, num_outputs],
                           values are 0.0 (no spike) or 1.0 (spike).
    """
    return float(np.sum(output_trajectory))

evaluator = GPUIZNNEvaluator(
    dt=0.05,       # integration timestep (milliseconds)
    t_max=50.0,    # total simulation time (milliseconds)
    input_fn=input_fn,
    fitness_fn=fitness_fn,
)
winner = population.run(evaluator.evaluate, n=300)

Key differences from ParallelEvaluator:

  • The GPU evaluator handles network creation, simulation, and fitness assignment internally. You provide input_fn (what to feed the network) and fitness_fn (how to score the output).

  • ParallelEvaluator takes an eval_genome function that returns a scalar fitness. GPU evaluators take separate input_fn and fitness_fn callables.

  • input_fn runs on CPU; the simulation runs on GPU; fitness_fn runs on CPU per genome.

Constraints:

  • Only sum aggregation is supported (required for batched matrix-vector multiply).

  • Supported activation functions: sigmoid, tanh, relu, identity, clamped, elu, softplus, sin, gauss, abs, square. Unsupported functions raise ValueError at evaluation time.

  • import neat does not load CuPy. CuPy is imported lazily when a GPU evaluator is created.

See also: Continuous-time recurrent neural network implementation for CTRNN details including the integration method.

How to: Save and Restore Checkpoints

Problem: You want to save evolution progress and resume later.

Solution: Use the Checkpointer reporter:

import neat

config = neat.Config(neat.DefaultGenome, neat.DefaultReproduction,
                     neat.DefaultSpeciesSet, neat.DefaultStagnation,
                     'config-file')

p = neat.Population(config)
p.add_reporter(neat.StdOutReporter(True))

# Save checkpoint every 5 generations
checkpointer = neat.Checkpointer(generation_interval=5,
                                  time_interval_seconds=None,
                                  filename_prefix='neat-checkpoint-')
p.add_reporter(checkpointer)

# Run evolution
winner = p.run(eval_genomes, 100)

This creates files: neat-checkpoint-0, neat-checkpoint-5, neat-checkpoint-10, etc.

Restoring from checkpoint:

import neat

# Restore from generation 50
p = neat.Checkpointer.restore_checkpoint('neat-checkpoint-50')

# Continue evolution for 50 more generations
winner = p.run(eval_genomes, 50)

Time-based checkpointing:

# Save every 10 minutes instead of every N generations
checkpointer = neat.Checkpointer(generation_interval=None,
                                  time_interval_seconds=600)

Note

Checkpoint compatibility: Checkpoints from v1.0+ are not compatible with v0.x due to innovation number tracking. See Migration Guide for details.

See also: neat.Checkpointer API documentation.

How to: Debug “Population Not Evolving”

Problem: Fitness isn’t improving over many generations.

Diagnostic steps:

1. Check fitness function is working

def eval_genomes(genomes, config):
    for genome_id, genome in genomes:
        net = neat.nn.FeedForwardNetwork.create(genome, config)
        genome.fitness = calculate_fitness(net)

        # Debug: Print fitness values
        if genome_id % 10 == 0:  # Print every 10th genome
            print(f"Genome {genome_id}: fitness = {genome.fitness}")

Check for these issues:

  • ✅ All genomes have valid fitness (not None)

  • ✅ Fitness values vary between genomes

  • ✅ Better performance = higher fitness

2. Check for sufficient diversity

[NEAT]
pop_size = 150  # Try increasing (e.g., 300)

[DefaultSpeciesSet]
compatibility_threshold = 3.0  # Try decreasing (e.g., 2.0)

Lower compatibility threshold = more species = more diversity.

3. Check activation functions

[DefaultGenome]
# If problem needs outputs in [-1, 1], use tanh not sigmoid
activation_default = tanh  # Not sigmoid!

# Allow evolution to try different activations
activation_mutate_rate = 0.1
activation_options = tanh sigmoid relu

4. Add diagnostic reporters

stats = neat.StatisticsReporter()
p.add_reporter(stats)
p.add_reporter(neat.StdOutReporter(True))

winner = p.run(eval_genomes, 300)

# Visualize fitness over time
import visualize  # From examples/xor/visualize.py
visualize.plot_stats(stats, ylog=False, view=True)

See also: Troubleshooting Guide for more diagnostic techniques.

How to: Control Network Complexity

Problem: Networks are growing too large (hundreds of nodes/connections).

Solution 1: Adjust mutation rates

[DefaultGenome]
# Reduce addition rates
conn_add_prob = 0.3      # Default: 0.5
node_add_prob = 0.1      # Default: 0.2

# Increase deletion rates
conn_delete_prob = 0.7   # Default: 0.5
node_delete_prob = 0.4   # Default: 0.2

Solution 2: Add complexity penalty to fitness

def eval_genomes(genomes, config):
    for genome_id, genome in genomes:
        net = neat.nn.FeedForwardNetwork.create(genome, config)

        # Calculate task performance
        task_fitness = evaluate_task(net)

        # Penalize complexity
        num_connections = len(genome.connections)
        num_nodes = len([n for n in genome.nodes.values()
                        if n.key not in config.genome_config.input_keys])

        complexity_penalty = 0.01 * (num_connections + num_nodes)

        genome.fitness = task_fitness - complexity_penalty

Adjust penalty weight (0.01) based on your needs: - Larger penalty (0.1) = strong preference for small networks - Smaller penalty (0.001) = weak preference, allow larger networks

Solution 3: Start with hidden nodes

[DefaultGenome]
num_hidden = 2  # Start with 2 hidden nodes
initial_connection = full  # Fully connected

This can help find solutions faster without excessive growth.

See also: Configuration file description for all mutation parameters.

How to: Handle Different Output Ranges

Problem: Your problem requires specific output ranges.

Common ranges and solutions:

Output Range

Activation Function

Use Case

[0, 1]

sigmoid

Probabilities, binary classification

[-1, 1]

tanh

Control signals, normalized values

[0, ∞)

relu, softplus

Non-negative values, quantities

Any range

identity + scaling

Custom ranges

Example: Control problem needing [-1, 1]

[DefaultGenome]
activation_default = tanh
activation_mutate_rate = 0.0
activation_options = tanh

Example: Multi-output with different ranges

def eval_genomes(genomes, config):
    for genome_id, genome in genomes:
        net = neat.nn.FeedForwardNetwork.create(genome, config)
        raw_output = net.activate(inputs)

        # Assume 3 outputs: probability, control, quantity
        probability = sigmoid(raw_output[0])  # [0, 1]
        control = tanh(raw_output[1])          # [-1, 1]
        quantity = relu(raw_output[2])         # [0, ∞)

        genome.fitness = evaluate(probability, control, quantity)

Example: Custom range scaling

# Want outputs in [5, 15]
raw_output = net.activate(inputs)[0]  # From sigmoid: [0, 1]
scaled_output = 5.0 + raw_output * 10.0  # Scale to [5, 15]

See also: Overview of builtin activation functions for all available activation functions and their ranges.

How to: Configure for Different Problem Types

Feedforward vs. Recurrent:

Use feedforward when: - Problem has no temporal dependencies - Each input → output is independent - Examples: XOR, classification, function approximation

[DefaultGenome]
feed_forward = True
initial_connection = full

Use recurrent when: - Problem requires memory of past inputs - Time-series or sequential data - Examples: control, game playing, sequence prediction

[DefaultGenome]
feed_forward = False
initial_connection = partial  # Or full

Simple vs. Complex problems:

Simple problems (like XOR):

[NEAT]
pop_size = 50                # Smaller population
fitness_threshold = 3.9      # Clear target

[DefaultGenome]
num_hidden = 0               # Start minimal
conn_add_prob = 0.5
node_add_prob = 0.2

Complex problems (like game playing):

[NEAT]
pop_size = 300               # Larger population
no_fitness_termination = False  # May not reach threshold

[DefaultGenome]
num_hidden = 2               # Start with some complexity
conn_add_prob = 0.7          # Allow faster growth
node_add_prob = 0.3

Fast exploration vs. thorough search:

Fast exploration (quick results):

[NEAT]
pop_size = 50

[DefaultGenome]
conn_add_prob = 0.7          # Aggressive complexification
node_add_prob = 0.3

[DefaultSpeciesSet]
compatibility_threshold = 4.0  # Fewer species

Thorough search (best solution):

[NEAT]
pop_size = 300

[DefaultGenome]
conn_add_prob = 0.4          # Slower complexification
node_add_prob = 0.15

[DefaultSpeciesSet]
compatibility_threshold = 2.5  # More species

See also: - Configuration Essentials for parameter explanations - NEAT Overview for algorithm understanding - XOR Example: Detailed Walkthrough for a complete simple example

Common Gotchas

1. Forgetting to set genome.fitness

This is the #1 mistake! NEAT can’t evolve without fitness values.

# ❌ Wrong
def eval_genomes(genomes, config):
    for genome_id, genome in genomes:
        net = neat.nn.FeedForwardNetwork.create(genome, config)
        result = net.activate([1, 0])
        # Forgot to set fitness!

# ✅ Right
def eval_genomes(genomes, config):
    for genome_id, genome in genomes:
        net = neat.nn.FeedForwardNetwork.create(genome, config)
        result = net.activate([1, 0])
        genome.fitness = calculate_fitness(result)

2. Negative fitness values causing extinction

If all genomes have fitness ≤ 0, species can go extinct.

# ✅ Ensure fitness is positive
genome.fitness = max(0.001, calculated_fitness)

3. Config file in wrong location

# ❌ Relative path may fail
config = neat.Config(..., 'config-file')

# ✅ Use absolute path
import os
local_dir = os.path.dirname(__file__)
config_path = os.path.join(local_dir, 'config-file')
config = neat.Config(..., config_path)

4. Not using context managers with ParallelEvaluator

Always use with statement to ensure proper cleanup:

# ✅ Right
with neat.ParallelEvaluator(4, eval_genome) as evaluator:
    winner = p.run(evaluator.evaluate, 300)

5. Mixing up node keys

  • Input nodes: negative keys (-1, -2, …)

  • Output nodes: zero and positive keys (0, 1, …)

  • Hidden nodes: positive keys assigned during evolution

Next Steps